Here's the thing, Brutus is a tricky character for a dramatist because he's a Stoic: his philosophy of life says he should never make decisions based on emotions. That's a problem for the dramatist because... that's boring. Every storyteller knows that decisions based on emotion are at the very heart of good story. So, does Shakespeare, one of the greatest storytellers of all time, kill off his main character's wife... for no reason?
Brutus' response to Portia's death is so fun to watch because we get to see it twice, in two totally different contexts. First, we see his response when he tells his best friend about it, and then we see his response when he "learns" about it from his subordinate (his employee). Which do you think is going to be the more honest response?
So here's this guy who has huge decisions to make (involving not only his own life and death, but also the success or failure of the entire Roman Empire, which he loves), and suddenly his wife kills herself. Is he upset? OF COURSE HE'S UPSET!!! But he deeply believes that he shouldn't let his emotions affect these huge decisions. Do they affect his decisions? OF COURSE THEY AFFECT HIS DECISIONS!!! But he can't show it. Now, look at this big argument he has with Cassius in Act 4 just before he reveals that Portia has killed herself. Is he entirely logical in that argument? No, he's totally off his rocker, actually, and once he and Cassius calm down, he admits to Cassius -- IN PRIVATE -- that he's upset because of Portia. The craziness of his argument up to that point is your main evidence for the assertion that Portia's death had a huge effect on him.
The wonderful thing about this scene, though, is how he responds to her death IN PUBLIC. We know now that he was all wigged out about her death, but he lets his general, Mesalus, believe that he knows nothing about her death. Mesalus delivers the awful news, but we, of course, know that Brutus is prepared for it (dramatic irony). Brutus responds exactly as a stoic leader with huge responsibilities should respond: calmly. And Mesalus is suitably impressed -- "Even so great men great losses should endure." It's easy to be tricked, as Mesalus is, into believing that Brutus is unaffected. Don't be.