Excellent question! To answer it you need to consider the definition of a tragedy that Artistotle provides us with and try to relate that to the downfall of the two central characters, Caesar and Brutus, to think about which character has the more "tragic" downfall.
Aristotle said a tragedy had to feature a man who would suffer a tragic reversal of fortunes because of a fault or a series of bad decisions that resulted from a deficiency of character. A tragedy could not feature a completely evil character, but needs to focus on the life of a normal human with failings that we can relate to.
Thinking about the play then, both Caesar and Brutus can be considered tragic figures in this sense. Caesar seems to be assassinated because of his desire to gain more power and his ambition. Brutus, on the other hand, suffers his tragic end partly because of his sense of honour and naivety - he ignores the advice of the more savvy Cassius and has a mistaken sense of honour that perhaps blinds him to his own ambition and envy. When considering how the label of "tragedy" applies to both characters we do need to remember that we know a lot more about Brutus than we do about Caesar - we need to infer a lot about his character, whereas because Brutus lives until the end of the play we have far more insights into his character.