This is a good question. Think about this: all the Romans in the Senate are noblemen. They together discuss the issues presented to them and together decide the best course of action. As members of the Senate, they are respected and regarded with reverence, but none of them is considered any higher in rank or power than the next...supposedly.
Then, along comes Caesar. The common people (the majority of Rome's population) love him. They want him to be Emperor--the single-most powerful ruler of Rome--which would make the Senate all but unnecessary.
So, what do Flavius, Cassius, and other noble Romans and members of the Senate fear and resent? They are jealous of his popularity and of the possible power which may be bestowed upon him by the common people...the people who outnumber the nobles almost 3 to 1. They are afraid that they will be unnecessary and unimportant. They resent that Caesar has been ranked higher than they are...and so they try to tear him down talking of his weaknesses in spirit (he calls for help when he is afraid of drowning), in body (he has the "falling down sickness"), and in character (he wants to be the all-powerful ruler...he is ambitious beyond what a true Roman with only the best for Rome at heart would be). They conspire to kill him and be down with him once an for all, in order to preserve their own position in government.