No matter what decision Brutus makes, he's going to be turning his back on somebody. Brutus believes that disposing of Caesar is the best decision for Rome, so if he kills Caesar, he has betrayed the trust and friendship of one of his good buddies. However, if he allows Caesar to live and be king, all of the pride and loyalty Brutus feels in being a Roman would cease. Brutus believes that Rome, as they know it, would be changed so drastically for the worse that action must be taken.
There are further instances of loyalty in his devotion to his wife, Portia. Although Portia makes a pretty good argument for knowing the truth (when's the last time you stabbed yourself in persuasion??), Brutus is smart enough to keep her out of the plan lest she be targeted in retribution.