It is difficult to find specific details of Caesar's health, though his political prowess is well substantiated.
One source reports that Caesar was of good health, but suffered from epileptic fits. Another source reports:
...he was a slightly built man, had a soft and white skin, suffered from headaches and was subject to epileptic fits.
It would appear that even with the few items above, not much is known of Caesar's early life. It is known that he took great care of his men. He never asked them to do things he would not do himself. He slept often in chariots, out in the open. These things seemed to improve his health; he was not a leader who need to be pampered, nor wanted it.
Of course, in Shakespeare's play, Julius Caesar, there is mention by Caesar of being deaf. Two things we know about Shakespeare. He was not afraid to change history or make things up for the sake of a plot. One source suggested deafness would have been a good excuse for not knowing of the conspiracy to kill him, but the truth seems to be that he was very aware of the dangers that surrounded him; however, he chose to live his life out in the open instead of hiding away. This is in keeping with the battles he fought, and the dedication to his men. He was a brilliant strategist and a strong soldier.
The other thing we know about Shakespeare is that he used the English language with an ease, dexterity and flare that is still admired today. It is quite possible that Shakespeare was manipulating Antony to his other side as they prepare to leave the room, keeping him close for further discussion between just the two of them, as Caesar speaks of Cassius. The right hand is often symbolic of a place of honor and trust, and this might also be a reason for moving Antony.
Come on my right hand, for this ear is deaf,
And tell me truly what thou think'st of him. (I.ii.219-220)