Hamlet's "madness" is one of the central concerns of the play. Hamlet says in asides throughout the play that he is not, in fact insane, but sometimes his performance is so convincing that it is difficult to tell. Indeed, Hamlet himself wonders if he is not mad, most notably when he sees his father's apparition in his mother's bedroom.
While he seems to have convinced Polonius and Claudius, in particular, that he is insane, both men seem to suspect that there may be something lurking behind his madness. Polonius remarks that "though this be madness, yet there is method in't" when confronted with Hamlet's nonsequitors, and Claudius worries that "[m]adness in great ones must not unwatch'd go."
Hamlet himself tells the audience that he will assume an "antic disposition" to bring about his revenge, and he assures Rosencrantz and Guildenstern that he is not actually mad, claiming "I am but mad north-north-west. When the wind is southerly I know a hawk from a handsaw." But at the same time he claims to be extremely depressed for reasons he does not know. So Hamlet's madness at times appears to be a ruse, which he says it is, and at times is very convincing. What seems certain, however, is that he is deeply unhappy and disillusioned with what has occurred within the royal family.