I would first like to say that this is a highly contested question. Critics continue to argue vigorously about it. My opinion, however, is that Hamlet does not actually become insane, but puts on an act meant to throw his enemies off balance.
Polonius understands this when he says of Hamlet's behavior: "Though this be madness, yet there is method in ’t.’’
Polonius is an old windbag, but he is also a shrewd courtier who, though appearing witless, has, in fact, survived by using his wits.
But beyond the words of a courtier, we have the steadiness of Hamlet's purpose throughout the play. He might spout nonsense from time to time, and he definitely gets depressed at the corruption he witnesses all around him, but he stays on point, so to speak. His task is to determine if Claudius murdered his father, and if he did, to avenge the death. He works on this goal in a systematic way: he stages the play within a play to see his uncle's reaction to a murder that looks very much like what the ghost described. Once he establishes Claudius's guilt, he refrains from murdering him during his prayers so that Claudius won't go to heaven. When he does stab Polonius by mistake, it is because he thinks he is Claudius. Had it not been a mistake, he would have picked a rational moment to kill his uncle, when he is eavesdropping in an underhanded way and is "in sin," not in prayer.
While the events of the play do push the introspective Hamlet towards instability, he never acts merely crazily or like a madman. As Polonius says, there's always a method behind what he does.