Towards the end of Scene 4 in Act 3, Hamlet states plainly to his mother that he is only pretending to be mad. These are the pertinent lines he speaks when she asks what he wants her to do:
Not this, by no means, that I bid you do:
Let the bloat King tempt you again to bed;
Pinch wanton on your cheek; call you his mouse;
And let him, for a pair of reechy kisses,(200)
Or paddling in your neck with his damn'd fingers,
Make you to ravel all this matter out,
That I essentially am not in madness,
But mad in craft. 'Twere good you let him know;
For who that's but a queen, fair, sober, wise,(205)
Would from a paddock, from a bat, a gib
Such dear concernings hide?
When Hamlet first arrived in her chambers, Gertrude was convinced he was mad, and she was afraid of him. She thought he had come to kill her and screamed for help, which was what led to the death of Polonius. Her son's eccentric behavior had just culminated in his staging a play which seemed to accuse Claudius of murdering his brother by pouring poison in his ear. Then when Hamlet kills Polonius, Gertrude is nearly hysterical. But he succeeds in convincing her that he is pretending to be insane for a purpose. His main purpose has always been to deceive Claudius, who has been trying intently to find out what is going on inside Hamlet's mind, even summoning Rosencrantz and Guildenstern to spy on their schoolmate. Does Hamlet suspect Claudius of killing the former king? Does Hamlet have some information Claudius doesn't know about? Is Hamlet smart enough to be able to figure out that Claudius committed a murder when no one else could have seen it? Are there really any poisonous snakes in Denmark? Is it possible that Hamlet is in communication with the ghost of his father? That might have occurred to Claudius, since people believed in ghosts in those days. If the ghost wanted revenge, he would certainly try to achieve it through his son. Claudius is consumed with guilt and fear of exposure. His behavior at the play-within-a-play has made a bad impression on Gertrude, whom he deserted when he fled the room. She can see, as Hamlet explains to her, that her husband is not worth "twentieth part of the tithe" (or 2%) of the husband she has betrayed by her adulterous union.
Hamlet also tells his mother in this same scene:
My pulse as yours doth temperately keep time(155)
And makes as healthful music. It is not madness
That I have utt'red. Bring me to the test,
And I the matter will reword; which madness
Would gambol from. Mother, for love of grace,
Lay not that flattering unction to your soul(160)
That not your trespass but my madness speaks.
It will but skin and film the ulcerous place,
Whiles rank corruption, mining all within,
He is saying, in part, that it was precisely his mother's "trespass" that has made him so furious, so rash, so vindictive. so misanthropic and misogynistic. He manages to convince her that he is sane. We see this when she has to go to her husband to report that Polonius has been killed while hiding behind the tapestry. But she is obviously now in a conspiracy with her son. She does not say anything about Hamlet's appearing to be talking with his father's ghost. She may not believe that really happened, but Claudius would believe it because of his guilt and fear. In fact, the appearance of a ghost would explain everything to Claudius if he heard about it. He would know that Hamlet planned to kill him at the earliest opportunity, and he would decide to forestall that act by acting first.
Hamlet tells his mother, Gertrude, that he is only pretending to be insane. Gertrude at first thought that her son had gone crazy after she witnessed him talking to a ghost. However, after Hamlet's explanation, she realizes that he is plain unhappy and melancholic about her marriage to Claudius, her brother-in-law who had killed her husband.