This scene occurs shortly after Hamlet has staged the "Mousetrap" play that reenacts his father's murder. Seeing Claudius's reaction to it, Hamlet is convinced that the ghost's story is true. At this point, Rosencrantz and Guildenstern come to him. They announce his mother wants to see him. Then Rosencrantz mentions that they once were friends and asks,
Good my lord, what is your cause of distemper? You do surely bar the door upon your own liberty if you deny your griefs to your friend.
This is the wrong time to be asking Hamlet this question. The confirmation that Claudius murdered his father is fresh in his mind, he feels beset by enemies, and he is working towards a frenzy. The reminder that the two courtiers have been recruited by Claudius to spy on him acts as an irritant.
The performance over, the players are passing by carrying recorders. Hamlet plucks one and wants Guildenstern to play it. When he says he can't, Hamlet compares not Guildenstern, but himself to a recorder. He is angry and, though he addresses Guildenstern, he asks both men why they are treating him as if he is as easy to "play" as a recorder. Hamlet states,
how unworthy a thing you make of me! You would play upon me; you would seem to know my stops; you would pluck out the heart of my mystery
Hamlet feels insulted and offended that these two courtiers are so obviously trying to use him as a tool and pump him for information that they can bring back to Claudius. He is warning them that he is not such a fool as they think and that they should treat him more carefully. Unfortunately for them, they can't truly hear what he has to say or the danger he represents to them.