The sudden arrival of the two gentlemen, supposedly just to visit him, must surely arouse Hamlet's suspicion. He has not had contact with his childhood friends for some time, and he sees their appearance as incongruous behavior on their part. Their unannounced entry, by itself, immediately raises doubt about why they are there.
Furthermore, their overly affectionate greetings make it quite obvious that they must have some hidden agenda. They seem intent on pleasing him and salute him with such respect that their actions verge on the obsequious.
In addition, when Hamlet asks about why exactly they came to seem him, the two are quite evasive and refuse to give him a direct answer. They answer his questions with further questioning.
Hamlet is evidently aware, at this point, that Claudius is very interested in what he has been doing. Hamlet has deliberately been acting strangely (what he calls his "antic" behavior) and is quite sure that he is being watched. His conversation with Polonius, just a short while before, provoked him into crying out "These tedious old fools!" His exasperated response indicates that he might have been annoyed by Polonius's probing questions and that he was indirectly also irritated by Claudius for probably having instructed his minister to get to the bottom of Hamlet's unusual behavior. (We should remember, though, that it's a little hard to be sure of what Hamlet means when he's pretending to be crazy—this is just one interpretation.)
The two gentlemen eventually confess that they have been sent for, confirming Hamlet's initial suspicion.
I believe Hamlet distrusts Rosencrantz and Guildenstern because they don't seem to be giving him a straight answer. He tries to get them to admit why they have just suddenly shown up at Elsinore, and they say that it is only to visit him. He questions further, and Guildenstern says, "What should we say, my lord?" I think by this time Hamlet is getting a bit frustrated, and he comes right out and asks if they were sent for. Finally they admit that they were sent for by the king and queen.
Hamlet is probably to the point now where he doesn't know who he can trust, except Horatio. These guys show up and it's just two more people for him to be on the watch for.
Check the links below for more information on this scene and the characters!