If I were to define their friendship in terms of psychological description, I wouldnot only call it inconsistent, but also dysfunctional. Like the previous poster commented, G and R are always together. They are a dyad in their own accord. Yet, strategically speaking for the sake of the organization of the characters, Hamlet denotes an inability to belong to a group, to a dyad, and not even to a society without causing himself some major drama.
Hence, the primary dyad would be Guilderstern and Rosencrantz, and Hamlet is the third leg which tampers with the balance of the relationship of all three friends. It is this lack of balance what makes the friendship dysfunctional, and it is even more so when Hamlet disposes of them in cold blood.
As far as we know, a true friendship between Hamlet, Rosencrantz, and Guildenstern existed before the play starts. As the play goes along, it becomes weakened. But I believe Rosencrantz and Guildenstern act as true friends. It is Hamlet who does not.
The only "sin" that Rosencrantz and Guildenstern commit against Hamlet is to try to find out why he is behaving strangely. While they may be doing this as a way to get in good with the king, I do not see where it is such a bad thing. It seems that it is appropriate for two friends to try to find out why their other friend is acting so strangely.
But Hamlet goes well over the line when he arranges to have them killed. He was doing it to try to save himself, but he surely could have done that some other way.