Holden's major criticism of almost everyone in the book is that they are "phony.' They are unable to stick to the ideals that they have or perhaps the ideals that he thinks they ought to have. Holden is the only one who sees the world as it is and the only one who can do anything to change it back to how it should be.
But, Holden too has a difficult time taking action on the things he feels. He is actually very eager for friendship and affection but instead he alienates almost everyone he meets by being loud, acting over-confident, picking fights, etc. He is unable to trust even Mr. Antolini. He goes to him for comfort and seems to have settled in but then when Mr. Antolini pats him on the head, Holden races out, telling himself that he was being a pervert.
He is also aware that his fondest dream, that of being the "catcher in the rye," is impossible. This also suggests that Holden is phony.