If nothing else, we get some validation regarding that statement. We learn that Crooks demands that other people stay away because he has become used to (and bitter against) a society that shuns him. We get a chance to see the real Crooks when Lenny comes to visit. Crooks demands that Lenny leaves - more an act of what Crooks thinks he's supposed to do than an act of what Crooks really wants. Crooks becomes genuinely excited about the opportunity to join Lenny, George and Candy on their ranch and lets his guard down momentarily. It is at this point that Curley's wife quite abruptly and harshly reminds him of his place in society. This snap back to reality causes Crooks to return to his previous demeanor (almost martyr-like) and back out of his offer to go to the ranch. Crooks obviously doesn't like the way he's treated or what his place in society is, but he is full of an immense amount of pride and self-respect. In his mind, continuing to work at the ranch where he knows what's expected is a better prospect than venturing out into the rest of the world where he may be treated in the same manner that Curley's wife just did.

An interesting question. Readers learn that Crooks can be lonely, malicious, and vulnerable—and that he can hope but stay realistic.
The distance he keeps seems chosen at first; he reacts to Lennie's intrusion with hostility. However, he soon is glad of the company.
Crooks is often vulnerable due to his race, and he plays with Lennie's head a bit, getting a little revenge and showing a malicious side.

His vulnerability is shown in how Curley's wife can threaten him, and his hopes, balanced with realism, are shown in how he responds to the dream of the farm.