Upon comparing the attitude and behavior of the gray man as he interacts with the older woman that wears the ermine toque, we could safely conclude that the word that could be used to describe him is "condescending".

To be condescending means to treat others as if they were inferior. Judging by the actions and demeanor of the man, there is little doubt that he feels that the woman has done something that renders her in a vulnerable position. Not only does he shake his head in mockery as she approaches him, but he also blows the smoke of his cigarette into her face.

...he shook his head, lighted a cigarette, slowly breathed a great deep puff into her face, and even while she was still talking and laughing, flicked the match away and walked on.

The action is also rude, but to use that word alone is not correct because it does not explain the source of the behavior. The word "condescending" has a precedence, which is the feeling of being superior as well as the need to show others their inferiority. The story does not specify the origin of the encounter between the man and the woman in the ermine toque. Yet, we could assume that she is an old prostitute or escort who asks the gentleman in gray whether he would want her company. This is evident because, after the man in gray takes off,

the ermine toque turned, raised her hand as though she'd seen someone else, much nicer, just over there, and pattered away.

Therefore, since the man knows the occupation of the woman, he has no trouble reminding her of her status by treating her rudely.