Holden has mixed emotions about his visit with Mr. Spencer. At first, clearly, he goes there, because he cares about the teacher and is determined to say good-bye to him in person. But once Holden gets inside the Spencer home, he regrets his decision to visit him. The sights and smells of their home disturb him, make him uncomfortable. Mr. Spencer is sick in bed with the grippe (flu), and Holden hates seeing him in his pajamas. He has a brief conversation with him but it then turns into a lecture from Mr. Spencer about why he continues to fail out of one school after another.
Holden becomes very uncomfortable with the advice that Mr. Spencer is attempting to give him and he decides to rush out. As he is leaving he believs that he hears the older man call out "Good Luck."
The phrase is repulsive to Holden. He rejects this phrase as good wisehs for his future. To Holden, he feels like a spell has been cast on him that will actually bring just the opposite. Holden tells us:
"I'd never yell "Good luck!" at anybody. it sounds terrible, when you think about it." (Salinger)
Holden doesn't believe in luck, it does not make sense, especially when Holden can't make sense out of his younger brother's untimely death. For Holden, luck is just a stupid phony saying when you can't think of anything else to say to someone when they are leaving a place. He resents the implication that luck could help him anyway. For Holden, at this point in the story, life stinks.