Gabriel is so self-absorbed, looking at the world through the lenses of his own needs and desires, that he is constantly misunderstanding and striking the wrong note with women. He is condescending, as when he asks the maid Lily if she is out of school, and when she says yes, replies that she will no doubt marry soon. This annoys, rather than pleases, Lily, who says men are all after only one thing.
Later, Gabriel makes a comment that is supposed to be a joke but could easily be interpreted as an insult when he says that:
It is not the first time that we have been the recipients—or perhaps, I had better say, the victims—of the hospitality of certain good ladies.
He interprets the ladies in question as turning red "with pleasure," but a reader could also interpret their "crimson" faces as expressing anger at his passive-aggressive dig about people being their "victims."
Gabriel's worst blunder, however, comes after the party. He is feeling amorous towards his wife and is stunned to discover she is not thinking of him at all, but of her dead first love, Michael Furey, a person he never dreamed existed. It angers him that his wife could have loved someone other than him or have had a life apart from him. He understands their disconnect when he thinks:
While he had been full of memories of their secret life together, full of tenderness and joy and desire, she had been comparing him in her mind with another.
Of course, she may very well not have been thinking of him at all, but that idea seems to be beyond Gabriel's comprehension. Whatever the case, he has a hard time relating to women, even his wife.