Tom was rude to Gatsby on more than one occasion, seeing him as little more than a vulgar, nouveau-riche upstart who was hardly worth the time of day.
To set the context for Tom's disdain, in one scene, Tom Buchanan and his friends the Sloanes, who are out horseback riding, stop at Gatsby's mansion for a drink. Gatsby is delighted to see them, but they want nothing more than to drink and run. Tom doesn't even remember having met Gatsby a few weeks earlier in New York City and is astonished that Gatsby doesn't understand when he and Mr. Sloane try to give him the brush off. Mrs. Sloane had just invited Gatsby to dinner: "Doesn't he know she doesn't want him?" Tom says as Gatsby goes off to get ready. The group leaves without him, telling him they can't wait, snubbing him entirely.
Tom is surprised to discover that his wife is having an affair with the, to him, low-class Gatsby. When Tom is at the Plaza Hotel with Gatsby, Daisy, Jordan and Nick, having just realized what is going on between Gatsby and Daisy, he treats Gatsby with contempt, ridiculing the idea that Daisy would ever go off with him. He sneers at Gatsby as "Mr. Nobody from Nowhere." If someone as high class as Daisy can run off with someone as low class as Gatsby, Tom thinks, the next thing will be black/white intermarriage.
Gatsby responds by insisting that Daisy never loved Tom, and only ever really loved him. Tom continues to heap scorn on Gatsby, wondering how he could get close to Daisy at all "unless you brought groceries to the back door."
Gatsby responds by appealing directly to Daisy to confirm her love. She begins to waver. In the end, class will win out.