First of all, Tom was sleeping with George's wife, Myrtle. This was as much Myrtle's fault as it was Tom's but we can say that by sleeping with his wife, Tom was hurting George and Myrtle's marriage. Faced with a failing business and a cheating wife, who no longer had any respect for him, George's misery partly had to do with the affair between Myrtle and Tom.
Tom is also subtly condescending towards George; note in Chapter 2 how Tom promises to sell George a car but keeps delaying the sale, saying he'll go elsewhere if George complains.
Perhaps the most dramatic instance when Tom victimizes George is when Tom tells George that Gatsby is the one who killed Myrtle with the car. (This is untrue; Daisy was driving.) In the final chapter, Nick confronts Tom, asking him if he blamed the accident on Gatsby. Tom replies:
"What if I did tell him? That fellow had it coming to him. He threw dust into your eyes just like he did in Daisy’s, but he was a tough one. He ran over Myrtle like you’d run over a dog and never even stopped his car.”
Tom told George that Gatsby was responsible for Myrtle's death. (George had already suspected Myrtle was having an affair and may have also concluded that Gatsby was the man she was having the affair with.) When Tom blamed Gatsby, this leads George to confront Gatsby and kill him, then to turn the gun on himself. Again, Tom plays a direct role in George's misfortune.