The principal evidence that Tom Buchanan is conducting an affair with Myrtle Wilson can be found in Chapter Two of The Great Gatsby. In this chapter Nick states,
I first met Tom Buchanan's mistress....
The fact that he had one was insisted upon wherever he was known. His acquaintances resented the fact that he turned up in popular restaurant with her and, leaving her at a table, sauntered about chatting with whomsoever he knew.
One day Nick rides with Tom through the Wasteland where Tom stops at the garage of Mr. Wilson, a spiritless man, "anemic and faintly handsome." But, Tom describes him as not even knowing he is alive. After leaving Wilson's garage, Tom drives down the road and waits for Myrtle, having told her he wanted to see her. They drive to the train station, but Myrtle sits in another car from Tom in deference the the "sensibilities of those East Eggers who might be on the train."
When they reach New York City, their cab stops at 158th Street before white apartment houses. Inside, Nick surveys a room with furniture that is over-sized--perhaps, the Buchanan's cast-offs. Myrtle phones the McKees and some other people such as her sister Catherine to come up and socialize. Then, she changes her dress and assumes a new identity.
Her laughter , her gestures, her assertions became more violently affected moment by moment and as she expanded the room grew smaller around her until she seemed to be revolving on a noisy, creaking pivot through the smoky air.
As the evening progresses, Myrtle becomes haughtier until she oversteps a boundary by bringing up Daisy's name repeatedly; then [M]aking a short deft movement Tom Buchanan broke her nose with his open hand." This break, then, leads to the end of the party.
Different editions of any book will have the same text on different pages, so there aren't any page numbers included in this answer.
In the first chapter, Tom is called away from entertaining Nick, Jordan Baker and Daisy by a phone call. While Tom has left the gathering to take the call, Jordan does her best to eavesdrop on the conversation before explaining the situation to Nick with the explanation, "Tom's got some woman in New York." Daisy's "tense gayety" when she and Tom return to the party, and her reaction when the phone rang again ("Daisy shook her head decisively at Tom") seems to confirm that these calls might be coming from a source not completely welcome.
The first time in the book that Tom and Myrtle are together, outside George Wilson's garage with George and Nick present, the attraction between the two is obvious.
She smiled slowly and, walking through her husband as if he were a ghost, shook hands with Tom, looking him flush in the eye.
Their conduct after rendezvousing at the train station openly acknowledges and reinforces their infatuation with each other.
In chapter seven, when Gatsby, Daisy, Nick, Tom, and Jordan are all headed to New York City, the car carrying Tom, Nick, and Jordan stops at Wilson's for some gasoline. When George Wilson speaks with Tom, Nick realizes that -- although Wilson knows his wife, Myrtle, has been having an affair -- "his suspicions hadn't alighted on Tom" yet (124), and Nick realizes how similar Tom and Wilson actually are: both of them are being cheated on by wives they had trusted. Further, Nick describes the terrible expression on Myrtle's face when she sees Jordan in the car next to Tom. He says that her expression was familiar, but that it seemed unexplainable "until [he] realized that her eyes, wide with jealous terror, were fixed [...] on Jordan Baker, whom she took to be [Tom's] wife" (125). She is incredibly jealous of Jordan, who she believes to be Daisy because she's never actually seen Daisy, because Tom's wife is her competition for his affection and time.
Moreover, Nick almost feels sorry for Tom in this moment because "His wife and his mistress, until an hour ago [were] secure and inviolate, [and] were [now] slipping precipitately from his control" (125). Tom seems to have felt completely secure until this afternoon, when he found out both that his wife is cheating on him and that his mistress will be "going west" with her husband.