Drawing an inference can only provide an "educated guess" as to why certain things happen in a story—most especially in that the author leaves certain details unclear. The reader, then, must search for meaning "between the lines."
An inference is present when one draws...
...a reasonable conclusion from the information presented.
In F. Scott Fitzgerald's The Great Gatsby, based upon the kind of person we know Tom to be, we can infer that he does not want to leave the East coast, or return to Chicago, because he has been having numerous affairs while married to Daisy—allegedly a "bad" one in Chicago.
Early on in the story, we find that Daisy and Jordan are certain that Tom's most recent mistress calls him on the phone in the middle of the day. Daisy doesn't take this quietly, but at one point excuses herself from the room; soon after, voices of husband and wife can be heard arguing.
In Chapter Seven...
"The rumor is," whispered Jordan, "that that's Tom's girl on the telephone."
We were silent. The voice in the hall rose high with annoyance: "Very well, then, I won't sell you the car at all...I'm under no obligation to you at all...and as for your bothering me about it at lunch time, I won't stand that at all!
As the chapter progresses, Tom admits to his infidelities. (The reader is made aware of the most recent one when Tom stops the car he is driving at the gas station to buy gas, and the owner's wife, Myrtle—Tom's mistress—stares out the window, believing Jordan is Tom's wife.) Tom says:
Once in a while I go off on a spree and make a fool of myself, but I always come back...
Tom has fooled himself into believing that his "mistakes" are excusable. However, when Gatsby tells Tom that Gatsby and Daisy have been in love for five years (not having an affair), Tom dismisses it as foolishness.
Daisy, on the other hand, has (seemingly) come to the end of her rope with Tom.
"You're revolting"...her voice, dropping an octave lower, filled the room with thrilling scorn: "Do you know why we left Chicago? I'm surprised that they didn't treat you to the story of that little spree."
The very fact that Tom refers to his affairs as "sprees," and now Daisy uses the same word again, clearly infers that Tom and Daisy left Chicago because of some "situation" that occurred there. (It seems to refer to an accident Tom had—with a woman in the car other than Daisy.)
Tom is a scoundrel, caring little for Daisy. However, as soon as Gatsby expresses his love for Daisy, Tom is ready to make amends:
I'm going to take better care of you from now on.
Gatsby tells Tom that Daisy is leaving her husband, and Daisy says that it's true. With Tom's casual attitude about Myrtle and his disregard for Daisy's feelings, it seems that the difficulty in Chicago arose from his infidelity, which had not stopped even after leaving that city behind.