The quote that you are referring to is:
Why they came East I don't know. They had spent a year in France for no particular reason, and then drifted here and there unrestfully wherever people played polo and were rich together. This was a permanent move, said Daisy over the telephone, but I didn't believe it – I had no sight into Daisy's heart, but I felt that Tom would drift on forever seeking, a little wistfully, for the dramatic turbulence of some irrecoverable football game.
Very little factual information is given about Tom and Daisy's wandering spirits, but the word "unrestfully" describes Nick Carraway's thoughts on the matter. Tom is crazy rich and comes from old money. He is a big guy, powerful, aggressive, and a bit brutish. He was accepted to Yale, which is an honor in and of itself, but then Tom also played football for Yale. He was likely one of the studs of the team too. So what you have is a guy that is rich, used to getting his way, and a sports star to boot. After college, though, his life turned into social outings, fancy polo matches, and doing calm, rich people yachting stuff.
Those things gave Tom a sense of unrest. He's got pent up energy and wants to find an outlet for it. So he and Daisy drift from location to location, because Tom is looking for his next "high."