I would expect that F. Scott Fitzgerald's "Babylon Revisited" makes use of the literary device of allusion in referring to Babylon. An allusion is a reference to a well-known person, place, event, piece of art, etc. In other words, it is:
...a direct or indirect reference to something which is presumably commonly known, such as an event, book, myth, place, or work of art...
When an allusion is used, there is a comparison being made whereby some aspect of a story being told is compared to some famous person or situation. It is necessary to understand the reference to "Babylon"—otherwise, the comparison is lost to the reader.
Babylon was once a fortified city, surrounded by thick walls and deep waterways that were not penetrated until the Persians attacked in 539 BC. After it was taken—at that time—and later when it was taken by Alexander the Great, the city regained its reputation of accomplishment:
Babylon became...a centre of learning and scientific advancement. In Achaemenid Persia, the ancient Babylonian arts of astronomy and mathematics were revitalised and flourished, and Babylonian scholars completed maps of constellations.
Biblically, Babylon is synonymous with the Tower of Babel. It was a structure that was being built to reach the heavens.
Come, let us build ourselves a city, and a tower with its top in the heavens, and let us make a name for ourselves; otherwise we shall be scattered abroad upon the face of the whole earth.
In the Bible, God did not want mankind to have one language for then they could do anything. So God took the understanding of each man away and they stopped building. The success they had enjoyed was gone. That Babylon was once a place of high regard may have given Fitzgerald the sense of people tumbling socially from great heights—from great wealth—to abruptly return to earth, just like the Babylonians.
In the case of Charlie Wales, he was once like one of the builders of the tower. He had enormous wealth that he made in the stock market prior to the Crash of 1929. He lived a wild and lavish lifestyle. Inevitably, Charlie developed an addiction to alcohol. Then his wife, Helen, died. Charlie enters a sanitorium, and their daughter goes to live with Helen's sister and brother-in-law.
When Charlie is "better," the money is gone, so he goes to Prague to rekindle his fortune, hoping also to return to Paris to get his daughter back. It is not important that Charlie visit Paris, but that he revisits—more objectively it seems—the way he lived—like the Babylonians in "high" places.
He remembered thousand-franc notes given to an orchestra for playing a single number, hundred-franc notes tossed to a doorman for calling a cab.
He tries to capture the glory of his past:
Charlie watched Honoria's eyes leave their table, and he followed them wistfully about the room, wondering what they saw.
As Charlie sees the old hangouts and meets old "friends," he is tested. He must be...
strong to get his daughter back—and he really tries.
"Can't do it." He was glad for an excuse. As always, he felt Lorraine's passionate, provocative attraction, but his own rhythm was different now.
Charlie comes back to get his daughter, but he is also curious to see if he can recapture the magic of that time, while resisting the charm of his old life. He revisits not the place where they lived, but the memories of the kind of lives they lived—he knows he must let the past go to give the future a chance.