The title is based on an allusion to the ancient city of Babylon, which was a symbol of orgiastic decadence. Note how this fits into the plot of the short story - Charlie is returning to France for the first time since the Wall Street Crash. Before this point, Americans in Paris lived frenzied, decadent lives, and this story is full of memories of this time:

He remembered thousand-franc notes given to an orchestra for playing a single number, hundred-franc notes tossed to a doorman for calling a cab.

However, as he comments, there had been a price for this wastefulness - the loss of his daughter, which is why he has returned to the site of so much folly of waste, to reclaim his daughter.

Thus the title is based on an allusion that links Paris in its hey-day before the Wall Street Crash of 1929 with the ancient hedonistic city of Babylon and refers to the various excesses that Americans were able to indulge in at that time.