The title of the story is an allusion to Psalm 137 from the Bible.
By the waters, the waters of Babylon
We sat down and wept,
and wept, for thee, Zion.
The passage is a lament by the Israelite people. At that time in history, they were in despair over their capture and removal from Jerusalem and the promised holy land. Even back then, Jerusalem was one of the world's leading civilizations. It was a center of learning and knowledge, and the people were seen as members of a great nation; however, all of that changed when the Babylonians came and conquered it all. Israel and Jerusalem had fallen from power. It was now the Babylonian empire that took the lead for learning and civilization. And like all great empires do, the Babylonian empire eventually fell from power.
The allusion connects to the story because the allusion is about a former great civilization that fell from glory. Benet's story is about a great civilization that also fell from glory. Despite New York, the Place of the Gods, being a hub of culture, technology, and learning, it was destroyed and removed from power. In the story, John stands on the banks of the Hudson. He is looking at what remains of a great civilization. That is eerily similar to what the people of Israel must have gone through as they stood by the waters of the great Babylonians and wept at the loss of their own greatness.