The answer to this can be found at the very end of “By the Waters of Babylon.”   At the end of the story, John resolves that he will make fundamental changes when he is the chief priest.  His people will no longer hold to their current superstitions.  They will not fear the “dead places” or believe that there were gods who lived in the Place of the Gods any longer.  Instead, they will understand the truth and they will “build again.”

At the time that this story takes place, John’s people know very little about the civilization that had existed before them.  They enter the “dead places” to get metal but for no other reason. They believe that the massive ruins to the east (where they were not allowed to go) had been the home of gods. They  are quite primitive in terms of beliefs and technology.

At the end of the story, after John has stayed in the Place of the Gods for a night, he resolves that he will change things when he is chief priest. He has figured out that 

they were men who built the city, not gods or demons. They were men.

He has realized that it was human beings who had all this technology and the ability to build tall buildings and great roads.  When he realizes this, he wants his people to progress. He wants them to learn more. He wants them to go back to “newyork” and learn more about the ancient civilization.  Above all, he wants them to become more technologically advanced. As he says in the last line of the story, “we must build again.”