The "place of the gods" is vague at first in the story. The first thing that the reader knows about the place of the gods is that it is east. John's people are forbidden to go east, because that is the place of the gods. As the reader continues to read though, it becomes clear that the place of the gods is New York City. It's an understandable thought for this far distant society. The world as we know it has basically come to an end. Most of the world's population has been killed. The remaining survivors are spread far apart and live a superstitious tribal life. Seeing New York City would be completely mind blowing. Huge bridges and buildings, metal everywhere—what else could make that except gods?
John eventually discovers that the place of the gods was a thriving city of people, not gods. He vows to slowly help his people begin learning about all of that lost knowledge in New York, so that his people can once again be great like the former New York residents.