In "By the Waters of Babylon", the Place of the Gods is really what is left of New York City after it was bombed. Man's technology eventually led to the destruction of the modern world, and John's society is the result of those who survived the blast. However, John lives hundreds of years in the future AFTER the bombing, and so the legends of the "land that burns forever" and "if you touch metal you die" were a result of the aftermath of the bombs. The metal held radiation, but no one who survived realized it was radiation, and so when people who touched metal began to die (from radiation poisoning), the legend began. It probably took a very long time for the fires to burn out after the bombing, and so the generations following the survivors told tales of the ground that burns forever. Since no one could live within the city limits after the blast, it came to be known as the Place of the Gods. There are clues throughout the story for the reader to follow in order to discover that the city is really NYC. The broken statue of "the man who wears his hear like a woman" with the inscription - ASHING- below it is a clue that this was a statue of George Washington. There are other broken clues for the reader to follow as well.
It is just a place that has been ruined by the bombing. Gods do not live there and destroy others. The buildings are ruined by the bombing and other things such as the roads are messed up.