Throughout the entire story, John and all of his people believed in all-powerful and mysterious Gods that had created, or been involved in somehow, the "Great Burning" that had destroyed civilization and made things so unsafe for the survivors. They had a fear and suspicious superstition of the Gods or the Place of the Gods; no one was allowed to go there, stories painted them as mysterious beings that were very different from them. They even called them Gods, placing them apart from themselves and putting distance and strangeness between them.
When John goes to the Place of the Gods and discovers that "they were just men" like him and his tribe, his entire world changes. He realizes that they were not gods, but men, men that were no different from himself. He realized at that moment that it was men's foolishness and selfishness that caused the great burning that killed so many, and that they had, essentially, destroyed themselves. They were not all-knowing or powerful, and were'nt god-like at all. He takes this information home and reveals it to his father, who wisely advises his son to reveal it piece by piece to their tribe so that they can process it and use each bit with prudence.
I hope that helped; good luck!
In my opinion, the most shattering discovery that John made is the fact that New York City (and therefore all the other dead places) was made by actual human beings. This would really shatter the whole basis of his society's faith.
The society in which John lives believes that New York City and all the other, smaller towns whose ruins they can see were made by gods. They call New York, especially, the place of the gods. But John finds out that people made the city. This means (to him) that his own people can become more "advanced" and can someday hope to be technologically sophisticated like the ancient people had been.