The main theme of "By the Waters of Babylon" is the importance of knowledge as a route to social advancement. Alongside this runs the idea that change based on new forms of knowledge is best introduced gradually.
John travels to the Place of the Gods and investigates it for himself instead of relying on traditional teaching. While there, he learns the important truth that it was built by humans, not gods. From that piece of information, he realizes that his people, too, can attain knowledge that was once believed to be godlike and out of reach.
This knowledge animates and excites him. He is eager to return to his village to spread a revelation that he believes will radically change his people for the better. However, when he gets back, his father says to him that
If you eat too much truth at once, you may die of the truth.
This cautions John to take care and slow down. He understands that the knowledge that his society can improve itself is important, but he also perceives that he could tear his society apart by shocking it with too much information all at once.
The story is a period piece, reflecting a time in which technological progress was seen wholly as a good. John never questions why the former, more advanced society he wants to emulate might have destroyed itself. Today, we might see more value in a hybridization of the ecological "primitivism" that John wants to abandon and technology.