John's innoeent and somewhat naive perspective in "By the Waters of Babylon" forces reader to eye his journey through the fallen remains of our own modern society through a fresh pair of eyes. John encounters many common place objects throughout the course of his trip to the Place of the Gods:
"great spike of rusted metal sticking out into the river"-- This obscure reference could possibly mean the remnants of the George Washington Bridge in New York.
"Everywhere in it there are god-roads, though most are cracked and broken." This quote describes broken highways and overpasses in New York City.
"there was a carved stone with cut—letters, broken in half. I can read letters but I could not understand these. They said UBTREAS." John is at the Subtreasury Building in New York.
"They got their food from enchanted boxes and jars." John unknowingly refers to canned goods and sodas. Later, when John flees the pack of wild dogs, he finds and consumes possibly a bottle of wine, commenting that the drink made his head swim.
"On one side of it was a bronze door that could not be opened, for it had no handle." John sees an elevator door for the apartment building, but of course, without electricity, he cannot make it open.
"There was a cooking-place but no wood" refers to an electric stove top in the kitchen. The kitchen proves quite a mystery to John, especially the sink with its labeled handles for 'hot' and 'cold'.
I think that you are probably asking about the sorts of things that John sees when he is in the Place of the Gods. Specifically, these are the things that he sees when he spends the night in what sounds like the apartment of someone who was relatively rich.
John mentions a few things that are particularly mysterious to him. He mentions the pontillist painting that looks different close up than it does from farther away. He mentions a statue of a bird made of some kind ceramic substance. He particularly mentions all the appliances -- the thing for cooking that uses no wood, the things that looked like lamps but had no wicks, things like that. Look in that area of the story to find more things that you could call mysterious.