The answer to this question can be found right at the beginning of “By the Waters of Babylon.” The narrator, whose name is John, and his father are allowed to take metal from the dead places because of the fact that they are religious leaders. The narrator tells us that
It is forbidden to go to any of the Dead Places except to search for metal and then he who touches the metal must be a priest or the son of a priest.
At the start of the next paragraph after this passage, we are told that John’s father is a priest and that he, therefore, is the son of a priest.
As to why only priests and their sons are allowed to touch the metal, we can only speculate. Clearly, the dead places have immense religious significance for the Hill People. The metal from those places is particularly feared. Maybe it is sometimes radioactive or maybe it is just that the Hill People fear that something so valuable might be cursed by its association with the dead civilization.
Whatever the cause, the Hill People fear the dead places and the metal that is there. The narrator and his father are allowed to touch it because the father is a priest and the narrator is the son of a priest (and a priest himself, later in the story).