You are right in focusing on how Poe uses the setting of this terrifying tale to exacerbate the horror of the story. Certainly the following six details of the setting seem to contribute greatly to the overall atmosphere and mood of terror and impending doom:
1) When the narrator first describes the setting when he gains consciousness, refering to the "sabel draperies" and the "seven tall candles" upon the table which become the source of such a strong feeling of nausea in the narrator.
2) Clearly, the way in which he is aware in his state of unconciousness of being borne "down--down--still down" by shadowy figures until the narrator becomes confused and dizzied by the "interminableness of the descent" likewise reinforces the horror of the setting.
3) When the narrator gains consciousness again and is met by "the intensity of the darkness" that "stifles" him. He has no idea of his location and his whereabouts.
4) The terrifying discovery of the pit that he could so easily have fallen into if it were not for a fortunate accident presents the way that death is always near at hand in this setting.
5) The way in which the pendulum descends from the ceiling of this cell and the torture in which the narrator is going to meet his end is likewise a key moment of terror.
6) Lastly, the way in which the walls of the cell become hot and move in on the narrator, pushing him towards the maw of the terrifying pit, represent the last horror of the Inquisition and their determination to end the life of the narrator.