In this paragraph of the story, the old man senses someone in the room; he can't see or hear anyone, but senses someone there despite this.  At this realization, he lets out a "groan of mortal terror" which the narrator describes as the "low stifled sound that arises from the bottom of the soul when overcharged with awe".  The narrator himself has felt this awe, this terror himself.  The rest of the paragraph has the narrator describing the increasing fear the old man must have felt as he listened, listened, and wondered about someone being in the room.  The narrator takes delight in that growing fear, and claims to know "what the old man felt", as if he is in the man's head.  Such arrogance is typical of this narrator, and is further evidenced with his cocky behavior when the police show up.

The groan is a symbol of the old man's impending doom; he should be feeling afraid, he should be groaning in "mortal" terror, as his mortality is about to be taken from him.  It is also symbolic because the narrator has claimed to have felt that feeling before, and now he rejoiced to have the old man feel it.  It is also a symbol of the narrator's power; he likes that the old man knows he is there, and that he is powerless, he likes feeling the old man realizes what the narrator is about to do.  It emphasizes the narrator's perverse attitude, need for control and power, and total arrogance.