It isn't actually true that there's only one instance of dialogue (if we mean spoken words within quotation marks) in this text. If you look towards the middle of the story, the narrator describes how the old man leapt up in bed when he heard the narrator approaching, and cried out "Who's there?" Obviously it is notable that there is no response to this—the narrator does not say anything. He has appeared like a ghost and sits extremely still, not wanting to engage in a dialogue with the old man. So, in a sense, if a dialogue is a conversation between two people (its literal meaning) then there is no dialogue in this story at all. The old man speaks, but he does not get a response. The narrator is not interested in engaging with him; his mind is already made up.

In the next paragraph, there is more spoken dialogue from the old man, who is pictured talking to himself, attempting to "comfort himself" by detailing aloud what he thinks might have caused the noises in his house, such as "the wind" or "a mouse" or "a cricket." Once again, however, this is one-sided.

The statement in the final paragraph, then, is notable because it is the only expression of speech aloud from the narrator—although, as discussed, not the only example of words spoken aloud in the text. By keeping his narrator quiet, Poe has forced us to live entirely within his head. The narrator has not responded to the dialogue from the old man. He has not wanted to engage in any discussion with anyone other than the voices in his head becoming "louder! louder! louder! louder!"

This explosion from the narrator at the end, then, is notable because it indicates the breaking of the tension, the first example of the narrator actually engaging with the world around him because the madness he feels has become too much to contain, and has erupted—he can no longer bear his guilt, and must explain himself.