The listener could be any kind of person imaginable, but Poe puts the reader himself in this role. In this way, he or she gets "involved," ready or not!
Poe used this kind of confessional narration in other famous short stories as well, for example "The Black Cat" or "The Tell-Tale Heart." The reader is considered a friend or a person of confidence (such as priest or doctor vowed to silence) to whom one can tell the darkest and ugliest secrets to!
With a stretch of the imagination, one could imagine Montressor telling the story to his next victim (perhaps already chained to the wall as Fortunado was) since the secret will die with him, too!