You are right to focus on this intriguing detail as something that helps us to build up a picture of the narrator in this excellent dramatic monologue. Let us first remind ourselves of how the text describes this moment:
And, last, she sat down by my side
And call'd me. When no voice replied,
She put my arm about her waist,
And made her smooth white shoulder bare,
And all her yellow hair displaced,
And, stooping, made my cheek lie there...
Let us note the way in which the narrator deliberately ignores his lover's voice. However, from what Porphyria then goes on to do, it seems as if this is a calculated response from the speaker to manipulate Porphyria into showing him her affection physically, as the quote makes clear. This incident therefore helps us build up a picture of the narrator as a coldly manipulative individual who is happy to use guilt and the goodness of other characters for his own ends. Porphyria is a woman who, because of her goodness, is therefore an easy victim to the speaker's heinous deeds.