If you look at the many different cues in the story you may find three facts about the narrator:
- he (or she, we never know) premeditates and seems to enjoy the killing (joy)
- there is the pain that you mention in some particulars of the killing (sadness)
- the narrator identifies himself as "nervous", "sleepless", and extremely emotional (anxiety)
While these are clear signs of a mentally ill and unstable individual who cannot control his own impulses, this is no ordinary murderer. This is not the typical psychopath or sociopath who is narcissistic and entirely devoid of empathy. This killer does show emotion. This one may be entirely empathetic by nature, but his physical impediments and mental conditions have rendered him unable to abide by his natural gifts. In fact, his brain might have rendered him limited to act like his nature would dictate him to.
It is a typical battle of nature versus nurture; of the body versus the mind. This conflict is particularly painful to the narrator, who is indeed pained by what he did. His entire confession begs for compassion and screams for help. You might as well not typecast the narrator as a typical murderer, but merely as a very crazy and disturbed man who is aware of his unhappy condition.
Therefore, you are correct in that there is conscience in the man. He loses it, in fact, he loses everything he is about as a result of his mental condition. He knows it, too. He cannot control himself. This is a great touch that Poe adds to make the situation all the more tragic.