The narrator does claim to have loved the old man. So, it would be logical for him to be remorseful after killing him. But the narrator is mad (crazy). He claims that his alleged madness is simply an overly acute attention to sensations and detail. Just before killing him, the narrator senses the old man's terror and expresses pity but also light amusement:
I knew what the old man felt, and pitied him, although I chuckled at heart.
When the narrator begins to kill the old man by dragging him to the floor and smothering him, he (narrator) "smiled gaily" to know that he was going through with killing the old man. After the old man is dead, the narrator was not really remorseful nor was he smiling or amused. He is momentarily relieved that he no longer has to feel the gaze of the old man's eye; then he goes into an anxious mode. He dismembers the body in order to hide it. He again notes that this is precaution, not madness. He brags about his precision in hiding the body.
When the police arrive, the narrator smiles to hide his guilt. After feeling that he convinced them of his innocence, the narrator feels relieved, "I was singularly at ease." However, his conscience gets to him and that momentary relief once again changes into precaution to anxiety to terror when the sound of the heartbeat increases.
So, after killing the old man, the narrator feels momentarily relieved, then anxious, momentarily relieved again, then anxious again and finally leading to complete terror resulting in his confession.