I have to take issue with your question, I am afraid. The theme you introduce suggests that the anonymous narrator of this story killed the old man as a "thoughtless action," yet if we examine the text we can see that this is certainly not the case, and that this murder was actually something that was premeditated and planned long before it was carried out. Consider the following quote:
You should have seen how wisely I proceeded--with what caution--with what foresight--with what dissimulation I went to work! I was never kinder to the old man than during the whole week before I killed him.
This quote clearly speaks of the way in which the murder of the old man was not therefore a thoughtless action, but one that the murderer had spent a long time thinking about.
The text however does clearly establish the way that whatever kind of evil action we commit does lead to guilt. The beating of the heart that the murderer hears at the end of the story of course symbolises the guilt that he is experiencing at some level in his consciousness, and, as it looks as if he is going to get away with the crime, he finds that to stop the sound he hears he must confess his deed and therefore assauge his guilt.