To answer this question, take a look at the second paragraph of the story. According to the narrator, the old man had an "evil eye" which made him feel very uneasy:
He had the eye of a vulture—a pale blue eye, with a film over it. Whenever it fell upon me, my blood ran cold.
In other words, the narrator felt compelled to kill the old man because this was the only way that he could stop the evil eye from bothering him. As a result, he immediately made plans to murder the old man.
However, if we look a little deeper, it becomes clear that the narrator killed the old man for another reason. Specifically, we can argue that the narrator killed the old man because the narrator was not of sound mind. In the week preceding the murder, he was clearly suffering from some kind of mental illness which caused a deep sense of paranoia about the old man. Although the narrator opens the story by claiming that he is not mad, his motivation for the murder suggests otherwise.