Theme in story-telling is usually one of the main focuses of the story, so whenever you are trying to find out the theme of a story, try summarizing the underlying subject matter in one or two words. In this story, it can be summed up with "insanity". We have a narrator who kills a man because the man's eye bugs him, then hears the man's heartbeat after he is dead. Sounds crazy, right? Well, the narrator is aware that we think it is crazy, and so tells the story from the viewpoint of one who is trying to defend his sanity. There can be a big debate about the sanity of the narrator, and so that is why that is a major theme. Tied into that is the concept of guilt; we know the narrator is guilty, he never denies it. But, did his guilt drive him to confess, or was it his "insanity"? So, how did guilt play a role in the story?
I provided a link below to further discussion of some themes in the story, and that should help too. Good luck!