From what the Misfit says to the grandmother in his conversation with her, it seems clear that the biggest reason why the Misfit is so tormented stems from his own feelings of being trapped and locked up in a society which seems to offer him no hope or possibility of a meaningful future outside of breaking the law. Note what he says to the grandmother in this fascinating quote:
"Turn to the right, it was a wall," the Misfit said, looking up again at the cloudless sky. "Turn to the left, it was a wall. Look up it was a ceiling, look down it was a floor. I forget what I done, lady. I set there and set there, trying to remember what it was I done and I ain't recalled it to this day. Oncet in a while, I would think it was coming to me, but it never came."
The Misfit, for whatever reason, is a character who is profoundly disengaged from society and from the basic norms and values that govern day-to-day living. He sees no possibility of ever entering "normal" life as the grandmother tries to persuade him to do, and is clearly alienated from society as well as from himself. Although we are not given the full details of his background and what has helped to produce his character, it is clear that this feeling of being a literal "mis-fit" in society is something that is behind his criminal activity.