The main thing that keeps Hamlet from taking his own life is his fear of what comes after death. He really has no idea what sort of world, existence, or non-existence is in place after one dies, and that is why he doesn't just kill himself. He wants to; he is miserable. He feels defeated and depressed, and everything in the world appears "rank" and "vile" to him. But, it is his own fear of the next world that stays his hand. He goes into great detail about his fears in his famous "To be or not to be" monologue that can be found in Act III, Scene 1. In this speech, he indicates that death is a "consummation devoutly to be wish'd," but that "the dread of something after death" keeps people from ending their own lives.
His own fear causes him great turmoil. He feels like he is a coward, who can't act. He says that his "resolution" or courage is "sicklied o'er witht he pale cast of thought," meaning, he can't do it because he thinks too much; his conscience steps in and analyzes the situation too much and stops his courage and action. This is a problem for Hamlet in the play, not just in the thought of suicide, but in enacting the promised revenge against his uncle as well. I hope that those thoughts helped; good luck!