By the time this poem ends, the speaker has gotten really tired of having the raven around. Throughout much of the poem, the raven's answers to the speaker's questions have been getting the speaker more and more hysterical. The speaker has come to see the raven as some messenger from the devil who has come to haunt him and make his life miserable.
Because of this, the speaker wants the raven to just get out. This is seen in the next to the last stanza of the poem. There, the speaker says
`Get thee back into the tempest and the Night's Plutonian shore!
Leave no black plume as a token of that lie thy soul hath spoken!
Leave my loneliness unbroken! - quit the bust above my door!
Take thy beak from out my heart, and take thy form from off my door!'
So he is telling the raven to just get out. Of course, all the raven says is "nevermore" and it does not leave so the speaker does not get what he wants.