The nightingale, as Keats describes it, is not a part of the real world "where men sit and hear each other groan..." The bird is a part of something greater and more wonderful. Paradoxically for us, Keats associates this other-worldliness with death. We have to understand, though, that death in this sense means belonging to this other, more wonderful existence.
The speaker wants to be removed from the ordinary, mundane world and join that of the nightingale. He hears the bird's song and remarks that "Now more than ever seems it rich to die, / To cease upon the midnight with no pain..." It is not so much that the speaker wants to die as much as he wants to be removed from the real world and become part of the nightingale's.
What qualities does the speaker want that the bird has? The bird is care-free and at peace. It has no worldly worries or concerns.