The mariner is old (ancient), described as having a long, grey beard. The mariner kills an albatross, apparently for no reason, even though the albatross had seemed to lead them out of the Antarctic. The mariner's crew first curse him for doing such a thing. But then they rationalize the shooting by saying that is was right to shoot the bird that brought the fog and the mist.
He and his crew become lost at sea and he realizes his crime is the cause of their situation. The crew then hangs the dead bird on the mariner's neck. (It is now his cross to bear.) His men drown. The mariner lives on (Life-in-Death) and must continue to travel to new lands in order to find a stranger and confess his sin.
The mariner is lonely and feels forsaken because he is cursed to an existence of repeating his confession. The wedding guest is at first terrified and then captivated and sympathetic to the mariner's speech. The one consolation (and/or moral) is that the mariner is temporarily assuaged by confessing his story, he is wiser from the experience, and his confession may teach a lesson to others. In this last stanza, "He" refers to the Wedding-Guest.
He went like one that hath been stunn'd,
And is of sense forlorn.
A sadder and a wiser man,
He rose the morrow morn.
The mariner is cursed because of a thoughtless act but he is also a kind of timeless teacher and this is both a burden and a duty.