Let us remember that this essentially is the tale of one man's sin or transgression and his slow path back to redeeming himself from that sin. Of course, in the process, both he and the rest of the crew that he is sailing with suffer greatly and see many strange sights. Even though the Mariner is the only one to survive his experience, it is clear that this survival is a mixed blessing, as he leads some kind of half-life, forever haunted by his experience.
The sin that he commits is of course killing the albatross with his crossbow, which was taken as a symbol of good luck by the sailors. Note the response of the other sailors to the Mariner's action in shooting the crossbow:
And I had done a hellish thing,
And it would work 'em woe:
For all averred, I had killed the bird
That made the breeeze to blow.
Ah wretch! said they, the bird to slay,
That made the breeze to blow!
The continuing breeze was linked to the "lucky" presence of the albatross, and therefore the sailors predict bad things will befall the ship and its crew thanks to the slaughter of the albatross.
The sin and punishment is symbolised in the way that the Mariner must wear the dead albatross hung around his neck. Symbolically, this continues until he manages to redeem himself in Part IV of the poem by blessing nature in the form of the water snakes that he sees in the sea:
O happy living things! no tongue
Their beauty might declare:
A spring of love gushed from my heart,
And I blessed them unaware:
Sure my kind saint took pity on me,
And I blessed them unaware.
It is this action of blessing nature in the form of the water snakes that is seen as being sufficient to counteract the harm the Mariner caused to nature by killing the albatross in the first place, and thus at this stage the albatross falls from the Mariner's neck and falls "like lead into the sea." As if to reinforce the redemption that the Mariner has gained, he is able to pray at this stage as well.