After killing the innocent albatross—a bird seen as a good omen by his crew—the Mariner is forced to wear its body around his neck. This symbolizes his guilt and shame, originating from the terrible crime he committed against the natural world and God.
The poem is ambiguous about what this curse actually means and how the Mariner might relieve himself of it. However, the albatross falls from the Mariner's neck directly after he blesses the water snakes and appreciates their beauty. This is described in the following passage at the end of part 4:
The self same moment I could pray;
And from my neck so free
The Albatross fell off, and sank
Like lead into the sea.
It is unclear precisely why the Mariner's curse is broken after he blesses the water snakes. To answer this, we ought to consider why the Mariner is punished in the first place.
Essentially, the Mariner is punished because he shows a lack of respect towards nature and therefore a lack of respect towards God. If the Mariner had valued the albatross as part of God's creation, he would not have killed it in such a nonchalant manner.
As such, the Mariner is arguably cursed because his killing of the Albatross signals an attitude of superiority and apathy towards the natural world, which in turn revealed his disrespect towards God. It therefore follows that when Mariner is finally able to appreciate the inherent beauty of value of God's creation, as evidenced by his appreciation of the water snakes, he is relieved of his curse.
This act of appreciation shows that the Mariner has learned from his past mistakes and finally understands that he destroyed something beautiful and inherently valuable.
It also must be considered that the Mariner does not come out unpunished after his misadventure. Although the curse of the albatross around his neck is broken, the Mariner is still cursed in the sense that he is compelled to tell his story to anyone who might need to hear it, including the wedding guest.
The Mariner does break his curse in a sense, but he still must endure the consequences of killing the innocent albatross. We might therefore conclude that the Mariner never truly breaks the curse.