The reference is to an episode in the famous Romantic poem The Rime of the Ancient Mariner by Samuel Taylor Coleridge in which the title character kills an albatross, thus bringing great misfortune upon himself and his crewmates. In one of his letters to his sister, Walton says that he shall kill no albatross and that therefore his sister should have no need to worry about his safety. The implication is that Walton believes that killing an albatross would bring him as much bad luck as the ancient mariner, and so he promises his sister that he won't make the same mistake.

The poem means a lot to Walton. As he goes on to explain to his sister in the same letter his passion for the ocean was kindled by Coleridge's masterpiece. It was The Rime of the Ancient Mariner that first gave him a taste for adventure on the high seas.