"The Seafarer" is an Anglo-Saxon poem which was undoubtedly sung and performed long before it was ever written down anywhere. It has a hundred and twenty-four lines and changes tone and direction (perhaps implying a second speaker or author of the poem) part of the way through. As you suggest in your question, the opening lines of the poem do make the poem seem elegiac.
An elegy, according to the eNotes "Guide to Literary Terms," defines elegy as "a mournful, melancholy poem, especially a funeral song or lament for the dead or a personal, reflective poem." The opening lines set the tone for what follows and do sound much like an elegy.
This tale is true, and mine. It tells
How the sea took me, swept me back
And forth in sorrow and fear and pain,
Showed me suffering in a hundred ships,
In a thousand ports, and in me.
The diction (language) is certainly mournful and melancholy: "sorrow," "fear," "pain," and "suffering." Though it is not a funeral song or sermon, it is a reflective poem. One speaker, the seafarer, talks about what it is like to live as a seaman; further reading shows that he has feelings of both love and hate for his mistress, the sea.
The fact that it is a personal account of life on the sea, spoken from the perspective of time and written in a melancholy, mournful tone, certainly suggests that "The Seafarer" is an elegy.