The swans themselves symbolize the stability, the sense of continuity that Yeats so deeply venerates, and which he sees embodied by the old families of the Protestant Ascendancy. In the midst of all the turmoil of Irish politics and society, the swans represent a much-needed sense of place for the speaker, reinforcing his almost mystical connection to this ancient plot. The world outside may be changing, but the nature of the wild swans does not.
And it is because their nature never really changes that it can serve as an additional symbol, that of youth. The speaker is old and weary and gives the impression of a profound disillusionment with life. Yet the swans, "unwearied still," never get old, and so they provide the speaker in his twilight years with a rare spark of vitality. So long as the speaker can count the swans each year, then he will still retain an element in his soul, however tiny, of his long-lost youth.