The answer to this question can be found in the third stanza, where the speaker contemplates what it is the reaper is singing about. Although he cannot discern the words that he hears, it is clear that the music definitely communicates some kind of universal human experience that touches the speaker's soul. The third stanza contains a series of rhetorical questions that ask what it is the reaper is singing:
Will no one tell me what she sings?--
Perhaps the plaintive numbers flow
For old, unhappy, far-off things,
And battles long ago:
Or is it some more humble lay,
Familiar matter of to-day?
Some natural sorrow, loss, or pain,
That has been, and may be again?
Whatever it is that she does sing about, it is clear that the reaper's words create a burden in the speaker's soul as he finds that her singing captures some kind of universal longing and yearning within him, and indeed within all humanity, that causes him to take with him the sound of her singing long after he ceases to hear it, as the final stanza indicates: "The music in my heart I bore, / Long after I heard it no more." The burden of the music therefore relates to the way in which the speaker finds that it allows him to transcend reality for a brief spell.