The two particular stanzas that you ask about really do not seem to address the theme that living without love leads to sorrow. The first stanza you cite describes a reaper out in the field, reaping barley and listening to the lady's singing. She sings "Like an angel" and the reaper calls her a "'fairy'" as a result of her beautiful, airy voice. The second stanza you cite describes where the lady lives as well as her own, "royally apparelled," appearance.
That being said, the three stanzas immediately following the ones you ask about do seem to begin to address this theme. She has "No time . . . to sport and play" and she must continue weaving or invoke the "curse [that] is on her"; she does not know what the curse is, and so she keeps weaving. "She lives with little joy or fear" and watches the other girls walking around the town in their red cloaks, but she cannot share in their company or happiness. "She hath no loyal knight and true," unlike the "damsels glad" who she sees. Her life, therefore, seems fairly empty because she lacks love.