In Lord Byron's poem, Canto III stanza 6 picks up an idea presented in the previous stanzas. Addressing Ada, his daughter, he is reflecting on his youth, when he sought answers. In Stanza 3 he refers to this persona as "the wandering outlaw of his own dark mind."
In stanza 5, he notes some questions we ask and things we learn as through profound experience, but not necessarily age. A person can become world-weary or cynical, "so that no wonder waits him," and cannot feel love, sorrow, or other emotions.
[H]e can tell
Why thought seeks refuge in lone caves, yet rife
With airy images, and shapes which dwell
Still unimpair'd though old, in the soul's haunted cell.
The reason for seeking refuge among the airy images—by which Byron means, figuratively, spending time alone with one's own imagination--is then given in the next stanza. The reason is so we can create.
Through creativity we give concrete substance to the things we imagine, "endow with form our fancy." This is what he (as a poet) does.
He then switches back to speaking to Ada. It is implied that because she is young, she matters, while he is "nothing." Even though she is physically not with him as he travels, her soul is in his thoughts and he sees her in his mind's eye—"invisible yet gazing."
He then positively contrasts the "glow" of those sensations with the kind of weariness mentioned previously, which he calls "my crushed feelings dearth." That is, even though he is away from his child, becoming a father has renewed his spirit.