Wordsworth included the final three lines of "My Heart Leaps Up" as an epigraph to his "Ode: Intimations of Immortality from Recollections of Early Childhood." In the "Ode," Wordsworth celebrates his experience with nature and life in his youth. He also laments losing the wonder he felt during these experiences, feeling that as he grew older, he'd lost some of that wonder:
Whither is fled the visionary gleam?
Where is it now, the glory and the dream?
However, by the end of the poem, Wordsworth finds reasons to remain hopeful, that he might continue to find profound significance in the experience of ordinary things: "To me the meanest flower that blows can give / Thoughts that do often lie too deep for tears." As an adult, he might be less enchanted with a rainbow (or a flower) but he is wiser and therefore can experience more profound meaning in such experiences.
Wordsworth intends a similar gesture in "My Heart Leaps Up." He does not go into such detail as he does in the "Ode", but he does indicate that he will continue to be emotionally moved by natural beauty, such as a rainbow, at all stages of his life.
Comparing the "Ode" and "My Heart Leaps Up," consider the idea that in Wordsworth's life, his childhood occurred before his adulthood. In a linear sense, the Wordsworth child existed before the Wordsworth adult, as if to say that the child lived before the adult and therefore the child's experiences taught the adult how to experience things. As an adult, Wordsworth recalls his childish wonder and learns from his childhood experiences. Just as a son feels loyal to his father, the adult Wordsworth feels loyal to himself as a child and he feels loyal to his future self and future experiences. This is what he means when he says that he wishes his days (his life and life stages) to be bound to each other by "natural piety" (natural duty).