How does the speaker respond to the rainbow? How does this response connect to the three different stages of his life?
Wordsworth believed the the "world" (all that is not nature, eg. the city) corrupted the natural piety that we were born with. This is a key Romantic idea, and it is clearly illustrated in Mary Shelley's "Frankenstein." The creature is created and is full of love and kindness, searching for it in return. When he meets "society" (whether in the city or in the "society of man" in the countryside), he encounters hatred and fear.
Wordsworth remembers the joy he felt in nature when he was younger, and is thrilled that he can still experience this joy when he comes back to it ("Tinturn Abbey"). Whereas many men become "practical" and lose their sense of nature and what they can learn from it, he has not. If the goal of human life is to grow in the moral sense, then "the child is father to the man" because the child has this wonder as part of his nature, and, in time, gives "birth" to the man who has often grown apart of the child.
In another of Wordsworth poems:
"The world is too much with us, late and soon,
Getting and spending, we lay waste our powers.
Little we see in nature that is ours."
You might want to read the notes on "Tinturn Abbey" below for a clearer sense of Wordsworth relationship to nature.
MY heart leaps up when I behold
A rainbow in the sky:
So was it when my life began,
So is it now I am a man,
So be it when I shall grow old
Or let me die!
The child is father of the man:
And I could wish my days to be
Bound each to each by natural piety
The speaker's response to the rainbow, he says, is that his heart "leaps up" for joy, when he sees ("beholds") a rainbow in the sky. He saw, and took pleasure in, rainbows when he was born and when he was a baby, he still takes huge pleasure in them now that he is a man, and he hopes - if not, he'd rather die! - that they'll be there for him to take pleasure in as he grows old.
The child is like the man's father, he says, because - if you look at it chronologically - the child grows into the father, and so is in a strange way "older", as he belongs to the past. There's the three stages of his life: child, man, and old man - all connected by the pleasure the speaker takes in a rainbow. And, as a child is "bound" (tied) to the man he becomes, Wordsworth's speaker hopes that each day of his life will be bound to the next with "natural piety" (piousness - respectfulness - for nature [or just, "that comes naturally"]).