The child that he was predicts the man he will become: just as it did as a child, the natural beauty and magic of a rainbow still has the power to make his heart leap. Despite the knowledge and understanding that comes with maturity (he now knows something of what makes a rainbow presumably), the beauty still moves him.
MY heart leaps up when I behold
A rainbow in the sky:
So was it when my life began,
So is it now I am a man,
So be it when I shall grow old
Or let me die!
The child is father of the man:
And I could wish my days to be
Bound each to each by natural piety.
When the speaker sees the rainbow, his heart "leaps up" with joy. He was happy to see rainbows as a baby, is still happy to see them as an adult, and hopes - one day - to be happy to see them as an old man.
Thus, his childhood enjoyment of the rainbow connects his childhood to his adulthood - and hopefully, will connect his adulthood to his old age. The rainbow and the pleasure the speaker takes in it is a constant point in his life: an unchanging pleasure in a changing world.
It is because his childhood enjoyment of the rainbow connects him as a child and an adult: hence, the man he is grew from the child he was. For that reason, the child might be thought to be father of the man.
I think the poet means that human life begins from childhood.A man cannot be a father without being a child.It is the child from whom the manhood begins.Yesterday's child is today's man and today's child is tomorrow's father
He previously speaks of the joy a rainbow causes him. This line is continuing the concept of the age referrence.
To be the "child is the father of the man" is referencing the continual growth and lessons you learn as a child which helps toward the development of the man he will be as he grows. As a child he found great joy and a natural connection to nature and he wants this to give birth to his perspective as an adult.