As I understand it, what the speaker means is that the object of the poem ("thou") will come to be one with time because these lines (the poem) are eternal. So what he is saying is that the object of the poem will never die any more than time will die.
This goes with the prediction in the lines before. This is because the eternal life that the object will have (because of this poem) will ensure that she does not die and that her beauty will never grow old. Because of being in this poem, the object will live forever, just as lines 9 through 11 predict.
In this sonnet "Shall I compare thee to a summer's day" Shakespeare compares an unnamed woman to a beautiful summer day. The woman comes out favorably in the comparison because sometimes the sun is too hot, or the wind blows too hard, and of course, summer ends.
In lines 9-11, he says that unlike the summer day, the woman's beauty will never end when he writes, "but thy eternal summer shall not fade" and that death can not "brag" that it has conquered her. Why? Because she "lives" inside the poem. When Shakespeare says the woman will "grow" within the "eternal lines to time" he means that people will remember her because they remember the poem. He closes with "So long as men can breathe or eyes can see/ so long lives this [the poem] and this gives life to thee."
Shakespeare seems to have had a lot of confidence about this poem living on and, as it turns out, he was right. Here we are discussing it many hundreds of years later.