The central theme has to do with the differences between and the blending together of reality and imagination. The speaker acknowledges that the birch branches hang low because of the ice weighing them down. But he imagines they sag because boys have been swinging on them. He uses other similes and metaphors to describe the swinging and sagging of the branches and this is all an exercise of the imagination. When he says, "So was I once myself a swinger of birches," he is being literal and figurative. He may have actually been a swinger of birches, but he also has been and continues to be someone who uses his imagination to "get away from the earth awhile."
Note this idea of "getting away from the earth awhile" as a way of describing the imagination. The speaker loves that life enables one to escape into the world of imagination but always comes back down to earth/reality. He likes to escape but also needs to come back because "Earth's the right place for love: I don't know where it's likely to go better."
Literally and figuratively, as he swings on the branches, they bend down and back up toward heaven. Going up towards heaven, he is getting away from the earth. He is also getting away from reality and up into his imagination. There is the notion that earth and reality are on the ground. Swinging up, he heads toward his imagination and heaven. So, in a sense he is swinging between life and death (the afterlife, heaven). Of course, he prefers to inevitably come back down to life.
Note in these last few lines, he would like to go Toward heaven, but come right back down again:
I'd like to go by climbing a birch tree,
And climb black branches up a snow-white trunk
Toward heaven, till the tree could bear no more,
But dipped its top and set me down again.
When he says "I'd like to go," it may be a reference to death in that he is going from life. In this interpretation, if he does "go," he would like to come back (whatever that might mean). Or perhaps, he would like heaven to be like earth. In this case, death would be like coming back down to earth/reality again. He prefers an existence (whether in life or in heaven) in which one can swing between two worlds: reality and imagination.