Most poems have many different themes, so it's possible to interpret this poem from many angles.
One theme that stood out to me as I read this is the theme of growth. Children grow into adults. Seeds grow into trees.
The first stanza of Ezra Pound's poem could be a metaphor for growth. The girl might be speaking, saying she is growing into a tree, which would be a metaphor for adulthood. She says the tree grows in her hands, arms, and breasts, and that it grows outward from her.
"The branches grow out of me, like arms."
This line could be talking about how our growth affects others. When we as humans grow, we are changed, but we also change our interactions with others and with the world around us.
The second stanza could be the grown girl talking to her younger self, or it could be the poet talking to the girl.
It expands the image of the tree to include moss and violets. This could be symbolic of life in general—not just the growth of the tree but the interaction with all elements of a forest. Violets could be a reference to physical beauty.
Finally, the last line, "All this is folly to the world," might be a reference to some of the obstacles that inhibit growth. Instead of allowing her to grow free and wild, the world's pressures force the girl to grow into an expected outcome. There's no room for folly, or creativity, or wild branches and violets growing untamed.