Frost is using the changing colors of the leaves to meditate on the continuing and unstoppable changes of life.
Youth is presented as the golden age, the best of all times of life. "Nature's first green is gold" - when life first emerges, it is at its best, bright with light and value. However, this "flower" of early life lasts briefly - "only so an hour." After that first glorious emergence of life, one stage "subsides" to the next, and even when the stages don't appear to hold tremendous differences from previous ones, time is marching on. There is nothing youth can do that would allow it to remain youthful forever.
Just as "Eden," the Garden of Paradise when the world was first created, "sank to grief" when sin entered the world and Adam and Eve were evicted from the garden, so too life is destined to move on as "dawn goes down to day." It is impossible to remain in the wonderful world of youth. "Nothing gold can stay."