This poem decidedly uses nature imagery and figures of speech. Remember that imagery is description that appeals to one of the five senses: touch, sight, hearing, taste, or smell. Nature figures of speech are non-literal comparisons between two things, one of them something found in nature. Looking at these two details in the context of the poem will help you understand the poet's purpose behind them.
Often in poetry and literature, nature is used to show change. Seasons change, living things grow, wind and water are constantly moving. Keep this in mind as you look at this poem, as it seems to apply. The subject of the poem is a woman and a relationship that the speaker seems to be lamenting.
If you make a list of all the nature images and figures of speech in the poem, then, look at and analyze each one with this in mind, I think you'll reach a correct conclusion. Here are a few to get you started (notes in italics):
original air-blue gown: to me, signifies lightness, as if the newness of the relationship is lighthearted and easy; this may also be a comment on beauty.
breeze in its listlessness: the speaker seems to be questioning a new feeling. Is it a memory of the woman, or only the listless breeze? (Likely he is the one who feels listless in sadness or loneliness.)
If you continue in this list with the rest of the nature details, jotting down your thoughts, you can look through all of them together to obtain a big-picture meaning of the poem.