I get a lot of things out this poem. First, the poet is clearly speaking to an older woman, but that woman still has a lot of spunk left. She is "faded / like an old opera tune / Played upon a harpsichord." I've never heard a "faded" harpsichord. Quite the contrary: they are loud instruments and have no way to dampen the sound, like pianos do. This old opera tune is beautiful, but it comes through loud and clear.
The woman is beautiful as the "sun-flooded silks / Of an eighteenth-century boudoir." We aren't told whether we're looking at the eighteenth-century boudoir now--in which case the silks would be quite faded, particularly if they've been in the sun--or imagining the boudoir in its glory days, when the colors were vibrant and striking. The implication is that the older woman the poet is addressing still has a lot of color to her and reminds the poet of beautiful--even romantic--scenes.
Her eyes are like "the fallen roses of outlived minutes," but they still smolder. She may be aging, but her spirit is still on fire. The "perfume of [her] soul / Is vague and suffusing, / With the pungence of sealed spice-jars," too. What is "the perfume of her soul," anyway? I think it is the essence of who she is and always has been--her character, her personality. It is vague--perhaps because this aging woman is more a person of thought than action now, but it suffuses the poet nonetheless. "Pungence" is a strong smell, but "sealed spice-jars" wouldn't put of much, if any smell (since they're, um, sealed). Here, again, we have a contradiction in ideas, just like the "faded" "harpsichord" combination. The lady in question keeps her ideas and emotions to herself, but the poet senses their underlying power.
The lady is a study in "half-tones"--she is and she isn't all these things all at once, a walking paradox, fascinating and unknowable. The poet cannot figure her out, but clearly loves her.
In contrast, the poet is much younger, and can feel her own "vigor"--her energy and perhaps youthful indiscretion--particularly when she is around the lady. She appreciates that the lady finds her interesting and amusing, if only in the same way that she might be distracted by a shiny bauble. Still, for the poet, it is enough that she is recognized and appreciated by the lady on any level.