In Shakespeare's play Twelfth Night, Duke Orsino is madly in love with Olivia, who has (repeatedly) rejected him. As the play opens, he is listening to music, and he says that if music can feed his love, the musicians should keep on playing, giving him an excess even, so that perhaps his appetite will sicken and die. He knows his love has been rejected, yet he cannot give up that love. It is still sweet to him, just like the sweet sound of the music around him.
That's where lines 5–7 come in. The Duke is partly talking about the music and partly about his love for Olivia, which is symbolized by the music. He says that the music (and love) comes over his ears like “the sweet sound, That breathes upon a bank of violets, / Stealing and giving odour.” He is using a complex simile here. The sound of the music is like a breath of wind that glides over a bed of flowers, both increasing their odor or taking it away depending on which way the wind is blowing. All of this symbolizes his love for Olivia. It, too, is like a breath over flowers. It should carry the odor of love, its joys and pleasures, toward him that he may relish in it, but since Olivia rejects him, it actually carries those joys and pleasures further away. He love remains unrealized and unfulfilled.
The Duke then orders the music to stop, for it no longer sounds as sweet to him. It reminds him too much of his unrequited love, the spirit of which will not leave him alone.