This is a poem that is at least largely about sex and love. The poem starts out with the line you cite. What the line means is that the speaker is asking herself what young man that she has had sex with (with me lay) gave her the most pleasure.
She then goes on to think about the different ways that love and sex have felt with the different young men she has known. The speaker is thinking about what the relationship is between love and sex. The speaker seems to be arguing that sexual pleasure is more important than love.