The answers to your question lie, in part, directly in the context of the lines you refer to in Eliot's "The Love Song of J. Alfred Prufrock." Here are the lines:
Shall I part my hair behind [to cover a bald spot?]? Do I dare to eat a peach [because he wears dentures?]?
I shall wear white flannel trousers, and walk upon the beach.
I have heard the mermaids singing, each to each.
I do not think that they will sing to me. (lines 122-125)
First, the mermaids will not sing to him because he's old. He is not socially gifted, and his prime, if he ever had one, is long past.
The mermaids can probably be identified by other lines that follow those above:
I have seen them riding seaward on the waves
Combing the white hair of the waves blown back
When the wind blows the water white and black.
We have lingered in the chambers of the sea
By sea-girls wreathed with seaweed red and brown
Till human voices wake us, and we drown. (126-131)
The mermaids are an allusion to the sirens of Ancient Greek myth, who would sing to lure sailors to their underwater caves, then stop singing (breaking their spell), thereby drowning the sailors. The sense here is that the speaker daydreams, and then is awakened from his daydreams by someone talking, since the voices are "human voices." This, of course, connects the mermaids/sirens to the women with, or, I should say, in the room, with Prufrock.
But neither the women, nor the sirens, who are not particularly picky about the men they drown, will sing to Prufrock.