When Prufrock begins to think of his questions, he first thinks of the other people in his life/society and what they will think of his questions. He speaks of the arms of the beautiful women, who will essentially tell him he's a fool if he asks his questions around them. He talks about being in the middle of a party where he wants to talk about important things, and everyone else just wants to talk about how bald he is getting. He describes feeling like an insect, "sprawled and wriggling on a pin," and how he feels insignificant like a bug would feel. So how should he presume to ask important questions in life, when an insect never does?
These interruptions of thought by memories of other people show us that Prufrock is too concerned with the opinions of others. His self-esteem is low, and he fears that he will be mocked if he tries to say anything important. So he never quite gets around to asking his questions, or starting the big conversations, for fear that everyone will laugh at him.
I feel the "overwhelming question" is one for the reader to interpret on his or her own. It is probably something along the lines of the meaning of life, etc., but again only Prufrock knows for sure, and he never said it out loud. So the readers can interpret that one how they will.