In this poem, Prufrock is trying to gather the courage to ask a woman a very serious question (many believe it is a marriage proposal, though Eliot doesn't state exactly what it is in the poem). He really wants to ask her something; however, he feels like the question would be too serious, drastic, earth-shattering, and out of place in the social gatherings that he has spent his life in. In the poem he describes the tea parties, the shallow discussions of women, the polite surface-level interactions that everyone has. He feels he has "measured out [his] life with coffee spoons" and not done or said anything of significance or importance. He has played a background role in every social setting. He has been the person to blend in, be polite, "cautious, meticulous" and concerned about pleasing everyone else. He is also very insecure and terrified of women's scrutiny and analysis.
So, given his insecurity, fear of rejection, and the boring, monotonous setting his life has been lived in up to this point, he feels that asking a woman this question will "disturb the universe". It will change everything. It is too serious, to life-changing, and the potential rejection might change the safe place he has in the world forever. Life has been flowing along in a lackadaisical and easy-going way for so long that dropping a serious bomb like a marriage-proposal might be too much, and change too many things.