This part of the dialogue comes from Act 2 of the Shavian play Pygmalion, during a conversation between Mrs. Pearce and Higgins. The situation at this point is that Eliza needs to be washed as part of her transformation, her old clothes need to be thrown away, and her hat will go in the oven as a way to disinfect it.
Aside from the physical cleanliness of Eliza, Mrs. Pearce tells Higgins that she is concerned about some of his verbal utterances, as well as many of his table manners. After Mrs. Pearce basically runs down a list of things begging Mr. Higgins not to do them for the sake of Eliza's education, Higgins realizes that he has been under the scope of Mrs. Pearce all this time. This is when he tells Col. Pickering:
You know, Pickering, that woman has the most extraordinary ideas about me. Here I am, a shy, diffident sort of man. I've never been able to feel really grown-up and tremendous, like other chaps. And yet she's firmly persuaded that I'm an arbitrary overbearing bossing kind of person. I can't account for it.
All the time while Mrs. Pearce listed all the things that Mr. Higgins does- which are all wild and show a lack of class- Higgins consistently denies that he ever does any of that. From what we can gather as an audience, Higgins has a much higher perception of himself than he thinks. He only hides his obstinacy by calling himself derogatory adjectives, such as "diffident" and "shy".
However, a man who does all the things that Mr. Higgins does at the table is anything but shy and diffident. He just likes to do as he pleases, which is the main reason that he declares to remain a bachelor for the rest of his life. In Higgins we see someone that has lived by himself too long and has lost tact of what other people's perceptions are of himself. Hence, he has created an idea of what he is like, which contrasts tremendously with what Mrs. Pearce witnesses every day of her life under his employment.