In The Picture of Dorian Gray there is an added tendency to correlate the East End of London, a place where anything goes, to people who completely detour from Wilde's idea of aesthetic. As the previous post accurately shows, Basil and Harry represent two different views of such movement, allegorically representing Ruskin and Pater respectively, and both having served as mentors to Wilde. Therefore, all the sleazy, swarthy, shady, and strange will undoubtedly be prescribed strictly to the East End scenes. There, we have the gypsies, the uncultured, the bejeweled Jew, the swarthy likes of James Vane, whose appearance made others speak. All of these descriptors attempt to describe that class as a whole.
Meanwhile, in the "right" society, the rich and artsy West End, men and women hide their shady, swarthy, strange, and sleazy alter egos under the mask of virtuosity and propriety. Just like Dorian hides his corruption behind the cursed picture sitting on top of his mansion, society hides their deadly sins masking them under the philanthropy movement, the elaborate balls, the fashionable teas, and by snubbing those considered beneath them in class.
Yet these whispered scandals only increased in the eyes of many his strange and dangerous charm. His great wealth was a certain element of security. Society--civilized society, at least--is never very ready to believe anything to the detriment of those who are both rich and fascinating.
In a dramatic contrast, the scandals of those of doubtful origins, being James Vane, Sybil's mother, Sybil herself, and whoever else belongs to the "lower classes" are to be put on public display; Sybil's shameful suicide was first page news; her mother was the subject of gossip, as she had an affair with a married, upper-class man who probably fathered Sybil. In all, anything that is ugly and off the path of what is considered ethnically, aesthetically, and even culturally "fashionable" will involve a description that instills ugliness of some kind.
It feels instinctively that manners are of more importance than morals, and, in its opinion, the highest respectability is of much less value than the possession of a good chef. And, after all, it is a very poor consolation to be told that the man who has given one a bad dinner, or poor wine, is irreproachable in his private life.