The outcasts are thrown out of the town of Poker Flat because they are the undesirables of the town.
Duchess, despite what her name implies, is not high class royalty in any way. She is the town prostitute; therefore, she is deemed a seedy character that should be expelled from the town.
John Oakhurst is a professional gambler. By all indications, he is a calm and impassive gentleman. My guess is that he won his poker games one time too many, and the losers wanted him gone.
Mother Shipton is an undesirable, because she is more than likely the madam of Duchess. Whether or not that is true can be debated, but Mother Shipton is cast out because of her close association with a known prostitute.
Uncle Billy is a confirmed drunk. He's a suspected thief too, but what is absolutely true is that he is selfish to the core. He escaped the coming storm and took the group's horses and supplies.