In Chapter 1 of The Great Gatsby, Nick's description of Tom as a man who is so rich that he can afford to bring a string of polo ponies with him wherever he goes demonstrates that Tom does not live in the same world as others. Because Tom is used to doing and getting whatever he wants (maintaining mistresses, covering up crimes and affairs, etc.), his morals do not match society's. As long as he can move away from "wrongdoing" or cover it up with his money, Tom's conscience is clear. He feels free to spout racist comments at the first dinner party because who would dare to contradict him? This obvious disregard of morality is what allows Tom and Daisy to remain "careless people" who go around smashing up people's lives and then retreating into their cocoon of wealth.
Unfortunately, not much has changed in America's wealthy sector. One often reads about young heirs and heiresses who "collect" DUIs, drug charges, tax issues, and other legal problems that would put many of us in debt or prison (or both) for many years but seem to have little influence upon or consequence for the wealthy. Their sense of morality seems to be based on what plays the best in the tabloids or what causes them to have to perform the most damage control--not upon a basic sense of right and wrong.