Why does Tom's defense of family life and traditional institutions amuse Nick?
Tom's lack of self-awareness is what seems to amuse Nick. How could Tom fail to realize that the values he is expressing in defense of the sanctity of family life are the same values he is trampling with his illicit affair (with Myrtle)? Also, Tom's intellectual limitations are clearly on display when he attempts to sound smart.
During Tom's tirade at dinner in Chapter I, Nick notes that "there was something pathetic in his concentration," suggesting that Tom is not at all in command of the ideas he is trying to discuss and probably not entirely capable of understanding what he is saying. The anger in Tom's tone, however, is thoroughly true to his character and perhaps more honest than anything he could say about racial superiority and eugenics.
These failings in Tom are recognized by Daisy and Jordan as well, as they make fun of Tom while he attempts to take on an "informed" point of view. It is no surprise then that later, in Chapter VII, Nick is amused as Tom takes on an informed and morally superior posture when talking with Daisy and Gatsby about family values.
"I suppose the latest thing is to sit back and let Mr. Nobody from Nowhere make love to your wife. [...] Nowadays people begin by sneering at family life and family institutions, and next they'll throw everything overboard[...]"
From a man who is cheating on his wife, toying with the husband of his mistress and who is bigoted on top of it all, the high moral tone that Tom strikes is exceedingly false.
"Completely without taste, culture, or sensitivity, [Tom] carries on a rather sordid affair with Myrtle Wilson. He pretends to help George Wilson, her husband, but allows him to think that Gatsby was not only her murderer but also her lover" (eNotes).
The fact that Nick expresses no surprise at the sudden moralizing attitude coming from Tom is suggestive of the low regard Nick has for Tom from the start. For Nick, Tom's statements on the sanctity of marriage and the values therein are completely at odds with Tom's character both intellectually and morally.
Tom is a "brute" and a violent man. His way of thinking is purely self-serving and there is no consideration, kindness or virtue ever attached to his character. Such a person taking on the claims of moral authority is preposterous and we can see this in Nick's inclination to laugh at the outrageous arrogance (and ignorance) of Tom's outburst.
Tom's defense of traditional and family values in The Great Gatsby amuses Nick because Tom is being so hypocritical. Tom spouts cliches like he's a Bible preacher who's always been conservative and has always placed his family first above all else:
Nowadays people begin by sneering at family life and family institutions and next they'll throw everything overboard and have intermarriage between black and white.
This from a man who is in the midst of an extra-marital affair. Not only is Tom hypocritical, but he is also bigoted and illogical. He demonstrates faulty logic--what does sneering at family life have to do with interracial marriage?
Nick sums Tom's speech up well:
Angry as I was, as we all were, I was tempted to laugh whenever he opened his mouth. The transition from libertine to prig was so complete.
When Tom has an affair it's okay, but when Daisy becomes involved with Gatsby, it's sneering at marriage and will ultimately lead to interracial marriage.