In Chapter 3 of Steinbeck's novella, Of Mice and Men, Carlson, a big man, knows that Curley selectively chooses those whom he punches; therefore, he insults Curley by calling him a coward in a manner that "puts him in his place," as the expression goes when a person desires to let another know that he is aware of what that person truly is. He tells Curley that his attempt to "scare" Slim has failed, so Curley tries then to intimidate him, "You keep outta this les' you wanta step outside." But, Carlson counters by saying he will kick off his head.
And, then, when Curley is further insulted by old Candy, he bypasses the old man to place his focus on a new man, Lennie, in order to regain his position as being tough: "What the hell you laughin' at?" Poor Lennie is caught unaware, "Huh?"
... Curley's rage exploded. "Come on, ya big bastard. ...No bit son-of*****is gonna laugh at me. I'll show ya who's yella."
When Curley sees Lennie looking "helplessly" at George, he quickly takes two jabs at Lennie's face and hits his nose hard, feeling now he has proven that he is not "yella."
Colors have differently meanings, based on culture and/or religion. Obviously, frogs' bellies are really yellow. Yellow can mean happiness, joy, energy, intellect, but in this situation, it means impatience, criticism, and cowardice. So he's basically calling him a coward or that he has no guts (which basically means the same thing).
That phrase just means that someone is a coward. It's an extremely country phrase haha. Carlson called Curley a coward in order to make him angry. I think the phrase "yellow bellied" is because yellow is a color that represents cowardliness and your belly is where your guts are. So if you're yellow-bellied that means you have no guts.
In J. Steinbeck's Of Mice and Men, we find a quote "You're yella as a frog belly" (p. 62). It means 'You're a coward', which Carlson says to Curley in order to provoke him.
There has been a rumor spread about Curley keeping one of his hands in vaseline. When Curley is challenged by Carlson his temper is roused. Carlson threatens him:
'You come for me, an' I'll kick your God damn head off.'
Candy is also mocking Curley, but having one hand makes him no fair match. It is then when Curley sees Lennie as a fairer physical opponent, even though he is clearly not a mental match.