The first post is correct about Holden thinking about himself being a coward when dealing with other boys. This information is significant, however, because it is dealing with how he perceives himself and not how others may see him. It also brings up the issues of manhood that face boys as they are growing up. Holden says, "What you should be is not yellow at all. If you're supposed to sock somebody in the jaw, and you sort of feel like doing it, you should do it" (89). There are many times throughout the book where Holden doesn't do something, like calling Jane, because he doesn't feel like it. He is constantly conflicted between what he wants to do, what he thinks he should do, and what he actually does when making decisions. Making connections like these between the numerous anecdotes that Holden tells gives more insight to who he is and what his mental and emotional troubles are. It also helps to understand why he is in a hospital telling this story in the first place.
In Chapter 13 when Holden is walking back to the hotel, he is thinking about "his gloves and his yellowness".
On his way to the hotel, he remembers when his gloves got stolen at Pencey and he states that even though he never found out who the thief was, he would have probably done nothing about it.
Moreover, he creates an hypothetical scenario in his head where he would find out who the thief was, confront him at first, but then back down.
Lastly, he states that he is very "yellow" and that one of his greatest problems is the he never cares too much when he loses something.
That's a really quick and superflous answer, but I am pretty sure it's correct.
Hope it helps!