Holden has issues with sex and manhood, and both are issues when he learns that Stradlater and Jane are going on a date. Since he can't communicate his disapproval of the match to them or himself, he lashes out in childish behavior, knowing full well that he will be punished the bully Stradlater.
First, Holden does not want the adult world to encroach on his childhood innocence. He's in a state of maturation denial. And he doesn't want others to reach adulthood either, especially a childhood girlfriend like Jane. Stradlater clearly believes the opposite: he uses sex as a means to experience the adult world sooner than he should. So, when Holden learns that Jane is about to cross the threshold from childhood to adulthood by losing her virginity to Stradlater, he expresses hostility.
Holden also is a masochist. He likes to see himself be victimized by other males in an effort to protect children and girls. He's playing the role of James Castle here: a chilvarous defender of innocence. Later, Holden will get beat up by Maurice because he refused to pay Sunny. In both cases, Holden likes to see himself bloody: it's a overture to suicide. He's a character in his own sadistic gangster film in which the bullies of adulthood beat up the innocent victims. Again, all of this is repressive behavior in an effort to remain in childhood.
In The Catcher in the Rye, Holden starts a fight with Stradlater after Stradlater returns from a date with Jane. Holden really likes Jane, and he becomes enraged when Stradlater starts bad-mouthing her. This is the outward reason for the fight between the two of them; however, it is likely that Holden has some deep-seated rage regarding Stradlater. First, Holden knows--even though he does not admit--that Stradlater is much more attractive than he is, so he feels a bit jealous that Stradlater is able to use his good looks to attract girls. Holden does not feel like he is a match for Stradlater, so he never even attempts to ask Jane out on a date. Next, Holden has much respect for girls, but Stradlater does not, so Holden is angry that girls still choose to go out with Stradlater even though he carries a low regard for them. So, Holden is generally angered by the fact that people are able to get by quite well on their appearance rather than on the merit of their character.