In Sir Gawain and the Green Knight, the challenge posed by the Green knight presents a challenge to the honor of King Arthur's court. Notice that the challenge is important and fussed over before anyone, except the Green Knight, even knows what the challenge is.
A challenge not only calls into question the knights' masculinity, but their character and honor as well. It's a question of living up to the chivalric code, or chivalry. The knights have to be chivalrous, filled with honor and a sense of duty and loyalty and bravery, and a sense of right and wrong, good and evil.
To fail a challenge would be a great failure.