In Shakespeare's time, people used the four humours (earth, wind, water, and fire - or- choleric, melancholic, sanguine, and phlegmatic) to describe all sorts of things. It was believed that each person had a little more of one than the others within them. These were the basic elements though that everyone had. A humour is essentially a trait of character.
In a comedy of humours, each of the four humours would be represented by a different character. The action of the comedy would demonstrate each character or humour's flaws and strengths.
In this scenario, it is important not to think of the word humour as funny, but as a trait.