It is actually Maria, Olivia's servant, who says "I am as mad as he, if sad and merry madness equal be." Maria wishes to trick Malvolio into thinking Olivia is in love with him, and so writes him a note in Olivia's hand, suggesting he engage in a number of strange behaviors which Maria knows Olivia will find annoying.
What Maria means is that, is that Malvolio's apparent "madness" (his odd behaviors he's displaying for Olivia's benefit, mistakenly believing he will win her heart) will end up "sad" as he will end up alienating Olivia further. But Maria, whose prank will wind up hurting Malvolio and making a fool of him, is "merry" and excited by the success of her trickery, which she is comparing to the same "madness" in Malvolio's behavior.
Malvolio is not really "mad" but only wants to please Olivia; but ironically, after he realizes he'd been made a fool of, he does lose his temper and have a breakdown, so as to appear even more "mad" than before. It is a subplot designed both for humor and pathos: as Malvolio is an unsympathetic character, but the cruelty of Maria's trick in the end does elicit some sympathy towards him.
but it is olivia who says this line, not maria, in line 14 of act 3 scene 4