Masquerades were costumed balls held in the Renaissance for the upper classes. The festivities included elaborate dances and people wore equally elaborate masks, or "visors." The Venezian mask, a very elaborate mask, made its way into Verona, Italy. Masquerade balls were sometimes set as a game in which the guests were unidentifiable and people had to guess who were the guests. They were also opportunities for people to secretly engage with one another; often people of different social classes would sneak into these masques or meet another masqued guest there.
Fortunately for Romeo, the ball for Juliet is a masquerade since he and Benvolio are able to be admitted. Juliet, like the other women, wears high heeled dress shoes, for Lord Capulet speaks of this in Scene 5 of Act I:
Welcome, gentlemen! ladies that have their toes
Unplagued with corns will have a bout with you. (1.5. 15-16)
There is no mention in the text of either a "masquerade" or a "ball." These are terms to describe certain types of parties that we have come to associate with Capulet's party, but that don't really bear any direct relationship to the text.
In Act I, when Capulet instructs his servant to go about town inviting those on his list, he merely tells him to say to them that his "house and welcome on their pleasure stay." When this same servant has Romeo read the invitation list for him, he calls the party "supper" and Romeo refers to it as a "feast." Later, just before he crashes the party, Romeo calls it a "masque," and Mercutio asks for a "visor" to put his "visage" in. A visor was a mask worn often a parties (especially by men) to conceal their identity, but not in the sense of dressing in a costume as we think of a costume or masquerade party today.
A masque was not a masquerade in Shakespeare's day. Here's the Enotes definition:
The masque was a form of festive courtly entertainment which flourished in sixteenth and early seventeenth century Europe. . . .Masque involved music and dancing, singing and acting.
Another thing to be aware of is that there is never a requirement when it comes to costuming a play. Each production has many things to consider that affect the costuming -- time period in which the play is to be set being a prime consideration. There have been many productions of Romeo and Juliet, for example, not set in 16th century Italy. For the masks or visors mentioned by Mercutio, productions have used everything from traditional Venetian Mardi Gras masks to Halloween masks to sunglasses. It all depends on the staging of a particular production.
So, though we have come to think of this party given by Capulet as a ball, there is no reference to this in the text, and costuming is not a matter decided by the script but by those who stage it.