The people who stood in front of the stage were called "Groundlings" during the Elizabethan era, & the name has carried through the centuries. The name derives from the fact that the patrons stood on the ground, rather than sitting in the seats of the balcony.
People who sat in the balcony were typically upper-class members of society who could afford to keep dry & clean while watching the play. The groundlings paid a penny to stand in "The Pit", also called "The Yard", just below the stage and watch the play. Standing in the pit was uncomfortable, and most times people were packed like sardines, keeping each other standing. Also, they were exposed to the elements, as the theatre itself was not covered.
Shakespeare used the term in Hamlet; the title character uses it as a derogatory label for the poorer members of the theatre audience. Also, a very popular LA-based improv group has been using the name for about 35 years, carrying on the classic theatre tradition.