The open class/closed class system was proposed by Huddlestone in Introduction to the Grammar of English. In it, the "open class" items belong to a category of morphemes that be changed by adding or removing other morphemes to it as part of the process of word formation. For example, the lexical morpheme in a word such as unquavering is the morpheme that has a meaning of its own.In this case, it is the morpheme "quaver". The fact that you can "add" to a lexical morpheme means that such morpheme is "open" for changes, or "able" to be changed.
Contrastingly, the prefix "un", and the suffix "ing" operate as grammatical morphemes and they do a separate job to re-create the verb "quaver" into an adjective. On their own, "un", and "ing" have no meaning nor can they have anything added to them to convey a new meaning. Those morphemes are the agents of change but cannot be changed on their own. As a result, it is said that they are "closed class items".