Concerning modernism and postmodernism, I'll deal with two mistaken notions in your thinking that your question reveals. These notions are fundamental to any genuine understanding of an answer to your question.
First, there isn't anything like three broad categories of Western literature. There may be two--classical and romantic-- if you interpret everything that is not classical as romantic. But the three you mention do not define or include all of Western literature--not by a long way.
First of all, classical, strictly speaking, refers to literature of antiquity: primarily, for Westerners, Greek and Roman literature. The Renaissance was an attempt to rediscover and imitate classical Greek and Roman culture, literature, etc. But the products of the Renaissance are products of the Renaissance, not classical products. Shakespeare is an English Renaissance writer, or an Elizabethan writer--not a classical writer. In English literature alone (and "Western" means much more than just English), numerous "categories" exist: Anglo-Saxon, medieval, Renaissance, metaphysical, neoclassical, romantic, Victorian, modern, and postmodern. And I skipped some. Postmodern is simply the name scholars have thus far given to the period that appears to have followed the modern. We may still be in the postmodern period, and no doubt the name the period is known as may someday change. Whatever it is known as, though, it is just one period of many. The three categories you mention leave out many literary periods, and postmodern is one of many periods, not just one of three.
Secondly, you may be misunderstanding these "categories," as you refer to them. No one wakes up one morning and decides to "invent" a new literary period. Writers react in some way to what has been written before them. Each literary period is a reaction to what came before it. In retrospect, later on, looking back, scholars and teachers and artists, historians, etc., eventually categorize or label literary periods. The Renaissance attempted to recapture the classical, thus the name, which means rebirth. Neoclassical writers did something similar, thus the name. Victorian writers wrote about certain issues in certain ways (I'm being simplistic) and during the reign of Queen Victoria, and thus the name.
Postmodernism, then, is just a name given to one literary period of many. In the 1920s and following WWII, in particular, writers reacted to what had come before them. The name thus far given to what they did and are doing is postmodernism.
I liked the last part of the question. The modern period was not "modern" enough for these thinkers. While the modernists went a great deal in questioning structures of authority and understanding, I think that the post modernists wanted to go one more level of question everything, such as philosophical foundations, epistemological premises, and levels of consciousness. I think that you will find post modernism to be much more philosophically driven to prove "nothingness" than modernism was. The experimentation with form, style, irony, as well as ensuring that moderism's emphasis was present helps to differentiate postmodernism from its predecessor. I would also suggest that the exploration of new narrative voice and form becomes a part of postmodernism. Whereas modernism stressed a sense of entropy in what was once taught, it still did so through traditional means and traditional mode. The postmodernist seeks to expand narrative voice and seeks to bring about a new mode of expression that, simply put, the modernist could not have envisioned. I think that this might be where there is some differentiation between the two movements. They are extremely similar, but with some differences between them.
you seems to say that modernism is the finale. never, there should be beyond modernism. this so calld beyond gave the birht to postmodernism. it is very important to note that this postmodernism also is not the finale. beyond this, today we speak of post-postmodernism. i remind you that this will not be the finale. there is a chance of another birth.
postmodernism is nothing but breaking the conventional ideas that were existed from the begining. the postmodernist try to break the formulated and well constructed ideas or practises. they are also trying to establish the idea of the centre which has no place in postmodernism. uncertainity is the main theme of the isms.