It is often hard to fit John Irving novels into the category of Literary Fiction because they are so popular. Here are a few ways I would try to justify it.
Firstly, literary fiction involves characters who have some sort of psychological depth and take inner journeys.
Literary fiction would also contain more figurative language for example metaphors and other types of imagery. There is also a fair amount of irony in the novel and play on words. The fact of Garps name is an example of this kind of humorous play on words.
Popular fiction tends to have mostly flat characters who do not develop nor is the reader interested in the character in depth as they are generally a type which is understood by all. Irving's characters are more complex than that and he also has a multitude of complex characters in the novels he writes.
Finally, popular fiction tends to be plot driven and concerns itself with the basic format of exposition, rising action, climax and resolution. "The World According to Garp" contains this basic structure but what makes it more literary are the depth and the many side stories which run through the main narrative.
These are some arguments I would use to justify "The World According to Garp" as literature.