Walter Mitty does not so much "retreat" into a fantasy as he lives in a fantasy world and is occasionally forced to cope with reality. He is bored with his life. He and his wife are "exurbanites." They are the kind of middle-class people who used to live in Manhattan but were starting to move out of the city for many reasons. For one thing, real estate was becoming prohibitively expensive in Manhattan. In those days people could buy houses in neighboring states for reasonable prices. Automobiles were becoming safer and more dependable by 1939 when this story was published. Cars were a necessity if people needed to live in country settings. Crime was becoming a huge problem in Manhattan, along with congestion, air pollution, and all the other problems familiar to those who live in big American cities.
Mitty and his wife are living in Connecticut, one of the first states to be invaded by New Yorkers fleeing Manhattan. Mitty must be a man like James Thurber himself who does not have to work in the big inner city but can work at home and travel to Manhattan occasionally by train. Many of the excellent stories by John Cheever are about the affluent people who live at some distance from Manhattan but, like Thurber, draw their money from the big city and have to remain within reasonable proximity. Here is how Thurber describes the Mittys' mode of living:
Walter Mitty began to wonder what the other thing was his wife had told him to get. She had told him twice, before they set out from their house for Waterbury. In a way he hated these weekly trips to town--he was always getting something wrong.
So the story is taking place in the town of Waterbury, Connecticut, a town about 75 miles north of Manhattan with a population of approximately 100,000 in 1939. Mitty is stuck in a house in the country outside Waterbury all week long with a woman he no longer cares for. Going into town is the big event of the week. Yet he gets no pleasure out of these trips. The fact that he is a poor driver is suggestive that they had been living in Manhattan for many years before deciding to get out of the city. Many people who live in Manhattan do not own cars because other forms of transportation are so readily available, including taxis. Therefore many Manhattanites are poor drivers or don't know how to drive at all. It seems likely that Mrs. Mitty does not know how to drive; otherwise she might insist on doing the driving--or else she might drive into Waterbury by herself.
There are advantages and disadvantages to almost everything in life. Living in the country is cheaper and more peaceful, but it can be boring--especially for someone who has been living in Manhattan. Mitty must spend a lot of time reading or gardening. He is bored and has fallen into the habit of indulging in fantasies. Suburban life is more appropriate for couples raising young children. It can be pretty deadly for middle-aged couples like the Mittys who are living in this modern Never-Never Land surrounding all our big cities, a land which is not city and not country.
Mitty's retreat into a world of fantasy is a flight from an external reality that is so devoid of satisfaction. Mitty's flight into a world of day dreams is fueled by a reality that is deadening. For example, he flies into his world of day dreams as a response to a world in which he is seen as useless, weak, or ineffective. In response to a world that sees him as a failure, he is a success in his dreams. When women laugh at him in real life, they fall all over themselves to be near him in his dreams.
Mitty's retreat into a world of day dreams is a direct response to the ineffectiveness of his own life. His desire to be something more than what he is becomes the base reason as to why his dreams are triggered. His retreat into this world is driven by the desire to live life different than what he does. The unsatisfying and emotionally deadening nature of his existence is the reason for his retreat into a world of fantasy that is something else. It is for this reason that Mitty retreats and goes into a condition of being vastly different than what he experiences.