Walter Mitty is older, retired, and he feels nagged by his wife, who constantly reminds him of forgotten appointments and errands. This is not necessarily negative; it is hinted that Mitty has some memory troubles, although he is otherwise able to function without help. His most important value is his personal space and time to think; his vibrant fantasy life comes from having most of his actual life regulated and controlled, so he envisions himself as "in charge" without anyone to tell him what to do. He even makes a point of this at the end of the story:
"I've been looking all over this hotel for you," said Mrs. Mitty. "Why do you have to hide in this old chair? How did you expect me to find you?"
"I was thinking," said Walter Mitty. "Does it ever occur to you that I am sometimes thinking?"
(Thurber, "The Secret Life of Walter Mitty," bnrg.cs.berkeley.edu)
Mitty wants to be left alone, to think, but he is constantly bombarded with information and noise from the people around him, who are unaware of his discomfort. Mitty wants only to have time for himself, not time for others or time to run errands; his fantasies are not desire for adventure, but desire for anything other than his regimented life. He doesn't necessarily want to be shot at in the war, but it would give him different mental stimuli and allow him to think about things other than his normal, everyday routine.