As the story opens, Michael Loomis and his wife, Frances, walk along Fifth Avenue toward Greenwich Village on a sunny Sunday morning in November.  Frances is happy to be spending the day with her husband, but she notices him whip his head around to follow the movements of a pretty woman. She attempts to nonchalantly scoff at him, telling him he’ll break his neck if he keeps doing that. They have a brief discussion about the woman. Michael defends himself by saying she had a complexion more fit for the country, and it surprised him to see her type of beauty in New York City.

Attempting to move past her husband’s wandering eyes, Frances expresses her joy at having had a lovely morning thus far with her husband. The couple had slept in and had breakfast together. Frances is excited for the remainder of the day, telling Michael that having breakfast with her husband “makes [her] feel good all day.” Frances suggests that the two of them spend their Sunday together, just them. They are often busy seeing friends or attending social events and thus rarely have extended time alone. She begins to suggest they go see a football game, then have a steak dinner, some wine, and attend a French film at the theatre. 

Her proposal becomes derailed when she catches Michael looking at another woman as they walk along the street. She wonders if he had even heard the plan she suggested, and though he heard her, he was obviously distracted by a dancer walking past. Upset, Frances asks him if her itinerary works for him or if he would rather just walk up and down Fifth Avenue gawking at women.

During this confrontation, it is revealed that Michael stops to admire women constantly. Whether they are in a restaurant or a lecture, his eyes wander. Michael insists he is a happily married man. He relies on his wife and enjoys the life they have together. He asks if she wants a fight, and she feels poorly for starting the conversation. 

Frances tells her husband that she has not laid eyes on another man since they became attached. Again, Michael insists he is happy and faithful. They stop in a bar to drink brandy and have a candid conversation about his tendency to cast admiring glances at other women. It is made clear that Michael is approaching middle age, and Frances is concerned that his interest in other women will lead to his being unfaithful to her. Michael readily admits that he enjoys looking at women but says that he has never been unfaithful. After a couple of brandies, Frances presses Michael, and he freely admits that he sometimes thinks that he would like to be free to experience other women.

Frances cries for a few minutes, and then pulls herself together and tells him that she doesn't want to hear him say complimentary things about other women's looks. She walks away from their table to call friends with whom they will spend the day. As she walks away, Michael thinks to himself that she is a “pretty girl” whose legs look “nice.” He seems to be ogling her in the same way he did the other women. The story ends on an unresolved note: it seems that his behavior and the hurt it causes is likely to continue.

“The Girls in their Summer Dresses” chronicles the origin and conclusion of a married couple’s quarrel as they walk through lower Manhattan on a Sunday morning in November. Michael and Frances Loomis have left their apartment and are proceeding along Fifth Avenue toward Washington Square. As they are walking along the crowded thoroughfare, Frances observes that Michael has turned to look at a pretty girl and remarks good-humoredly about it. Michael, who seems unaware that she has previously noticed his habitual girl-watching, also makes light of the incident. He explains that the girl’s complexion drew his attention to her, a country girl’s complexion seldom seen in New York.

The conversation, sprinkled with jokes and patter, suggests that so far they have had an enjoyable weekend. Frances expresses a need for more time with Michael and urges that they call off a previously planned country outing with friends so that the two of them can spend the day in the city together. After Michael readily agrees, she begins making plans aloud for his approval, while they continue their walk. They will attend a Giants’ football game, have a steak dinner at a famous restaurant, and go to a film.

As Frances is making her plans, selecting the activities that she knows her husband will enjoy, Michael’s eyes stray to another attractive girl, and this time Frances is unable to conceal her frustration and dismay. She intimates that because he is so interested in the girls, he might prefer to spend the day walking along the avenue. Michael’s point of view is that he takes only an occasional glance. Further, he tells Frances, there are few really attractive girls to be seen in the city. Frances dismisses this conclusion, making it clear that she considers Michael’s behavior habitual and ingrained. Indeed, she can describe not only the frequency of his girl-watching but also his manner of looking and the time he spends on each. The effect of the episodes on Frances is to increase her insecurity, and, as the story progresses, she reveals deepening anxiety and resentment. She takes no comfort from his insistence that he is happily married, for she believes that he looks at every woman who passes with the kind of look he once gave her. In reality, Michael takes pride in his wife but enjoys watching other women. To Frances this seems both contradictory and threatening.

Sensing the beginning of a quarrel that will ruin their day, Michael suggests that they have a drink, even though it is not much past breakfast. Frances rejects the idea and changes the subject temporarily. When they reach Washington Square Park, they decide to walk among the people there, but soon Frances’s insecure mood returns and she begins talking about Michael’s annoying habit. In an effort to reassure her, he claims that his habit is harmless and that he has always been faithful, but Frances remains troubled and gloomy. After a brief time, she agrees that they should go for a drink.

At a bar on Eighth Street the couple order brandy from a Japanese waiter, having decided that brandy is the proper drink to have after breakfast. As they drink, the conversation returns to the troublesome subject, and Michael now admits that he enjoys girl-watching. After ordering a second brandy, he becomes expansive and loses all restraint. He talks of the countless beautiful girls in New York, classifying them as to places they are found, professional types, racial and national types, and girls who belong to different seasons, among them, “the girls in their summer dresses.” Frances believes that he wants the women, and Michael, now no longer caring, acknowledges that he does. When she pathetically declares that she, too, is attractive, he agrees that it is true. When she suggests that he would like to be free, he hesitates and then admits that at times he would. Now weeping, she presses him further, believing that someday he will “make a move,” and Michael replies, after further hesitation, that he will. Regaining some of her composure, Frances asks that he not talk to her about the attractiveness of other women and Michael acquiesces.

They call the waiter and, to his astonishment, order a third round of drinks. No longer intent on spending the day alone with her husband, Frances suggests that they telephone their friends, who will take them for a drive into the country. After Michael agrees, she walks to the telephone, and he watches her, thinking, “What a pretty girl, what nice legs.”