The plot of the Swimmer is strange but simple. Neddy Merill, an affluent man of middle age visiting friends after a weekend of drinking decides he will swim his way home through all the neighborhood pools. He is excited at the prospect of his plan and considers himself a brave explorer and adventurer.

The conflicts of the story are gradually and subtly revealed. Neddy's view of himself and his life begin to unravel. The reader notices that time in the story seems distorted from Neddy's perspective. There are indications that the seasons change in the course of his journey. In addition, his physical strength and vigor atrophy, as his swim trunks start to feel loose and he struggles to climb out of pools. Also, in his encounters with various neighbors, including a former lover, the neighbors's dialogue points to personal struggles in Neddy's life.

At the climax of the story, Neddy arrives home during a storm and finds his house locked. Ultimately the story can be read as an allegory for alcoholism. Neddy's immersion in water functions as a symbol for his drinking, and the story's distortion of time indicates the time lost and the damage wreaked on Neddy's life by addiction.