Hola!

The Swimmer is a story about a man called Neddy Merrill who was rich, powerful, had friends, a mistress, and lots of social contacts. He is quite egocentric and he has a fixation with swimming through the world.  He feels that the best way to do this is by swimming through his neighborhood pools.

Right there, you realize that he is kind of crazy, but it could be just that he is eccentric. However, the plot thickens.

As he swims every pool, he becomes more disruptive. Every time he enters a new pool it is as if the people around him are beginning to be cold to him. Some of the pools he expects to find along his path are drained out, or the neighbors have moved out.  

The narrative does not say it flat out, but we do realize that something wrong has happened to Neddy, and he is either not aware of it, or he has forgotten, or he has gone crazy. We find this out when he visits the Hallorans, and Mrs. Halloran tells Neddy that she is "sorry" to hear of his misfortunes. Even his mistress rejects him.

Now is when we realize that Neddy had gone bankrupt, his family abandoned him, he lost everything, and somewhere in his mind he buried it all, substituted reality with a fixation of swimming the world, and went to live in this fantasy, leaving everything else behind.

Only as he realizes it, he runs back to his home only to find nobody there. He had been living in insanity.