The third person point of view can be either omniscient or limited. An omniscient third person perspective means that the narrator can reveal the thoughts and feelings of all of the characters in a story, while a limited third person point of view only discloses the thoughts and feelings of one character. In "The Swimmer" the third person point of view is limited, as the narrator only reveals Ned's thoughts and feelings.
In this case, the effect of the limited third person point of view restricts our understanding of Ned's life and what's happened to him, because Ned is an unreliable character. As the story progresses, it becomes clear that he is confused about his past and his environment. He doesn't remember things about the people living in the suburbs that he's swimming through, like when the Levys visited Japan, or that the Welchers have moved and sold their house, or when he had an affair with his mistress: "They had had an affair last week, last month, last year. He couldn’t remember." He also doesn't know what Mrs. Halloran is talking about when she mentions his recent losses.
As he was pulling himself out of the water he heard Mrs. Halloran say, "We’ve been terribly sorry to hear about all your misfortunes, Neddy."
"My misfortunes?" Ned asked. "I don’t know what you mean."
"Why, we heard that you’d sold the house and that your poor children . . .”
“I don’t recall having sold the house,” Ned said, “and the girls are at home.”
It's evident that Ned has suppressed his memories of the past, particularly those related to the collapse of his idealistic suburban life. Knowing this, it's difficult to understand what exactly is going on with Ned through the limited third person point of view. We have to piece it together through dialogue and exterior action as we aren't granted access to the thoughts of characters outside of Ned who have a more reliable perspective of the reality of his situation.