In The Swimmer society is presented as an on-going "get together" in which alcohol is always present. In the case of Neddy, we know that he was going downhill in debt, in his professional life, and in his family life. However, the story does not present it right away. Yet, we see that Neddy is indeed an alcoholic and uses alcohol to wash away his reality. As his illness progresses, and his obsession with swimming across the county becomes stronger, he would become more impertinent by crashing yard parties, getting heavily intoxicated, and moving on. In one of such "crashings", he finds out that alcoholism had nearly taken the life of a fellow peer, Eric Sachs. He could not remember this.
Hence, it is clear that alcohol is a self-medicated solution to the collective problem that seemed to be going around men like Neddy: They could not keep up with appearances and used alcohol to escape. In Ned's case alcohol aggravated his already-created condition, which came in turn as a result of him having lost everything.