I just want to add a little more to janeyb's excellent answer.
The west side was the poorer side in the 1950s. Jerry doesn't like it because he feels the pressures of society to be a part of the upper class is the cause of his suffering. This belief by Jerry connects to the theme of his loneliness and feelings of being isolated. Jerry asks Peter, "Say, what's the dividing line between upper-middle-middle class and lower-upper-middle class?" Jerry belongs to neither of the classes, and the illusions he has about Peter's life on the east side are very near to what Peter's life actually is. By going to the east side, Jerry can blame Peter for all of the suffering he has felt in his life because Peter symbolizes the upper class that Jerry can never be a part of.