Albee's The Zoo Story examines how isolated modern man is in the way he lives and the role capitalism plays in this isolation. If we compare Peter and Jerry, we can see that each man is isolated, though Jerry is far more aware of this fact than Peter.
Peter is the quintessential family man: he is solidly middle-class and his life is going "according to plan." Peter does not realize that the trappings of his middle class life are a literal trap: he is caged by the comfortable theories and material items with which he surrounds himself. In order to not face his aloneness, Peter surrounds himself with things and ideas that prevent him from facing his true condition: an abyss of isolation and disconnection.
We can interpret the bench as symbolic of all that Peter has (and, thus, all that he fears losing). If he "loses" the bench, he loses his entire stable sense of himself and the world. Losing it to someone like Jerry is particularly troubling to Peter because, in Peter's view, Peter has done everything right: he has the marriage, the job, the family, and the money that are supposed to make him content—that are supposed to herald the accomplishment of the American Dream. That Peter has "followed all the rules" and still feels isolated elucidates Albee's view on the way we deceive ourselves because we are scared of truly seeing and knowing ourselves.
Jerry is Peter's foil: he is lower-class, lives in a "bad area" of the city, and desperately seeks human connection with an overtness that Peter finds unmannered. For Jerry, the world is a zoo: everyone is separated from everyone else, as though we are different species living in our own separate cages. Not only are we separated from one another, but we are also separated from our own basic nature—the desire for connection.
Jerry is the symbol of the isolated modern man, but he also fights men's separation from each other. He wants to know the "essential nature" of the human condition, as his experiment with Cerberus reveals. Jerry's failure to get Cerberus to like him (despite bribing the dog with hamburgers) represents how connection is not something that can be purchased, nor can it be easily accomplished. Thriving in a capitalist society means only that you have money; it does not guarantee real contact and connection. Jerry's death allows Peter to learn this lesson, and Peter must live the rest of his life really seeing the people his middle-class life has, until now, protected him from.