The "why" of this question is more complex than the "what." It is part of what makes "The Zoo Story" so compelling.
Jerry never reveals a true motive. Is it that he is envious of Peter's lifestyle? Peter reveals to Jerry that he lives in the affluent East Side and Jerry tells him he resides in the slum-ridden West Side. Peter has a family; Jerry does not.
Does he want fame? Jerry makes a strange comment about the evening's headlines that is never truly answered. Perhaps he wants notoriety and is anticipating the reaction and attention committing murder will bring to him, even if from afar.
Is he simply psychotic or a severe personality disorder? There is evidence to support this argument as well. There is no reason for the psychological cat and mouse game, but plenty of dialogue to show Jerry enjoys the pursuit of his prey.