Paul is deeply alienated from the mundane reality of his life. At the same time, he is gifted with an extraordinary imagination and a flair for performance. The theater, and the actors there, seem to him to be examples of a kind of life he would want. He craves their acceptance and their apparent freedom. He also comes to believe that his identity is a matter of performance and that he can "be" whatever role he chooses.
His decision to steal the money and take off for New York is an outgrowth of that realization. Through this action, he has made his fantasy into reality. The comforts of his hotel room, his new clothes, and the city around him fill him with a sense of finally being the person he was meant to be. However, the trappings of wealth do not ultimately satisfy him. While they might provide a reassuring picture of him in his new identity, they do not in and of themselves provide that identity. Instead, they give Paul the opportunity to explore that identity for himself.
His suicide is an outgrowth of two things. First, his night on the town with the rich boy from Yale ends badly. While the story is not explicit about this, it's clear that Paul had hoped that this might be a romantic relationship, and the fact that it ends badly suggests that Paul's advances have been scorned. Second, news of the theft comes to him in New York, along with the knowledge that his father is on his way to retrieve him. Paul is completely alone, and the idea of returning to Pittsburgh in humiliation is too much to contemplate.
What links these three things is Paul's sense of emptiness and a desperate desire to explore his sexual identity. Once having become the person he feels that he is, the idea of turning back is impossible.