The reason for Jimmy's anger against people and the Church is that when he was a young boy, he sat with his father to keep him company while he was dying. It took six months for his father to die and Jimmy was there with him daily. This is explained in Act II after Jimmy Helena and Alison get ready for church and Jimmy plays the trumpet before discussing Hugh's mother who gave the money to start the candy store, an ironic choice of business for someone as bitter and soured on life as Jimmy.
The reason Jimmy has outbursts of anger toward Alison is because of what he calls her "fence-sitting." She was born and raised in the upper classes and she isn't interested in ridiculing and changing class distinctions like Jimmy is. He thinks of this lack of ill will toward upper classes as "fence-sitting," an inability to understand and take a strong position against injustice.
Jimmy is angry at the Church because it is part of the Establishment power structure that is set up by the upper class people and controlled by them. Jimmy hates all things that depend upon social class distinctions and have power over people from the lower classes. Cliff doesn't fit into this upper class hatred, Jimmy truly likes him, but Jimmy still gets angry at him and insults cruelly because Cliff somehow isn't good enough.