In Act II when Hale questions John and Elizabeth Proctor about their faith, Hale raises the fact that John has not attended church very frequently. John responds with a number of different excuses before finally identifying what his problem with the church is. He firstly says that Elizabeth was sick over the winter and that when he was not able to go to church he prayed in his house instead.
However, the real reason emerges when Hale pushes John Proctor further, and he admits that his reasons for non-attendance are based around the Reverend Parris and how he insists on the trappings of wealth in his church, as evidenced through gold candlesticks. This is something that John finds difficult when he tries to pray:
It hurt my prayer, sir, it hurt my prayer. I think, sometimes, the man dreams cathedrals, not clapboard meetin' houses.
This quote identifies that there is a personal problem between Parris and Proctor, and that Proctor objects to the rather grandiose ideas and approaches of Parris. This is the real reason why he has not been attending church as frequently as was expected in that time.