I have always been under the impression that Reverend Parris does not really like his position in Salem. I don't know if he doesn't like being a minister, or if he doesn't like being a minister in Salem. If I had to pick, I think it is the latter of the two.
Early in the play, Reverend Parris and Abigail are talking about why Betty might be sick and about what the girls were doing in the forest. I've always gotten the impression that Parris is first worried about his reputation and second about his daughter. During the conversation, Reverend Parris says the following:
"Abigail, I have Sought here three long years to bend these stiff-necked people to me, and now, just now when some good respect is rising for me in the parish, you compromise my very character."
For me, that quote encompasses his dislike for the people of Salem. It's clear from the quote that he is worried about his reputation, but it is also clear that he doesn't particularly like the people of Salem. He refers to them as "stiff-necked people." That's not exactly an endearing phrase. He also mentions that he has been trying to bend them. Not lead, guide, or help them. Bend them. That sounds rough and painful, and things usually break when they are bent too far.