In Act One, Parris, Proctor, Giles and Putnam all fight and argue over numerous things--firewood, salaries, sermon topics, land boundaries, lawsuits and loyalties. Parris himself reveals his sensitivity and insecurity in several areas. First of all, he feels that people in the town don't like him, that they are always rising up against him and trying to halt his progress in the town. Secondly, he feels like he isn't getting enough money or respect; he wants a larger salary, the deed to the church, and for all of the townspeople to respect him more. So, when Proctor criticizes Parris's sermon topics as being "only hellfire and bloody damnation," it strikes a nerve. Parris immediately jumps in to defend himself, stating that Proctor has no right to determine "what is good" for him to hear in sermons, and that Proctor "may tell that to [his] followers." In this, we see how sensitive Parris is to criticism, and that he feels like Proctor, and others, are gathering parties of opposition against him.
Reverend Parris is very ready to have "outbursts" because he is highly insecure about his position in the town, and feels himself the victim of unjust practices and opinions. As a result, he wastes no opportunity to voice his victimized position in society, to make sure that everyone knows his position. I hope that those thoughts helped a bit; good luck!