What does Reverend Parris mean when he says, "There is a faction that is sworn to drive me from my pulpit"?
A faction is an organized group of people within a larger group or community which has a different point of view or opinion than the majority. Such a group fights for its ideals and the acceptance of its perspective. Reverend Parris believes that such a group, whom he further contends is led by John Proctor, Rebecca Nurse, and Giles Corey, are out to have him dismissed from his position as parish priest. The Reverend exposes this sentiment when he confronts his niece, Abigail Williams, whom he had discovered dancing in the woods with his servant, Tituba, and some other girls from the village, including his daughter Betty.
Abigail has informed him that there are rumors of witchcraft around and her uncle believes that his enemies will use the fact that his niece, his daughter, and his slave were all involved in the shenanigans that occurred in the forest. He is, after all, supposed to be the guardian of his community's moral and religious values and has to ensure that these are maintained. A rumor that so many members of his household were complicit in these inappropriate events will destroy his career.
Reverend Parris has been accused, on numerous occasions, that he is materialistic and uncaring. John Proctor, specifically, has criticized him for seeking the title deeds to the rectory (the first Reverend to have done so), and that he preaches "gold candlesticks." John has become so disillusioned by what he deems Parris's avarice and worldly rhetoric that he has stopped attending church. Rebecca Nurse also expresses concern that so few families bring their children to church.
When Rebecca Nurse also expresses concern that so few parents bring their children to church, Reverend Parris states that he does not preach to children, for they are not the ones who are "unmindful of their obligations to the ministry." In saying this, he is intimating to those who do not support him. Furthermore, it becomes apparent that Rebecca enjoys greater respect and stature in the community than he does. Reverend Parris resents her and sees her as part of the faction, and his dislike of Giles Corey stems from the old man's stubbornness and Giles's support for John Proctor's criticism of his ministry.
In the end, Reverend Parris's fears and insecurities drive him to obsession. He becomes so fixated on the idea of getting rid of his so-called enemies that he completely immerses himself in the court's proceedings (much to Judge Danforth's irritation) and repeatedly accuses John Proctor of various indiscretions. The Reverend's bitter resentment and animosity find reward in the executions of John and Rebecca and when Giles is pressed to death for refusing to confess. His victory, though, is an empty one because he loses all his life savings to theft, is voted from office, and eventually leaves the village.
Reverend Parris says that line to Abigail Williams in act one of the play. He is trying to stress to Abigail that Betty's catatonic state could ruin him and get him removed from being the town's minister.
Parris is a self-serving minister. He's got a giant "holier than thou" type attitude, but men like John Proctor and Giles Corey see right through it. They see that Parris is a greedy man. They know that appearance matters more to Parris than substance. Parris knows this about himself too, which is why he is worried. Men like Proctor and Corey have not been shy about their distaste for Parris. Other people in the town likely feel the same way, which is why Parris mentions that there is a "faction sworn to drive me from my pulpit."
Parris is worried that the people of the town will think that Parris's own household has witchcraft in it. The people of the town will then use that as evidence that Parris is not truly a man of God and should be removed from the pulpit. I tend to agree with Proctor and Corey, because at that moment in the play, Parris is more worried about his own reputation than his daughter.