The major fear that Parris has is of losing his position in society and having people not respect him anymore.
He is the minister for the town, which was a very important position in Puritan New England. As such, he is in a very visible position.
When his daughter seems to be all paralyzed and people are saying it's witchcraft, he becomes very worried. If the town minister's daughter is involved in witchcraft (or is even possessed by Satan for some other reason) the minister is going to look bad and people are not going to respect him anymore.
We can also see that he's really concerned about his image and reputation because he gets so angry when Proctor and others say anything bad about him.