Reverend Parris is most concerned about his reputation. Audiences are very aware of this concern of his early in the play. His daughter, Betty, is sick on the bed, and nobody knows what is wrong with her. His number one concern should be Betty, but instead it is his image and reputation among the people of Salem.
Parris: Now look you, child, your punishment will come in its time. But if you trafficked with spirits in the forest I must know it now, for surely my enemies will, and they will ruin me with it.
A few moments later he follows up the previous comment with this one.
Parris, studies her, then nods, half convinced: Abigail, I have sought here three long years to bend these stiff-necked people to me, and now, just now when some good respect is rising for me in the parish, you compromise my very character.
The entire conversation between Abigail and Parris is focused on how the events in the forest reflect on him. He just isn't as concerned about Betty.
What's very interesting about this play is that John Proctor is also very concerned about his reputation among the people of Salem. A difference between Proctor and Parris, though, is that Proctor is willing to ruin his good name to save the lives of other people. That's exactly what Proctor does, too. He admits to an affair with Abigail in order to save his wife and discredit Abigail's testimonies. Parris, on the other hand, is still focused on his own reputation, which is why he tells the court to dismiss John's words. He warns Danforth, “Beware this man, Your Excellency, he is mischief.”