The cause for concern in the Parris household is the strange condition that Betty Parris has fallen into after dancing in the woods and engaging in some childish form of witchcraft. Ever since that point, she has not stirred or woken up, being unreceptive to any plea or request. This is made much more alarming by the rumours that are circulating about her, as Abigail reports to her uncle:
Uncle, the rumour of witchcraft is all about; I think you'd best go down and deny it yourself. The parlor's packed with people, sir.
Because Betty appears to have nothing physically wrong with her, but is curiously unresponsive and insensible, the concern is that her condition is a result of some witchcraft of work of devilry. Given the setting and the context at the time, which placed a lot of emphasis on the devil and witches, this is a very dangerous position for the family of a pastor to be in. Parris is clearly concerned, and perhaps it can be argued he is concerned more about his position and how it will be impacted by these rumours than he actually is about his daughter.