There's no real basis, really. There is no real proof to support the accusations due to the nature of the crime as put across by Danforth. The supposed 'victims' accused people whom they had grudges against, or whose death would be beneficial to them in terms of increasing their own assets. For example, it is mentioned in the book that Thomas Putnam said that George Jacob's sentence allowed him to buy up his land.
However, the accusers think up stories wherein the accused has hurt them in some way, or that they have seen them with the Devil, or that the accused have taken a different form that only they can see and that is influencing them to collude with the Devil too. One example is where Ann Putnam accuses Rebecca Nurse of sending out her spirit and murdering her seven babies soon after their birth. Another is when Mary Warren turns on John Proctor, accusing him of forcing her to sign the book of the Devil. Abigail Williams also pretends that she can see Mary Warren who has changed her form and is threatening to harm her. The court believes all of this bullshit.
Those judged as being witches or having colluded with the Devil are accused by other members of Salem- especially the girls led by Abigail Williams- who claim to be victims of this so-called 'witchery'.
This point is very succinctly put across by Judge Danforth. He was putting down Rev. Hale's pleas that Proctor get a lawyer to present his evidence of the false accusations being turned up by Abigail and the other girls. Danforth identifies witchcraft as 'ipso facto', or an invisible crime. Since nobody can be witness to it but the witch and the victim, it is natural for the court to place belief in the victim's accusations since the witch of course will not come up and identify themselves on their own.
'therefore, we must rely on her victims- and they do testify, the children certainly do testify. As for the witches, none will deny that we are most eager for all their confessions. Therefore, what is left for a lawyer to bring out?'