This conversation occurs in Act II of this play, where Hale visits John and Elizabeth at their house before Elizabeth is taken away for her supposed involvement in witchcraft. What John Proctor tells Hale is that he doubts the accusations of witchcraft are actually real, as he doubts that there are any witches amongst the inhabitants of Salem. He of course knows that, as Mary Warren says, it was just "sportin'", rather than any truthful and accurate reflection on the level of witchcraft in Salem. Note what he says to Hale when Hale mentions how he has heard a rumour that John Proctor does not believe in the existence of witches:
I have wondered if there be witches in the world--although I cannot believe they come among us now.
This of course implicitly attacks Abigail and the girls and argues that their many accusations are not truthful at all. This is significant in the way that it ties in with the suspicions that Hale is beginning to have about Abigail and the veracity of her claims. It also foreshadows the conflict that will occur in Act III when Proctor goes to court in order to testify against Abigail and argue for his wife. John later backs this up by telling Hale about the conversation that he had with Abigail in Act I where she admitted to him that the claims of witchcraft were false.