It is widely believed by the local townsfolk that Mr. Hooper committed some sort of secret sin with a young lady who's recently passed away and over whose funeral Mr. Hooper himself presided. It's never actually spelled out, but it would appear that the secret sin in question is some kind of affair.

A couple involved in the young lady's funeral procession engages in a spot of speculation concerning the alleged relationship between Mr. Hooper and the girl he's just buried. One of them reckons that she's just seen the spirit of the young lady walking hand in hand with the minister. This would appear to indicate a belief that the two were a little too close to each other than they ought to have been.

For a man in Hooper's position, a man of God, not to mention an engaged man of God, having an affair would be a very grievous sin indeed. It is further speculated that his overwhelming guilt over this affair is the reason why he goes around wearing a creepy, mysterious-looking black veil.

Mr. Hooper's not telling, of course, and indeed takes the secret with him to the grave, but it's not unreasonable to surmise that he did have an affair and that the black veil is an outward sign of the overwhelming guilt he feels about it.