Your question points to one of the enduring mysteries of Nathaniel Hawthorne's short story "The Minister's Black Veil" ("TMBV"). Reverend Hooper's congregation, so unnerved by his appearance in the veil, fail to place the symbol of the veil in the context of his sermon given moments after he appears in the veil. The sight of their gentle leader masked by a black veil has turned their world upside down, and their communal response, horrific as it is, precludes even the exercise of common sense:
'I don't like it,' muttered an old woman, as she hobbled into the meeting-house. 'He has changed himself into something awful, only by hiding his face.'
'Our parson has gone mad!' cried Goodman Gray, following him across the threshold.
A man whom everyone knew to be a gentle, loving preacher--one who "strove to to win his people heavenward, by mild persuasive influences"--has, by adding a veil to his face, become an object of suspicion and even horror.
Reverend Hooper's sermon, given just minutes after his congregation is seated (but still in fearful awe of the veil), takes as its subject
. . . secret sin, and those sad mysteries which we hide from our nearest and dearest, and would fain conceal from our own consciousness, even forgetting that the Omniscient can detect them.
Even though each listener seems to feel the effect of Hooper's sermon, perhaps aided by the power of the veil--that Hooper is able to detect each person's "iniquity of thought and deed"--no one appears to connect the sermon's subject (everyone has secret sins) with the symbol or emblem of secret sins, the veil itself. Even the village's physician remarks on the veil's effect on him, "a sober-minded man like myself," but even he fails to discern the relationship between the veil and the sermon because his intellectual capabilities are no match for the fear inspired by the unknown.
Much has been made of Hooper's failure to discuss the meaning of the veil with his congregation, and there is no doubt that he loses a "teaching moment," but the fact is, he clearly links the symbol of secret sins with his discussion of secret sins. But the superstitious fears of the congregation veil their own eyes and ears to Reverend Hooper's attempt to teach them that secret sins prevent them from truly knowing each other and understanding their equality in God's eyes.