The congegration is at first greatly taken aback by the sight of Mr Hooper's black veil. In fact they are so 'wonderstruck' that they are unable to respond to his 'kindly' greetings as he makes his way to the church for his first sermon there. The people are perplexed by the veil not just because it is such an unusual thing for a minister to wear, but also because in every other respect Mr Hooper's appearance is entirely ordinary. This makes the veil stand out all the more and renders it quite grotesque. It is the only 'remarkable' thing about the otherwise quiet and unassuming Mr Hooper.
The people's initial astonishment rapidly deepens to a real disquiet as Mr Hooper proceeds to conduct his sermon with the veil still hanging over his face.
Such was the effect of this simple piece of crape, that more than one woman of delicate nerves was forced to leave the meeting-house.
The veil, then, begins to actually upset and frighten the people; it becomes the object of fearful awe and speculation. Mr Hooper never removes it for the rest of his life (or, indeed, even in death) which makes many people draw away from him, including his fiancee Elizabeth.
The people imagine that Mr Hooper wears the veil to hide some dreadful personal secret, but he reveals it to be a general symbol of human sin - something which other people always try to hide. Mr Hooper merely chooses to make his sin visible on his own face. Thus the wearing of the veil is revealed to be quite simple in its meaning, although potent in its effect. Symbolism, as so often in Hawthorne's works, plays a significant role in this story.
The congregation is shocked to see Mr. Hooper wearing a black veil over his face because it isso out of character for him to do so. He is a quiet man and not the type of person who would call attention to himself. The townspeople react so negatively to his wearing the veil and they believe Mr. Hooper is either hiding something or feels guilt over something.