Mr. Hooper's sudden adoption of a black veil makes his congregation uneasy because nobody really knew why he did it, hence, the curiosity that the imagination creates leads to all sorts of ideas: Whether he had gone mad or not, whether he was hiding a sin, whether he was showing the congregation the "secret sins" that they call carry around, or whether he was turning into something else.
Having the veil was a sacrifice that came from the minister's own initiative to proof a point: But which point? That is what made everyone uneasy. The fact of not being able to tell nor bring closure to the question is twice as desperate than knowing for sure what he wanted to say. Some villagers thought that the minister had literally "turned into something awful" under the veil. Still, this mystery was what made him so powerful and so enthralling to the congregation. Therefore, as uneasy as it was at first, it was still the fact that the veil made the minister a man above all others, until his death.