It is important to understand that Reverend Hooper is wearing the black veil to make the point that all men sin, and they keep their sins hidden. In his desire to make this point to his congregation, Reverend Hooper is alienated by society. The irony of this situation is that Reverend Hooper was revered by people in the town before he started wearing the veil.
“Mr. Hooper had the reputation of a good preacher, but not an energetic one: he strove to win his people heavenward by mild, persuasive influences, rather than to drive them thither by the thunders of the Word.”
Clearly, Reverend Hooper was not a fire and brimstone sort of preacher; people liked that about him. Before he put the veil on, people would try to be the one to walk by his side after services. They would invite the unmarried reverend to dinner because he had no one to cook for him.
“None, as on former occasions, aspired to the honor of walking by their pastor's side. Old Squire Saunders, doubtless by an accidental lapse of memory, neglected to invite Mr. Hooper to his table, where the good clergyman had been wont to bless the food, almost every Sunday since his settlement.”
The Reverend clearly has a good reputation; he is an agreeable person who does what his parishioners want without a single thought. “ Hitherto, whenever there appeared the slightest call for such interference, he had never lacked advisers, nor shown himself adverse to be guided by their judgment.” In short, Reverend Hooper was well respected, but it certainly did not take the citizens long to start suspecting he had done something wrong when he started wearing the veil.