The description of the congregation as being "pale-faced" refers to the way in which the sudden donning of the black veil by Mr. Hooper impacts his flock and makes them scared and worried. An interesting aspect of this story is the way that the congregation react to Mr. Hooper's sudden decision to wear a black veil to hide his face from the world. What is fascinating is how the reactions of the congregation seem to reveal more about them themselves than they do about Mr. Hooper. Consider, for example, how the congregation reacts to Mr. Hooper during the funeral that he takes:
The people trembled, though they but darkly understood him, when he prayed that they, and himself, and all of mortal race, might be ready, as he trusted this young maiden had been, for the dreadful hour that should snatch the veil from their faces.
We can see in this quote that the fear that the veil inspires in the congregation makes them think about their own death and the way that they all have "veils" that they are wearing metaphorically, which will only be removed fully when they die. The meaning of the "pale-faced" congregation is therefore based on the way that the appearance of the black veil makes them think of the own veil that they themselves wear.