The ministers appear in the community to hold a higher social position than Hester because they are deemed by the community to be in charge of religious and moral development. Remember that this is a Puritan community, so the people believe that they need to live their lives strictly by their communal interpretation of the Bible and its teachings. As a result, the ministers are in charge of delivering the word of God and for interpreting the Bible for the people's daily lives. Further, they are in charge of upholding God's law and seeing that any who break God's law are duly punished. While it is true that Dimmesdale is presented as a weak man, the reader must remember that dramatic irony is at play--the reader, Hester, and Chillingworth know that Dimmesdale is guilty of having an affair, but the other people in the community do not know this fact. Therefore, to the majority of the people, Dimmesdale is still seen as a paramount figure in town, while they see Hester as a condemned sinner.