The reason why the people from the settlement choose to not understand Dimmesdale's confession is simply because, in their Puritanical world, the pastor can simply do no wrong. In the specific case of Arthur Dimmesdale, Hawthorne ensures to remark the way in which Dimmesdale's charm and charisma seems to give him an otherworldly aura that seems to attract people instantly; this makes the reader wonder if this same aura and charm is what made Hester so quick to fall under his "spell". The way in which he is described almost resonates in the reader's mind as if he were being characterized as a sort of supernatural being.
[he] kept himself simple and childlike; coming forth, when occasion was, with a freshness, and fragrance, and dewy purity of thought, which, as many people said, affected them like the speech of an angel.
This being said, it is his persona, the position that he holds, and the overall approval of the settlement's aldermen that has seared within the psyche of the community the idea that Arthur Dimmesdale is simply flawless.
Neither...had his dying words acknowledged, nor even remotely implied, any--the slightest--connexion on his part, with the guilt for which Hester Prynne had so long worn the scarlet letter. ...[T]he minister, conscious that ...the reverence of the multitude placed him already among saints and angels--had desired, by yielding up his breath in the arms of that fallen woman, to express to the world how utterly nugatory is the choicest of man's own righteousness.
The fascination that Hawthorne bestows upon Dimmesdale's followers is mainly a way to demonstrate the extents to which religious fanaticism can make the most obvious things still become twisted and turned around. Dimmesdale's confession made their fixation on him even stronger. It goes to show also the ignorant and limited nature of the people who so harshly judged Hester Prynne.