The residence of Governor Bellingham is clearly identified as being very different from the rather grim and dour houses that surround it. Note how the narrator presents his mansion to the reader when Hester goes there with Pearl to plead for the right to keep her daughter with her and bring her up herself:
It had, indeed, a very cheery aspect; the walls being overspread with a kind of stucco, in which fragments of broken glass were plentifully intermixed; so that, when the sunshine fell aslant-wise over the front of the edifice, it glittered and sparkled as if diamonds had been flung against by the double handful. The brilliancy might have befitted Aladdin's palace rather than the mansion of a grave old Puritan ruler.
What is noteworthy about this description is the contrast the narrator himself draws between the "brilliancy" of the walls of the mansion and the character of the "grave old Puritan ruler" who lives in this mansion. Hawthorne is clearly making a point about the hypocrisy of Governor Bellingham and of Puritanism in general. Puritanism was a strict form of religion that openly and excessively punished those who were publicly identified as having committed what was deemed a "sin," but ignored other aspects of people's behaviour that were dubious, such as Governor Bellingham's excessive wealth. The irony of this situation is increased when the reader remembers it is Governor Bellingham who holds the fate of Hester Prynne and Pearl in his hands.