In this introductory part of the novel, the surveyor who will be the narrator of Hester's story, finds Hester's scarlet letter eons after the fact. He notices that it is made of:
. . .fine red cloth, much worn and faded. There were traces about it of gold embroidery, which, however, was greatly frayed and defaced; so that none, or very little, of the glitter was left.
A Puritan would have shunned wearing anything that was so decorative, anything that had "gold embroidery". He notes that there was little of the glitter left, symbolizing how Puritan society might have squeezed out any "glitter" (joy) from living, which the reader later finds out to be true in Hester's case.
The cloth is further described:
It had been wrought, as was easy to perceive, with wonderful skill of needlework; and the stitch (as I am assured by ladies conversant with such mysteries) gives evidence of a now forgotten art, not to be recovered even by the process of picking out the threads.
The time required to produce something so fancy would also have been labeled vanity by the Puritans. The reader finds out later in the novel that although the Puritans admired Hester's skill at needlework, they only sought out her needlework for special occasions like baptisms, etc. One thing they did not use her fancy needlework for was weddings, because Hester had committed adultery and therefore her work was not appropriate for wedding attire. Most of the times, Puritans dressed in drab, dull clothing, so again, the scarlet letter is something outside normal Puritan society and illustrates the exclusivity of that society and how it persecuted those that did not conform. In fact, the next quote:
for time, and wear, and a sacrilegious moth, had reduced it to little other than a rag,—
advances this symbolism and indicates that the Puritan society had tried to reduce Hester's life to a symbolic "rag" and for a time, it succeeded. When the narrator puts the letter up to his chest, it burns him. This foreshadows what is to come, as the story of how it burned Hester is about to unfold.