During the era in which The Scarlet Letter is set, adultery was considered a major sin and crime. Hester, as a woman, is more susceptible to punishment than the man involved would have been. In her pride, and her knowledge that the father is a person of high status in the community, she takes all the stigma on herself.
Hester is also compelled to silence by the knowledge that her lover, Dimmesdale, is one of the people persecuting her:
"I will not speak!" answered Hester, turning pale as death, but responding to this voice, which she too surely recognised. "And my child must seek a heavenly Father; she shall never know an earthly one!"
(Hawthorne, The Scarlet Letter, eNotes eText)
Whether from public pressure (as it is his responsibility to see to justice and moral issues) or from his own shame (he has committed a grave sin and now is refusing to incriminate himself), Dimmesdale is now a lesser man in her eyes than when they committed adultery. Rather than be thought of as a liar, slinging accusations to defer her own fate, Hester remains silent, choosing to live her life as best she can. The result is that Dimmesdale suffers great shame and guilt and eventually confesses of his own free will.