Arthur Dimmesdale's letter on his chest is a manifestation of his guilt since for seven years he has kept his secret sin within his heart. As he has told Roger Chillingworth in Chapter X, some people conceal their sin because they yet possess a zeal for God's glory and man's welfare:
...they shrink from displaying themselves black and filthy in the view of men; because, thenceforward, no good can be achieved by them; no evil of the past be redeemed by better service...."
Thus, the hidden mark upon his bosom is this "black and filthy" conscience that he possesses. Even when he goes in the night to stand on the scaffold, Dimmesdale cannot bring himself to confess. And, his watching Hester suffer alone for the sin which he has committed with her tortures him until his flesh manifests this terrible guilt.
In Chapter X when Chillingworth seeks to elicit from Dimmesdale a confession, knowing that the minister's sickness is a manifestation of his spiritual illness, he tells the minister that some black herbs have grown out of a man's heart whose hideous secret he did not reveal. But, Dimmesdale is unable to reveal "the secrets that are buried in the human heart." So, eventually, his guilt rises to the surface of his flesh and manifests itself. This manifestation of his guilt is what gives the evil Chillingworth such delight:
But, with what a wild look of wonder, joy, and horror! With what a ghastly rapture...Had a man seen old Roger Chillingworth, at that moment of his ecstasy, he would have had no need to ask how Satan comports himself, when a precious human soul is lost to heaven, and won into his kingdom.
In Chapter 11, which is significantly entitled "The Interior of a Heart", we as readers discover what Chillingworth has already discovered - that Dimmesdale is scourging his own breast and has scourged out an "A" in his flesh to match the "A" that Hester Prynne so proudly wears on her exterior. It is clear that this self-harming results from the Minister's own sense of guilt and self-loathing. Hester Prynne is bearing the punishment for both of them, while he, who has "sinned" according to the Puritain perspective, is adored and revered as a great clergyman. Thus, because of this conflict, this form of self-punishment allows Dimmesdale to appease his own guilt. Note how the narrator comments on the enacting of this mortification:
His inward trouble drove him to practices more in accordance with the old, corrupted faith of Rome than with the better light of the church in which he had been born and bred. In Mr. Dimmesdale's secret closet, under lock and key, there was a bloody scourge.
It is this "bloody scourge" which is the result of his internal conflict that results in the scarlet letter on his own breast - which of course becomes a symbol of his own guilt and shame. Tellingly, though, whilst Hester Prynne is able to wear hers on the outside of her clothing with pride, Dimmesdale has to bear his secretly in his interior, letting it become a mark of shame.