Nathaniel Hawthorne was a descendant of John Hathorne, a judge in the Salem witch trials. That was always a source of personal shame for him--so much so that he even changed the spelling of his name in order to distance himself from his guilt.
In these chapters of The Scarlet Letter, Hawthorne is clear in his belief that what's happening on the inside is directly related to what's happening on the outside. He shows a man, Arthur Dimmesdale, who is literally deteriorating physically as he carries the weight and burden of his unforgiven sin and guilt. He walks like an old man, he constantly places his hand over his heart, his voice is weak, and he startles easily. He is a picture of a man sick with his own sin.