When Hester is on the scaffold, she thinks back several years and sees the face of an old man who looks like a scholar. What she is thinking about is Roger Chillingworth and the conditions and events that lead her to the scaffold. I'm not sure if this memory helps her endure her punishment because the memory is shortly followed by the recognition that the person she has remembered, her husband, is standing in the back of the crowd watching her punishment. From a conversation between Chillingworth and another bystander, we learn that Hester had been set to the New World alone in order to set up a house for herself and her husband. But, years past and her husband never appeared, leading her and some of the townspeople to think her husband probably drowned at sea. Now that he has reappeared only to find she has a baby in her arms that can't possibly be his, Hester must dread seeing him.