Looking into the crowd from her place on the scaffold, Hester recognizes her husband, who she has not seen for some two years now, because "one of [his] shoulders rose higher than the other." He was believed to have been lost at sea, but it appears that he was shipwrecked and then abducted by natives of the continent. Our first impression of him is not a positive one, as the horror that contorts his face is compared to a "snake gliding swiftly over" his features. In western literature, snake (or serpent) metaphors are almost always negative in connotation because of their connection to the devil in the Garden of Eden. Hester's fear of being alone with him seems to confirm the impression that this man is likely capable of evil.
Later in the chapter, Hester vows never to reveal the father of her baby. She says that she would "'endure his agony'" as well as her own, and this may lead us to the conclusion that she does truly love this man. She is willing to suffer alone rather than force him to suffer, publicly, with her: a significant sacrifice on her part.
Hester recognizes her husband, Roger Chillingworth, who has returned from the Indian settlement for ransom. He recognizes her, but she signals him to be quiet.
Through his conversation with a townsman, Chillingworth learns the father of Hester's baby is unknown; despite the pleading of man of the powerful men in town, Hester refuses to talk. Chillingworth then vows that he will find out the identity of the man that is the father of the child and helped cause Hester to be outcast.