I would build upon Chillingworth's and Hester's physiognomies. As the novel progresses and Chillingworth becomes more and more bound to his obsession in first seeking out the father of Pearl and then punishing him, we see him follow a physical trajectory as well as a moral one. When Dimmesdale and Chillingworth move in together, there is a blackness associated with Chillingworth that people attribute to his experiments, but that symbolically represents his obsession. Then as Hester seeks him out to tell him she will no longer keep his secret, we see that not only does he have the blackness to his complexion, but that he has become entirely stooped over, and as he continues gathering his herbs, he scurries along the ground, his beard almost touching it, almost like an animal, representing his complete moral descent into obsession.

Hester's appearance is likewise important. When Hester first emerges from the prison door, Hawthorne dwells upon her beauty, which, juxtaposed to that of the goodwives, indicates her sympathetic nature, but it also establishes her as a nonconformist. Throughout the novel, Hawthorne provides us with little questions and insights that show us that Hester is not given to believe the same things as the rest of the Boston community, but that she keeps these feelings hidden, and won't even confess them to herself. When Hester is not true to herself and these beliefs, and allows the scarlet letter to be her identity, she fades into the background and loses an element of her beauty. But when she seeks out Dimmesdale, sheds the letter, and embraces the passionate woman she really is, we see her beauty return immediately, only to be hidden again once Pearl demands she put back on the oppressive symbol.

The two most notable characters that can be explained in such a way are Roger Chillingworth and Arthur Dimmesdale.
Chillingworth was born with a physical deformity that does, in fact, represent a part of his personality. His slumped shoulders (resembling a hunchback) represents his inability to show outward affection for others. His inability to form and maintain deep relationships (which is also represented by the nature of his name--cold, without warmth, etc.) is juxtaposed with his drive for academic sucess. Chillingworth takes great pride in obtaining deep knowledge of medicine and pharmacuticals.
Arthur Dimmesdale's personality is described as weak and frail, which does mimic some of the choices he makes throughout the book. No one knows he is the man with whom Hester had the affair; she did not turn him in, and he did not admit to it. Because his sin has been kept from the community, Dimmesdale is forced to live with the guilt, which eventually begins to take a toll on him physically. His guilt manifests, becoming the illness that tragically ends up causing his death.
Hester Prynne's physical characteristics do not necessarily reflect her moral and psychological traits, other than the "A" that she is forced to wear by her community. She is also forced to wear drab clothing, which is completely opposite from her personality. She is very maternal, loving, and has tremendous strength to stay within a community that looks down upon her for her actions.