As the descendent of Puritans, especially an uncle who was directly involved in the Salem Witchcraft trials, Nathaniel Hawthorne was most concerned with Puritanism, its hypocrisy, and the resulting treatment of sin. While all three major characters of The Scarlet Letter have sinned, the secret sin of Arthur Dimmesdale is torturous to the soul of its owner, while the sin of violating the heart of a fellow human being is a corrupting sin for Roger Chillingworth, causing him the very loss of his humanity rendering him a fiend. Thus, for Hawthorne, the effects of sin and the treatment of sin by the Puritans in their austerity and hypocrisy and self-deception is the unifying element of Hawthorne's narrative; it is his main theme.
Hawthorne concludes his novel by referring to this theme,
No man, for any considerable period can wear one face to himself, and another to the multitude, without finally getting bewildered as to which may be true.
and by expressing his moral,
"Be true! Be true! Show freely to the world, if not your worst, yet some trait whereby the worst may be inferred!"
Because Hester has "been true," she is forgiven her sin and perceived as an "Angel" and caring member of the community and can live an authentic existence, whereas Chillingworth and Dimmesdale must dissemble whenever they encounter any members of the community, and in this falseness of face, they are ruined.
There could potentially be several different and equally good answers to this question--but I'm going to submit that the main connection between all four main characters (Hester, Dimmesdale, Chillingworth and Pearl) is the sin that earned Hester the letter and the resulting guilt.
For Hester, the sin is public therefore the shame is public. She feels it on a daily basis because she is reminded of it from everyone in town, the letter she must wear and even her daughter.
Dimmesdale on the other hand, suffers as much or more than Hester--in secrecy. His guilt burns much stronger because he does not have any accountability or public repentance for his shame. In the same way Pearl is a daily reminder to Hester of her shame, Chillingworth is a secret daily reminder to Dimmesdale of his.
Pearl and Chillingworth, therefore, are both connected as a result of Hester and Dimmesdale's common sin--but serve as opposite sides of the guilt. Pearl is a symbol of purity and truth. Even in suffering openly, Hester cannot grasp her real redemption--though Pearl earnestly tries to lead her to this realization. Chillingworth is a symbol of evil and punishment. He secretly punishes Dimmesdale throughout the novel, and because of his shame and guilt, Dimmesdale never grasps the evil that lives right there in his very own home and therefore cannot escape it.