Chillingworth tells Dimmesdale that they grew from a man's unmarked grave, "out of his heart, and typify...some hideous secret that was buried with him, and which he had done better to confess during his lifetime." In other words, hint, hint, Dimmesdale! 'Fess up! I suspect you committed the crime and are harboring it, and if you don't confess, your heart will end up as dried, withered weeds.
A long, lengthy discussion between the two ensues, where Chillingworth attempts to nudge Dimmesdale towards confession: "lay open to [me] the wound or trouble in your soul!" Dimmesdale refuses, saying, "I commit myself to the one Physician of the soul!" So, he's going to wrestle alone with himself and God, and leave Chillingworth out of it. The lesser of two evils perhaps? Confess to God, or confess to the man who Hawthorne describes in the previoius chapter as "Satan himself, or Satan's emissary."