In chapter 10 of The Scarlet Letter Chillingworth takes advantage of a question asked by Dimmesdale in order to, once again, take a jab at the latter's hidden secret which, as we know, Chillingworth is dying to have him confess. As the reverend asks the nature of some odd, flabby plants found by Chillingworth, the latter replies
I found them growing on a grave, which bore no tombstone, no other memorial of the dead man...They grew out of his heart, and typify, it may be, some hideous secret that was buried with him, and which he had done better to confess during his lifetime.
Clearly, this reflects the deep interest that Chillingworth has in Dimmesdale's confession. He is using just about any object, moment, and resource to make Dimmesdale feel guilty and to have him admit that it is he who fathered Pearl with Hester Prynne; that it is he, the well-respected, highly admired, nay-doer Reverend Dimmesdale who committed the highest offense ever seen publicly in the settlement.
Hence, just like the weeds were meant to have grown out of the guilt of some nameless dead man's heart, Chillingworth feels that Dimmesdale's heart also has something growing badly within and that will manifest at some point quite horridly, like those horrid weeds.