In Chapter XIV of "The Scarlet Letter," when Hester speaks with Roger Chillingworth to try to rescue Dimmesdale from his influence, she is "shocked and well  wondersmitten" as she witnesses the changes which have come over her former husband.  For, his appearance has changed from that of a "studious, intellectual, and vigorous" face to a "fierce, searching, yet guarded" aspect. Hester perceives a "blackness" to him, with a "red light in his eyes." She now realizes that Chillingworth has embraced evil into his own heart and gone beyond the original intentions Hester thought them to be when he tells her in Chapter IV:  "He [Dimmesdale] will be mine."

Realizing that Chillingworth has transformed himself into a "devil" who has "undertaken a devil's office,"  Hester's greatest fear is for her beloved Reverend Dimmesdale.  When she declares that she has betrayed her duty to the minister by not warning him about Chillingworth she adds,  "You search his thoughts!  You burrow and rankle in his heart! Your clutch is on his life, and you cause him to die daily a living death...." 

Hester perceives that when a human heart embraces an evil purpose, the owner of this heart becomes "a fiend" who torments the soul of others--a frightening revelation.

http://www.enotes.com/scarlet letter/characters

Hester was married to Roger Chillingworth (his real name being Prynne) in England before moving together to Amsterdam.  Hester was the first to leave for America, and Roger followed.  She believed him to be dead since he took so long getting to her, but we learn later that his ship sank and he, surviving the wreck, lived with the Indians before making his way to Boston.

Hester is afraid of Chillingworth because she knows what he is capable of doing.  He is a learned man and has devoted his life to the healing arts.  Having been with the Indians, he has added herbs and medicinal plants to his sagacious accomplishments. He   His humpback signifies him as a man capable of rage and revenge which he proves with his torture of Dimmesdale. 

When Chillingworth makes his presence known and discovers his wife's predicament, he turns his attentions and knowledge to finding the guilty party upon whom he directs his full knowledge of the arts of medicines.

One gets the feeling that their marriage was not one based solely on love, but more of an arranged union.  At any rate, his jealousy is abundant that she has found someone with whom she is in love enough to risk having a child out of wedlock and be branded an adulterer.  His revenge is swift and exact.  She knew it would be, and so her fears are founded.  This is why she goes back on her agreement not to disclose his identity to Dimmesdale.