Roger Chillingworth is the antagonist in that he's the main impediment to Hester and Dimmesdale's happiness. This is because he represents the stern moral values of Puritanism, with its strict moral code and firm, unwavering belief in the fundamental depravity of each human being.

Instead of using his medical training—not to mention his religious values—to help a clearly troubled soul, Chillingworth abuses his authority to torture Dimmesdale psychologically, actually becoming more physically repulsive the more he torments his patient. It's no exaggeration to say that Chillingworth lives for destroying Dimmesdale and Dimmesdale's plans for happiness. So determined is he to carry out this diabolical scheme that he buys a ticket for the same ship as Dimmesdale and Hester, hoping to follow them to Europe.

It's instructive that once Dimmesdale has publicly confessed of his sins, Chillingworth effectively gives up the ghost. Not long after Dimmesdale's death, Chillingworth follows him to the grave, eaten up by a hatred that now has no object.

Chillingworth is the antagonist in The Scarlet Letter because he not only tortures Hester Prynne and Dimmesdale, but he also manages to keep them apart and eventually helps along Dimmesdale's demise. When Chillingworth first shows up at Hester's jail cell, after having tarried coming to meet her in Boston, he taunts her. Instead of comforting her, he asks her, "Art thou not afraid of nightmares and hideous dreams?" When he learns of Hester's sin of adultery (which affects him, as he is Hester's husband), he mysteriously changes his name to Chillingworth and keeps the reason secret.

The reason for his secrecy is that, under the guise of being Dimmesdale's personal physician and roommate, Chillingworth taunts Dimmesdale. By maintaining his constant companionship of Dimmesdale, Chillingworth worsens Dimmesdale's health and exacerbates the reverend's suffering. The pain that Chillingworth causes Dimmesdale is lengthy and continual. Hester asks Dimmesdale, "is there not shade enough in this boundless forest to hide thy heart from the gaze of Roger Chillingworth?" The reality is that even the forest can not hide Dimmesdale from Chillingworth's all-knowing gaze, so Hester proposes that she and Dimmesdale escape to the sea. However, Chillingworth foils their plan by booking passage on the same ship, and Dimmesdale realizes that there is no escape from his guilt and dies soon after. 

Not only does Roger Chillingworth act against main character Hester Prynne, but he focuses his evil particularly against Arthur Dimmesdale.

There are two particular events that are salient in Chillingworth's quest to seek revenge. The first is when he makes Hester keep the secret of his true identity.

Recognise me not, by word, by sign, by look! Breathe not the secret, above all, to the man thou wottest of. Shouldst thou fail me in this, beware! His fame, his position, his life, will be in my hands. Beware!

This is a clear threat to both Hester and, unbeknownst to him, to Dimmesdale. Chillingworth has tied Hester to a secret that she knows will harm someone that she loves. Additionally, she fears that the identity of Dimmesdale is also uncovered by the anger of Chillingworth.

The second even that marks Chillingworth as an antagonist is his joy when he discovers the carved letter "A" on Dimmesdale's chest.

Had a man seen old Roger Chillingworth, at that moment of his ecstasy, he would have had no need to ask how Satan comports himself, when a precious human soul is lost to heaven, and won into his kingdom. But what distinguished the physician's ecstasy from Satan's was the trait of wonder in it!

The manner in which his discovery is described involves the mention of the devil himself; this is clearly an indication that Chillingworth is undoubtedly the antagonist of the novel.