In Chapter IV when Roger Chillingworth visits Hester at the prison, he asserts that he will learn who the man is that has committed sin with Hester because
"[T]here is a sympathy that will make me conscious of him. I shall see him tremble. I shall feel myself shudder, suddenly and awares. Sooner or later, he must needs be mine."
This "sympathy" between Chillingworth and Dimmesdale is in their pride. When he identifies this pride in Dimmesdale, Chillingworth questions the minister and learns that Dimmesdale feels that even a sinner can help his congregation. Thus, the old physician surmises that Dimmesdale hides his sin of adultery in order to continue to minister to the members of his church.
It seems that Roger Chillingworth suspects Reverend Dimmesdale from the first when he stands in the crowd and watches Hester Prynne in her public humiliation. At the least, he witnesses the minister encourage her to respond to the questions of the Reverend Wilson and speak the name of the "fellow-sinner" who "hath not the courage to grasp it for himself--the bitter, but wholesome cup that is now presented to thy lips!" As he says these words, Dimmesdale's voice is broken and "tremulously sweet"; when Hester refuses, the minister murmurs, "She will not speak!" and he has placed his hand upon his heart when awaiting Hester's response.
The sympathy which Roger Chillingworth recognizes in the Reverend Dimmesdale is that both of their good natures have been wasted. The physician Chillingworth suffers as he has overstepped his human role as a man, and takes upon himself the quest of a man's soul. Dimmesdale has also wasted his life in pretense; as a man of the cloth, he has been hypocritical in preaching sermons against sin when he himself is guilty of the serious sin of adultery. Thus, in his sin of pride, the physician for Dimmesdale feels the sympathy of this same cardinal sin. For, in his pride, the minister has rationalized that although a man be guilty of sin, he may yet retain "a zeal for God's glory and man's welfare," and aware that if his sin were known he could achieve no good, he hides this sin in order to help his fellow man and serve God. It is at this point that Roger Chillingworth discovers the minister's chest holds the answer for him that he has suspected, "[W]ith what a wild look of wonder, joy, and honor!" he gazes at the mark upon the minister's chest, the reason for his guilt. Thus, Reverend Dimmesdale's secret sin of adultery is revealed with the mark that the fiend who seeks revenge witnesses.