Pearl effects the Reverend in many ways. She is his child, so he feels affection toward her, but she acts so strangely that he is alienated from him. She is suspicious of him and notices things that a child normally would not. She asks him why he holds his heart in the the same way her mother touchs her scarlet letter. She asks him to stand on the scaffold in the daylight when everyone can see the three of them there instead of in the dead of night when only the three of them (and Roger Chillingworth) know they're there. She refuses to come near them in the woods when her mother's hair is down and the letter is not on her breast. It is almost as if the child is forcing them both to face the grief and guilt they have brought on themselves by committing adultery. She is the mixed blessing...she can be a sweet child, but more often she is like the child of the earth. Grounded in her sinful conception and birth and determined to be the constant reminder of the sin her parents committed.
Arthur is never truly comfortable around Pearl because she seems so much older and wiser than her years. She seems to have knowledge beyond what she should have, and Arthur feels that each time they meet, he is being judged. His guilt gets the best of him aside from the poison that Roger Chillingworth is giving him.