This occurs in chapter 19. Literally, Pearl requests that the letter be returned to Hester's chest because it is a familiar piece of her mother that makes her mother identifiable to Pearl. It is a part of Hester's identity.
This incident is so strange because Pearl gives Hester great affection when the letter is replaced and she even kisses the letter on Hester's chest.
Figuratively, Pearl is representative of evil. She has been referred to as a devil-child throughout the text. The evil in her does not want the shame and guilt of what occurred to Hester to be removed. She wants Hester to be forced to keep living with it. This greatly pains Hester.