Both stories can be read as showing the destructive outcome of desiring perfection in this world.
When he is first hired by the lawyer narrator who tells the story in "Bartleby the Scrivener," Bartleby works diligently, but then he stops entirely. Every time he is faced with a project, he says he "prefers not to" do it. At the end of the story, the lawyer hears that Bartleby once worked at the Dead Letter office in Washington, D.C., where letters that never arrived to their destinations were sent to be destroyed. His lawyer employer believes that, when constantly faced with poignant and sometimes tragic communication misfires, Bartleby became depressed. Bartleby eventually passively accepted institutionalization and death rather than continuing on in an imperfect world of communication failures.
Likewise, in "The Birthmark," Alymer longs for a more perfect world, and in his case a perfect wife, which he believes can only be achieved if he removes a birthmark on her face which runs deep into her body. He—and eventually Georgiana herself—come to desire the perfection they feel will be attained from removing the birthmark. Georgiana passively puts herself into her husband's hands, risking death, and eventually dying, rather than remaining imperfect in his eyes.
In both cases, a common theme emerges: the world cannot be made perfect. The desire for perfection leads to death. Becoming obsessed at the world's imperfection, be it through failed communication or a birthmark, is a road that leads to destruction.