As was sadly typical among slaves, Douglass's family was torn apart by the realities of slavery. He was the child of a slave woman named Harriet Bailey and a white man. Douglass assumes that the man in question was his mother's master. In any case, he was permanently separated from his mother as an infant, as she was hired out to a different plantation. In one of many poignant vignettes in his Narrative, Douglass describes his mother coming to visit him at night, after her work was done on her plantation:
She made her journeys to see me in the night, travelling the whole distance on foot, after the performance of her day’s work. She was a field hand, and a whipping is the penalty of not being in the field at sunrise...I do not recollect of ever seeing my mother by the light of day. She was with me in the night. She would lie down with me, and get me to sleep, but long before I waked she was gone.
She died when he was seven, and he lived with his Aunt Hester, who he described as being brutally beaten by his master, Colonel Lloyd, for consorting with one of Lloyd's male slaves. His grandmother is also cruelly treated, left to fend more or less for herself when she became too old to work. Douglass soon moved to Baltimore, and lost connections with whatever kin he had left. The corrosive effects of slavery on the family are an important theme in Douglass's work. His family was almost nonexistent, destroyed by slavery.