Frederick Douglass's first master was Captain Anthony, who is an extremely violent, cruel man. As a child, Douglass recalls witnessing Captain Anthony brutally whip his Aunt Hester, and it is implied that the Captain was also sexually interested in her. Douglass's first master, Captain Anthony, has three children: Andrew, Richard, and Lucretia, who is married to Captain Thomas Auld. All three of Captain Anthony's children live on Colonel Lloyd’s estate, where Douglass grows up as a slave. In chapter five, Douglass is around seven or eight years old when he is selected to go to Baltimore to live with Captain Anthony’s son‑in‑law’s brother, Hugh Auld.

Upon arriving in Baltimore, Douglass is greeted by his new master, Thomas Auld, and his lovely wife, Mrs. Sophia Auld. Initially, Mrs. Sophia Auld is kind and gracious to Frederick Douglass. Tragically, Thomas forbids her from teaching Douglass to read and write, which motivates him to learn on his own. As time passes, Douglass notices Mrs. Sophia Auld develop into a callous, severe woman. Douglass uses her dramatic transformation as an example of how institutionalized slavery negatively affects the masters as well as the slaves.