When the story begins, it is November of 1843.  Lyddie is thirteen years old, and Charles is ten.  Rachel is six, and the baby, Agnes, is only four (Chapter 1).  The children are living with their Mama in a small cabin in rural Vermont.  Their Papa, a landowner who had "loved them and worked hard for them", was nonetheless "an unlucky man" when it came to making a living off his small farm, and had gone West to try and make his fortune.  Papa had gone away three years ago, promising to return soon, but he never had.  In his absence, Mama has grown increasingly nervous and unstable.  The children, especially Lyddie and Charles, have had to shoulder most of the responsibility for keeping the farm running and the family fed.  They have been forced to grow up quickly, and are capable far beyond their years.

The story ends exactly three years after it began, in November of 1846.  Lyddie at this time is sixteen, determined to get an education and make it on her own, before possibly returning to the farm to make a life with Luke Stevens, who loves her.  Charles is thirteen and has found a home with with the Phinneys, a kindly family who welcome him as an apprentice and a son; the Phinneys, having always wanted a daughter, have taken in nine-year-old Rachel as well.  Agnes, who had been taken by Mama when she went to live with her sister Clarissa back in 1843, has died.