My Side of the Mountain is the story of twelve-year-old Sam Gribley who runs away from his home in New York City to live in the Catskill Mountains. However, the narrative begins later on with Sam huddled in a hollowed-out tree with a falcon and a weasel as a winter storm rages outside. Through a series of flashbacks, Sam recalls how he left New York City and his large family to find his great-grandfather's farmhouse in the mountains. In fact, his dad sends him off expecting him to give up and come back the next day. However, Sam continues on and finds that the farmhouse no longer stands. Instead of turning back, he decides to move into a hollowed tree.

Sam still occasionally goes into a nearby town. He learns about caring for falcons at the town's library. He uses this knowledge to catch and raise a falcon he names Frightful. In town, he also meets Tom Sidler, who informs Sam that he is attracting some publicity as a mysterious boy living in the woods, sharing some news clips with him.

Sam becomes companions for a time with a local English professor he calls Bando who lodges with Sam for ten days in the woods. He also hosts an aspiring reporter Matt Spell, who writes a feature on Sam under the agreement that he will keep the location of Sam's forest home a secret.

There are people Sam tries to avoid as well. In one episode he hides for two days from a forest ranger who finds his cookfire. In another, he sneaks away from a hunter to poach his deer.

Over the course of the novel, Sam learns more and more survival techniques and hones his sense of independence. However, his independence is threatened at the end of the story when his family shows up and announces that they are all moving to the old family farm. Sam grudgingly accepts this change, concluding that it is better than living in isolation.