The entire novel is presented through a series of Charles's recollections; the subtitle, The Sacred and Profane Memories of Captain Charles Ryder, highlights this theme. The narrative opens in the present day with Charles unexpectedly finding himself near the Brideshead estate, which stirs the memories that form the majority of the story. The novel concludes with Charles again in the present, taking a brief walk through the mansion, encountering Nanny Hawkins, and savoring additional memories of his friendship with the Flyte family.

Charles is a man who holds the past in high regard, whether it is factual or imagined. The first part of the book is titled "Et In Arcadia Ego," a Latin phrase translating to "I, too, lived in Arcadia," alluding to a pastoral and mountainous region of ancient Greece often depicted in art and literature as a kind of utopian paradise where life is idyllic and satisfying. This initial section recounts Charles's introduction to Sebastian and the joyous times they shared.

Throughout the novel, Charles maintains the belief that the past is superior to the present. In the Prologue, he critiques the contemporary behavior of "Young England," lamenting their lack of education, their fashion choices, and their speech. Even during his time as a student at Oxford, he grumbles at the beginning of book one when women arrive for a week of dances and parties, feeling unsettled by the alteration in the college's atmosphere.

All of Charles's recollections of his first year of friendship with Sebastian are idealized. For example, in book one, chapter four, Charles fondly remembers a summer spent almost exclusively with Sebastian, saying, "I, at any rate, believed myself to be very near heaven, during those languid days at Brideshead." Sebastian also acknowledges that this summer of their youth will be a treasured memory, expressing, "If it could only be like this always—always summer, always alone, the fruit always ripe."