The idea of memory in the story is directly related to the themes of isolation and destruction. As a result of the third-person and mostly omniscient point-of-view, the reader is invited into the much more complex inner workings and personality of Edgar than his actions and limited communication reveal of him to others. Memory plays a key role in this complexity.
At the very beginning of the story, the reader is introduced to images of Almondine in what would be Edgar's "earliest memory," suggesting that even as a baby, Edgar has an accute perception of his dog, one which sticks with him for life. As the mute only child of two arguably introverted parents, Edgar's isolation from the world outside of the dog kennel seems to be almost created and fostered by his circumstances. As a result, Edgar often lives very much inside his own head. He is frequently reflective on his own past, and very much interested in the past of his parents (one which they never share the complete truth of, but the stories of which almost become as dear to him as his own memories). He is both happy and seemingly healthy.
When his father mysteriously dies and Edgar is the only witness, he cannot remember the details of his father's final moments clearly enough to recall them to Glen (the town policeman). It is as if his mind is blocking the memory to protect him. Soon after, when his father's ghost visits Edgar in the kennel, Edgar experiences a physical sensation that the ghost has touched his heart and given him all of his father's memories. The feeling is so powerful Edgar thinks it might kill him. For one instant he holds complete understanding of who his father is and what his intentions always were, but in the next instant, the feeling, and the memories, are gone. He wakes the next morning wondering if it was all a dream. He now harbors a secret he cannot explain to anyone, because he's not even completely sure of the reality of it himself. His isolation increases and so begins his destruction.
When he runs away from home, his memories haunt him. Because he is isolated from all other humans, all he has is time to think of the past. Memories of Almondine are the worst because they are the most tangible and realistic. If he dwells on them, dread and despair threaten to crush him. At night, he dreams of his father and thrashes in his sleep, but wakes up unable to remember any details of the dream. He is constantly thinking of memories of his mother and Claude, and looking for clues in the memories to further understanding his father's death (and how he can explain it). He constantly fears forgetting more and hates himself for the things he cannot recall.
In many ways, the author uses the memory motif as a means of setting up characters who, though far from ordinary, are certainly happy and healthy. Even in his isolation, when his father is alive, Edgar is both. Then, the same motif is used to show how a death in the family can immediately plague the happiness and health of the survivors. Edgar's isolation evolves from something that was once something he lived with, to something he fears will kill him.