Most of the moving and departure scenes occur when Richard is a young boy. The first move Richard experiences is the move to Memphis. The small family consisting of Richard, his brother (Leon), his father (Nathan), and his mother (Ella), move into a one story brick tenement. It is this move to Memphis that first highlights Richard's awareness of his father as an abusive, mercurial, emotionally inaccessible, and autocratic parent. He also discovers that his mother has a strange propensity to use emotionally oppressive language fraught with religious condemnation to discipline him. These early episodes of violent discipline from both parents are to lay the groundwork for Richard's later antipathy and rejection of religion.
The departure of Richard's father to live with his mistress is the catalyst for Richard's initiation into the sordid world of degraded sensual pleasures. With his mother working to try to make ends meet, Richard becomes the victim of saloon patrons who sport with him, ply him with alcohol, and teach him obscenities. Richard learns that his father's departure means being consistently tormented with an abiding, gnawing hunger; he tells us that he had never equated his father's presence with food in the house until he left.
When Richard's mother cannot make the rent payments, both Richard and his brother are sent to live in an orphan home run by the overbearing Miss Simon. This move reiterates in Richard's mind that adults are never to be trusted; instead of being human shelters of safety and stability, the adults in Richard's life are violent narcissists who cruelly seize upon his youthful inexperience as evidence of depravity. When Ella realizes that she could never depend upon her husband's pity to support his sons, she decides to take the boys to her sister Maggie's place in Elaine, Arkansas. On the way, they stop by Granny's house in Jackson, Mississippi. Granny is a strict Seventh Day Adventist, and Richard learns that she views the act of reading fiction as devil-inspired. With his propensity to blurt out things that he has heard adults say (but does not understand the meaning of), Richard soon learns that words can make some people so angry that they will resort to physical violence. His words earn him beatings from Granny and his mother.
When Uncle Hoskins, Aunt Maggie's husband, is shot by several white people, Richard finds himself moving again, this time back to Granny's house. He starts to understand that there are no safe places on earth and no stability for children like him. He is not surprised when both his mother and Aunt Maggie soon tire of Granny's self-righteous, religious routines and move the children to West Helena, Arkansas. Here, he finds that his mother and Aunt Maggie work long, tiring hours. The cycle of abandonment is again perpetuated when Richard and his brother are left to their own devices most of the time.
You can see that most of the moving and departures in Richard's young life become the catalyst for Richard's later struggles with trust and religious faith.
Thanks for the question! I hope that what I have written has proven useful.