The overall setting of this short story is North Carolina in the early 1900s. We get a sense of this geography through the mention of places like the cotton field and the swamp. Within the backdrop of the American South, there are several more specific locations where the action of the story takes place—in and around the narrator's childhood home.
To begin with, the narrator first mentions that in his present-day, he sits remembering the story's events from his "cool green parlor" (1). None of the scenes of the plot happen here, but it is the frame within which the memories of the events emerge, a space of calm distanced from the turmoil of the past.
The childhood home is where the real events of the story are centered; the interior where they grow up, the breakfast table where Doodle walks in front of his parents for the first time, and the yard where the scarlet ibis falls from the bleeding tree. This area is the characters' home base: where they begin and where they presumably, eventually, return.
Across the cotton field from the house is Old Woman Swamp, the secluded pocket of Southern wild in which the narrator teaches Doodle to stand and to walk. The solitude and isolation of the swamp are what allow it to be a place of learning and teaching; the narrator benefits because his shame in his brother can be solved in private, and Doodle benefits because he has a hidden place in which to slowly make progress before showing the rest of the world what he can do.
The story climaxes and concludes while journeying to and from Horsehead Landing. This is the place where the narrator wants to teach Doodle to swim, and it is a place we only hear about briefly, as the relevant action happens while on the way there and back. Most importantly, it is the place from which Doodle will never return.
Between the Landing and the house is a sort of nature no-man's-land, a space between spaces that is open to the weather and must be braved in order to reach the destinations on either side. The narrator makes the decision to walk there with Doodle although harsh weather is on the way and return by running through the rain. It is here, stranded out in a storm, that the narrator runs home without waiting for Doodle, returning later to retrieve him and finding that his brother has become yet another fallen scarlet ibis.