While we often associate the color red with love, that isn't how it is used in "The Scarlet Ibis." Instead, Doodle's poor health is associated with the color red:
He seemed all head, with a tiny body that was red and shriveled like an old man's.
Later, the bird that comes to rest in a tree outside their home is also red:
He slipped out into the yard, and looked up into the bleeding tree. "It's a big red bird!"
Neither Doodle nor the bird is well-suited to the environment, and each dies as a result. Thus, the color red in this story is symbolic of death.
While blue is often used to represent calmness, the author again crafts an unexpected symbolism by intentionally using this color to depict struggle. Consider how the narrator pushes his brother to his physical limits so that he won't be "different from everybody else" once he begins school:
I made him swim until he turned blue and row until he couldn't lift an oar.
Thus, the color blue is associated here with a lack of oxygen. It symbolizes weakness and physical toil.
Green symbolizes life in this story, which is a common use of the color in literature. This is significant because it creates a contrast between life and death, which is a central theme in "The Scarlet Ibis." When the dying scarlet ibis rests in the tree, it is surrounded by life:
Its wings hung down loosely, and as we watched, a feather dropped away and floated slowly down through the green leaves.
Also notice that the poison used to kill the rats in the story is green:
A screech owl flapped out of the box into our faces, scaring us and covering us with Paris green. Doodle was paralyzed.
It's interesting that the author chose to use a color typically associated with life as the color of a poison in this story; this same poison covers Doodle, whose health is precarious, which foreshadows that his life will be cut short. He seems to recognize this omen and begs for his brother not to leave him.
Gold is often the color of success and achievement, and after the narrator teaches Doodle to walk, he believes that anything is possible:
Success lay at the end of summer like a pot of gold, and our campaign got off to a good start.
Yet after working all summer, the success that the narrator has envisioned seems like an almost impossible feat:
School was only a few weeks away, and Doodle was far behind schedule. He could barely clear the ground when climbing up the rope vines, and his swimming was certainly not passable. We decided to double our efforts, to make that list drive and reach our pot of gold.
The narrator's ultimate success would be to help his brother achieve a sense of normalcy compared to his peers, yet as summer winds down, this seems more like a dream than a true possibility.
The color white is often symbolic of youth and innocence. Consider the description of young Doodle's surroundings:
He might, as long as he lived, lie on the rubber sheet in the center of the bed in the front bedroom where the white marquisette curtains billowed out in the afternoon sea breeze, rustling like palmetto fronds.
While the narrator's heart is bent toward selfishness in his interactions with Doodle, Doodle is often described as almost angelic. His heart is pure, and the love he has for his older brother is genuine. This contrast between the narrator's need to have a "successful" brother and Doodle's pure heart is captured in the use of his white surroundings.