The setting of "Desiree's Baby" contributes to the action of the story by showing the home of Armand and Desiree as falling into despair.
Madame Valmonde had not seen Desiree and the baby for four weeks. When she reached L'Abri she shuddered at the first sight of it, as she always did. It was a sad looking place, which for many years had not known the gentle presence of a mistress, old Monsieur Aubigny having married and buried his wife in France, and she having loved her own land too well ever to leave it. The roof came down steep and black like a cowl, reaching out beyond the wide galleries that encircled the yellow stuccoed house. Big, solemn oaks grew close to it, and their thick-leaved, far-reaching branches shadowed it like a pall. Young Aubigny's rule was a strict one, too, and under it his negroes had forgotten how to be gay, as they had been during the old master's easy-going and indulgent lifetime.
This allows an active reader to pick up on the foreshadowing of what is to come. The home, "was a sad looking place" with "big solemn oaks". One could come to understand that the setting echos the action of the story by enhancing the themes of sadness and solitude.
Therefore, knowing that each time Desiree's mother comes upon L'Abri she shudders, one can assume that the home and movement of the story will not be one filled with happiness and companionship.