In her Author's Note to Ethan Frome, Edith Wharton explains that she deliberated on how to tell the story of Frome as she felt that she must not "elaborate and complicate" the feelings of the characters or else risk misrepresenting them because the "granite outcroppings"-- an allusion to the characters--are "half-emerged from the soil, and [are] scarcely more articulate."
Wharton's tone is, of course, rather condescending as she suggests that she must act as an intermediary between the simple characters and the more sophisticated minds to whom she writes. By the same token, Wharton insists upon realism and she wishes to realistically portray a people who have lived many through many a stark winter in a remote area, deprived of opportunities.
Thus, "granite outcroppings" suggests that the characters are somewhat cryptic in their communication of ideas as they have never been taught the subleties of language and feeling. Like the landscape of Ethan Frome, the people in the warm houses are much like this stark, unforgiving environment.