Cassie's father is trying to make a point about giving up in the Mildred Taylor novel, Roll of Thunder, Hear My Cry. When Mr. Avery announces that most of the black families who have agreed to refuse to patronize the Wallace's store are caving in, Cassie asks her father if "we giving up too?" David draws upon an analogy of the fig tree in their yard, surrounded by the much larger oak and walnut trees. Because the fig tree's
"roots run deep... it belongs in that yard as much as the oak and walnut. It keeps on blooming, bearing good fruit year after year, knowing all the time it'll never get as big as them other trees... It don't give up. It give up, it'll die... we're like it. We keep doing what we gotta, and we don't give up. We can't.
The trees serve a double meaning: The fig represents the Logan family, surrounded by larger trees. Despite its small size, the fig can still produce ample, quality fruit and thrive. Additionally, the larger trees represent the larger, white families--the Grangers, Simms and Wallaces--who would like to starve out the fig tree--the black family of Logans.
I believe he is trying to say that they are all different in their own ways yet they are still the same. (trees)