In literature, seeds have traditionally been regarded as a metaphor for life's potential. Just as the seed will grow if given the right conditions, such as good soil, plenty of water, and so on, so too will life develop to its fullest potential if the circumstances are propitious.
In the case of Lee's father, Kuo Yan, however, the circumstances were very far from propitious. He led an unstable, peripatetic existence, due largely to the racist persecutions of ethnic Chinese by the Sukarno regime in Indonesia. In metaphorical terms, just when it seemed that his "seed" had found good soil, it was unceremoniously uprooted and thrown to the wind.
This brings us to the other side of the metaphorical coin, so to speak. As well as being a metaphor for life, the seed can also represent death. Once again, it largely depends on the environment in which the seed of life must grow. But even if such an environment is propitious to a seed's growth, the seed may still remain poisoned and withered. That's certainly what seems to have happened in the case of Kuo Yan. Despite managing to escape the hellhole of an Indonesian prison and settling in the United States, he has been so traumatized by his experiences that he is unable to thrive in his new environment.
In his dreams, Lee sees his father with pockets full of seeds. He wonders why he still keeps them. One could argue that it is because the seeds represent what might have been—the full, happy existence that Kuo Yang could've enjoyed had it not been for the sudden uprooting of his "seed" and its being deprived of the conditions necessary to make it grow and thrive.