As can be seen in Chapter 7 of the book, there are two major pieces of evidence that Diamond uses to prove this point.
First, Diamond points out that crop development today is a long and difficult process. This is true even though we have advanced scientific techniques and understandings that would not have been available to people who were first domesticating plants. If crop development today takes years of trial and error, ancient plant domestication must have done so as well.
Second, we know that plants have changed a great deal from their earliest forms. We know that wild strawberries are tiny while domesticated strawberries are large. We know that most wild almonds are bitter while domesticated almonds are not.
Because the plants have changed so much, and because it would take a long time to change a plant that much even today, we can infer that plant domestication took years of trial and error in ancient times.