Diamond does not and cannot tell us why the phases happened exactly when they did. He cannot say why the first phase happened around 10,000 years ago rather than around 15,000 years ago. All he can really say is why the phases happened in the order that they did.
Diamond's phases are arranged in order of difficulty. The easist crops to cultivate were, not surprisingly, domesticated first. These were crops that were already edible and high-yield in their wild forms. After that came crops like fruit and nut trees that were relatively easy to cultivate, but which did not bear for years. This would only be worth doing when your society has gone away from hunting and gathering and come to stay in one place long enough to benefit from something that takes years to grow. The last stage was one that included trees that have to be grafted and treated in other complicated ways. This could not be done until the agricultural practices of a society had become very advanced.
So, the phases went according to the ease of cultivation. As societies moved farther away from hunting and gathering, their agricultural practices became more complex.