The answer to this question can be found in Chapter 6. There, Diamond talks about why the move to farming was relatively slow and "piecemeal."
One reason for this is that it is not like the early hunter-gatherers had examples of full-fledged farming communities to learn from. They couldn't just look at their neighbors and copy them. They really would not even have known that a farming society was possible because no one had ever done it before.
The other major reason for this is that the advantages of farming are not obvious until farming is firmly entrenched. A person would have to decide between doing a lot of work right now (planting, weeding, etc) in hopes of getting a return many weeks later and doing some amount of work right now that would have a more likely return (going and collecting shellfiish).
Today, we know that farming is more efficient than hunting-gathering. But this would not have been obvious to people at the time.