What Diamond is saying in this book is that civilization is sort of an accident -- places that have good climates and good luck can have lots of people and civilizations. In Chapter 2, he is saying that high population densities come about by luck in these islands too -- places where farming is possible can have large populations, places where people have to be hunter-gatherers cannot have dense populations.
The only real "construction" that allows for high population densities is agriculture. Agriculture allowed people to grow excess amounts of food and, thereby, to feed more people.
Agriculture was not as possible on many Polynesian islands as it was in other areas of the world. Because of this, the people were hunter-gatherers, which does not allow a big enough food supply to provide for a dense population.