In Chapter 6, on their way to investigate the "beast," they approach a cliff which Jack says will make a great fort, most notably because they could topple rocks down onto anyone whom they don't want to climb up the cliff. This entire time, Ralph has been imploring everyone to focus on the shelter and rebuild the fire. He keeps asking if they indeed want to be rescued. They do but Jack and his followers keep getting distracted by things like the beast and the fort, moving further from a civilized commune to a savage mob.
Ralph gets angrier and angrier at their lack of interest in the fire, rescue, and survival, especially because they are being distracted by such juvenile things as making a fort and rolling rocks.
Ralph and Piggy have gone to Jack to negotiate. They are hoping that they will listen to reason and somehow stop focusing only on hunting. They also want to get Piggy's glasses back as Piggy cannot see without them. Piggy brings the conch as a symbol of their authority.
Roger sees his opportunity when Piggy advances and pushes the huge rock from high above. It smashes the conch and knocks piggy off the narrow bridge and he falls to his death forty feet below with his head smashed open.
Ralph's reaction is to be utterly speechless. Golding writes that "Ralph’s lips formed a word but no sound came." Which is notable because this marks the end of Ralph's attempts to try and have any organization.
Jack then leaps up and shouts that there is no longer a place for Ralph's vision. Then Ralph has to flee for his life and the hunters rule the island.