Why is Ralph considering giving up leadership of the group?
In Chapter 5, the fragile order that the boys have established on the island begins to erode. Ralph is furious that the hunters have let the fire on top of the mountain go out, causing a passing ship to have missed seeing them. He has called a meeting, even though it is getting late in the day. On the way to the meeting, he realizes the "wearisomeness of this life," that one has to be constantly vigilant to stay on the right course. As leader, he is the only one who seems to feel the burden of keeping the boys on the course of civilization. He realizes that his thinking is not always clear or focused. Comparing himself to Piggy, he realizes that Piggy has a stronger intellect, but that "Piggy was no chief." So Ralph understands his inadequacies and the burden of leadership.
When the meeting deteriorates to the point where the boys are insulting each other, dancing in the darkness, and scaring the littluns, Ralph realizes his ability to hold the group together is tenuous at best. He fears to blow the conch because, if the boys don't respond, he says, "We'll be like animals. We'll never be rescued." He postulates to Piggy that the rules of society may not function "here on this island." With all this weighing on him, he says to Piggy, "I ought to give up being chief." Piggy and Simon persuade him that he is the only thing that stands between them and Jack's hatred taking a violent turn, which, of course, foreshadows later events.
Ralph considers giving up leadership of the group because it seems like everything is falling apart. Remember the assembly in Chapter Five? He speaks, and the kids are rude to him. He insists on the rules, and nobody follows them. He insists on keeping a fire lit and tries to get people to follow rules for good hygiene (relieving themselves away from the food or where they live), and the kids laugh at him. Jack's challenge is part of the reason, of course, but it is more how the kids respond to Jack's push for hunting than just Jack. The way the kids focus on what's fun and what feels good in the moment rather than what's right upsets Ralph.