Lack of rules did not contribute to the lack of civility in Lord of the Flies; Ralph and the boys had created a number of smart rules to help the tribe function more successfully. Rules alone were not enough to ensure the boys' continued good behavior and moral actions; what was needed on the island was a more successful form of leadership to enforce the rules. Tried as he did, Ralph was not equipped to rule the boys successfully, because he did not understand how to enforce the rules consistently. He attempts to reign in the boys' behavior in chapter five when he calls the emergency tribal council meeting, but Jack undermines his authority. Ralph never was able to achieve a balance between discipline and limit-setting to encourage the boys to follow the rules that were in place.
The bottom line is that the boys could have had the most elaborate set of rules in the history of civilization, but without the proper leadership and authority to enforce them, these rules were meaningless and ineffective.