When he says, "Maybe it's only us," Simon understands the concept of psychological projection, or "blame shifting," as it is commonly called.
This observation of Simon's goes to the heart of William Golding's allegory of human nature. Simon, who is intuitive and sensitive to the spirit of others, eventually recognizes the innate bestial (i.e. brutal or savage) qualities that lie within the human heart. For instance, in Chapter Four, he has observed Jack's beastly cruelty to Piggy as Jack punches Piggy in the stomach and breaks his glasses as Piggy is hit in the head.
In Chapter Five Ralph calls a meeting and outlines the important things that must be done. But, his attempts to restore order are disrupted by talk of the beast. One little boy named Percival suggests that the beast might come out of the sea because his father has told him that there could be creatures in the ocean. As the boys argue among themselves about the beast, Simon makes an effort to explain that they are trying to objectify what is actually something in themselves; they are "blame shifting." However, Simon stumbles,
"What I mean is...maybe it's only us....We could be sort of..." [Penguin edition does not contain "that we're afraid of"]
Simon tries to suggest that the beast may be something within themselves, but he
...became inarticulate in his effort to express mankind's essential illness.
When he, then, tries to give a comparative example, this, too, fails. It is not until later in the novel when Simon confronts the Lord of the Flies that he finally acquires the capacity to articulate the evil inherent in humans.