First let's define a motif so we are sure we are on the same page. According to the Free Dictionary Online, a motif is a theme in a piece of literature that recurs throughout the written work. For example, if an author wanted to explore the theme of power corrupting, he would place several incidents in the piece that illustrates that concept.
In chapter 5 of Lord of the Flies, we see the recurring theme of the beast as it relates not only to the boys fear, but also to the nature of evil and how that evil rests in all of us as humans.
In previous chapters there has been mention of a beast on the island by the littleuns. In this chapter, a littleun says he actually saw a beast and the boys toy with the idea that it rises out of the ocean each night. Jack uses the boys fear of the beast to take control...promising that he and his hunters will protect them all from the beast.
The image of the beast is also used to symbolize evil in the form of Beezlebub (satan) and the evil in us all.