They believe they are better than the animals and with fire-power they should be able to take the farm. Jones was embarrassed. Even though the other farmers didn't get along they did agree that no more animals could get infected withthe poisonous ideas of Animal Farm. They allowed several of their own workers to help Jones try to get it back.
There were rumors of cannibalism, animals torturing other animals, and having 'females in common'. None of the farmers wanted to find their resources to end up this way.
I assume that you are talking about the attack that happens in Chapter IV. This is the attack that leads to what is called the Battle of the Cowshed. The battle is won by the animals.
The reason the people try to take the farm back is because they are worried about their own farms. There have been instances of rebellions on other farms and lots of animals are going around singing "Beasts of England." So the farmers are afraid that their own animals might rebel.
They think that if they retake Animal Farm, their own animals will know it's no use rebelling.