We might argue that the greed of the pigs and the corruption they display is the cause of the farm falling away from its ideals and principles.
We might argue, alternatively, that the pigs were allowed to take more power than they deserved and were not challenged enough by the other animals. The gullibility of the animals coupled with their desires to 1) keep Jones from returning at all costs and 2) live a life of ease can be cited as the reasons that Jones does return (in the guise of Napoleon) and a life of ease always escapes them.
Yearning for a paradise of their own, the animals choose again and again to believe that they have achieved it instead of keeping their leaders in check.
Another argument to be made is that the farm's downfall is due mainly to the animals lack of intellect. When they are fooled, they only sense this is the case. They are incapable of proving it.
I think that there might be a couple of ways to approach this particular question. On one hand, it can be argued that the farm does not suffer a downfall. Napoleon has brokered contracts and agreements with his neighbors and the farm will probably experience a great deal of success. If one argues that the farm suffers from a downfall, it is because they believe that the leadership has deviated from its goals in the revolution. The fact that the farm has become more of the same under Jones, the name of it has reverted back to its original state, and that the animals peering through the window are unable to tell the difference between the humans and the pigs all serve to represent how far the farm has come from its professed aims of change. In viewing the farm in this manner, one concludes that the farm is a failure because it has done everything that Old Major warned against in his speech. The farm is not a source of liberation for the animals, but rather a source of repression for them. Yet, this is precisely Orwell's point in that there is a difference between declaring independence and maintaining it. There is a difference between those who fight for freedom and those who maintain political control. This becomes one of the basic themes of the work and being able to judge the leadership of the farm becomes one of the central points of the novel.
In theory the Farm is not having a downfall - it's richer than when it was under Jones, but only the Pigs and the Dogs can benefit from this, while the other Animals continue to suffer.