The rabbits are shown to live mostly in harmony with nature, since they are small prey animals and need to keep a low profile. As long as they avoid predators, they have happy lives, and do not affect nature much except to eat grass and other plants. Humans, in contrast, are shown to have no thought for either their effect on nature or on the rabbits; since humans assume that animals are non-sentient, they have no reason to avoid harming or killing rabbits if they deem it necessary.
"[Toadflax] was clear-headed for a short time before and I remember something he said. Bluebell had been saying that he knew the men hated us for raiding their crops and gardens, and Toadflax answered, 'That wasn't why they destroyed the warren. It was just because we were in their way. They killed us to suit themselves.'"
(Adams, Watership Down, Google Books)
The humans have no comprehension that the rabbits are wilfully living rather than just existing, and so see nothing wrong with trapping and eating them. To the rabbits, the humans are simply predators higher on the food chain. Without any possible communication, each is mostly unaware of the motivations of the other, and so what seems like an idyllic life for the rabbits is just wildlife to the humans. Human exploitation of the rabbits is not intentionally cruel, but predatory; in that sense, the rabbits are correct. Human actions are generally seen as destructive, since the rabbits don't understand human society; rabbit actions, to the humans, can also be destructive, since they eat gardens and undermine fields.