They leave because Fiver has a dream in which the warren is covered in blood. He's something of a seer: able to witness some, but not all of the future. So, the others that leave take his dream seriously. They believe that danger is imminent, & they want to escape before that danger comes. Of course, not everyone believes Fiver. In fact, most of the other rabbits mock & ridicule them for leaving, choosing instead to stay where they've been safe all their lives.

I don't think they left too early. I suppose any time could be considered too early, because they were not used to being on their own, & none of them really knew how to survive outside a warren. But they learned quickly, & they escaped the fate awaiting those who stayed behind. If they had waited any longer, they may have lost the nerve to leave, or would perhaps have been talked out of their plan. As it was, they encountered many other dangers on their adventure, but they were safe from the poisoning and bulldozing the original warren suffers.