What was happening to Nellie, and why did Billy have to kill her to take out the colt?
Nellie's muscles were contracting desperately in trying to birth the colt, but the colt was turned the wrong way inside her and could not be delivered in the normal way. In most cases, colts, which weigh 70-90 pounds at birth, are positioned inside the mother's womb so that the front feet emerge first, followed by the nose and head. If the colt is in the wrong position and cannot be repositioned with help, the delivery will have to be by cesarian section, a process by which the baby is removed surgically, or the colt and the mother will die. Billy had no access to medical help, so in order to save the colt and spare the mother from prolonged agony, he killed Nellie by smashing her head, and quickly cut her open to get the colt out.
Nellie, Jody's mare in The Red Pony, was having trouble giving birth because the colt was turned the wrong way inside her. Billy knew that she would trying to give birth and that the colt would probably die as well. By killing the mare, he does the only humane thing he can think of to do; rather than let both horses die in agony, he kills the mare, cuts her open, and removes the colt.
Nowadays, we would probably call a vet to help with the delivery, or some horse owners might even try to turn the colt the right way around for birth.