The incompleteness of the fossil record is a result of the unusual conditions necessary to make a fossil. Normally when an animal or plant dies, decay eliminates the remains entirely. Soft materials decay by bacterial and fungal action while harder materials like bone and enamel may require additional action of weathering, but by and large most organisms decay completely.
Once in awhile, though, a dead animal or plant is prevented from decomposing by some aspect in the environment. For instance, if a tree falls in a peat bog, the low oxygen concentration of the bog will prevent the microbial action that would lead to decay. This first stage of fossilization, protection from the environment and biota that might degrade the body, is reasonably rare just on its own. Most animals are killed and eaten by other animals, not sealed under a protective layer of thick mud when they die. Simple protection is enough to preserve a trace fossil (footprints, etc) for millions of years if mud hardens into sedimentary rock. Body fossils (bones) generally only survive tens or hundreds of thousands of years if all they have is this level of protection.
For long-term preservation of body fossils, mineralization is necessary, and there are several kinds. Most require the fossil to be exposed to ground water with a high concentration of minerals. The minerals precipitate out in the empty spaces of the bones, and as the bones degrade, the minerals fill the voids and form hard crystals. This is rare on its own, and only a fraction of the protected body fossils will mineralize.
Finally the third chance event that must take place is that the fossil must be found and recognized. Most of the bones that have ever fossilized have been broken into chips by time and natural erosion. For those that have not been ground to powder, it almost always takes an expert to recognize them in the field.
That is why the fossil record has gaps. While every ancient animal or homonid must have died, only a small fraction were protected from decay, a smaller fraction yet had their bones mineralized, and only a tiny fraction has been dug up and classified by a paleontologist. As such, the majority of ancient species have never been found in the fossil record and classified.