The effect of this failure was that it made Broca's experiment much less valid. It made it much less likely that his results truly showed what he claimed that they showed.
When doing an experiment, you have to make sure that there are no confounding variables that might invalidate your experiment. You have to be as sure as possible that your independent variable (in this case, sex) is the only cause of variation in your dependent variable (in this case, brain size). In Broca's case, height (along with body size in general) is a confounding variable. It could be that the men had bigger brains because they were bigger people. Because Broca did not take this possible factor into account, we can be much less certain that his experiment really proves what he says (that men are more intelligent) that it does.