Why is anything controversial? Because people disagree about how they think the world should be.
Part of what makes feminism controversial is that different people seem to define it differently. The most basic definition is "women should have equal rights to men", which used to be highly controversial (and still is in some parts of the world), but at least within First World countries is largely accepted and agreed upon. There are also much stronger, more specific definitions that fall under the umbrella of "feminism", which can include everything from legal abortion to affirmative consent laws. Many of these more specific policies are where the real controversy lies: Simply agreeing in principle that women deserve equal rights doesn't necessarily convince you that abortion should be legal under all circumstances. They aren't wholly unrelated, but additional premises are needed to tie them together, and that fact often gets ignored.
But part of what makes feminism controversial is... sexism! A lot of people around the world still have some really deep-seated beliefs and attitudes that are extremely sexist. Some of these were probably learned in childhood, or picked up gradually as life experiences; some may even be linked to genetic traits that evolved thousands of years ago. But for whatever reason, a lot of people really do feel strongly that men and women should have fundamentally different roles in society and should not be treated the same way---and those people are going to get angry when you try to do otherwise.
Especially in cultures like the US and Europe where we all agree in principle that women are equal, people tend not to think of these attitudes and beliefs as sexist; but by definition they still are. For instance, there is a common norm "men should hold doors for women"; well, that doesn't sound like such an awful thing, right? But it's directly assigning different social roles to men and women---and that makes it by definition sexist. Benevolent? Sure. Harmless? Maybe. But sexism nonetheless. A lot of people have general ideas about men and women that they don't think of as bias, they just think of them as "how things are".
Finally, one thing that makes sexism in particular very complicated in a way that other forms of bigotry such as racism and ethnocentrism are not is that sex is a real thing. The way we define cultures, nations, and even races is basically arbitrary; it's a socially-constructed phenomenon with little if any real basis in underlying traits. But sex isn't like that; humans actually are, biologically, a sexually dimorphic species, and certain differences between men and women really are genetically defined---testosterone levels, height, and upper body strength are obvious examples. Sometimes it's fairly obvious that a given difference is purely societal (the stereotype "women are bad drivers" can't exactly be something we evolved thousands of years ago), but often it isn't. Many feminists try to deny this and say that there are no real genetic differences between men and women, but that just isn't true. We don't know what differences are genetic; perhaps only a few are. But some definitely are, and we need to face up to that. Clearly human behavior isn't entirely genetic, and I'm not aware of anyone who thinks it is. But it is partly genetic, and many feminists have been strangely unwilling to accept that fact---which very likely contributes to why feminism is so controversial. I think feminists would do themselves a favor by admitting that some differences between men and women are at least partly genetic, and trying to figure out ways to ensure equality despite that fact.