The term cognitive estrangement comes from Darko Suvin and goes hand-in-hand with its key element called "novum." Together, they form the distinctive features of what "true" science fiction is (at least in Suvin's mind): an exploration of something completely new, something that has no equal or counterpart in the real world. It is a difficult thing to achieve, since all fiction is limited to our imagination, and the idea of trying to conjure up something beyond that imagination seems paradoxical. Literary scholars tend to point toward Stanislaw Lem's Solaris as one of the few examples of a literary work achieving this nigh-impossible concept—that and Ursula K. Le Guin's The Left Hand of Darkness.
There are many levels and themes of cognitive estrangement in this novel, but the one that receives the most attention—and is therefore the most successful in provoking discussion—is the gender/sex topic.
On the planet Gethen, the people have no fixed gender. All of them are capable of bearing children, but they spend most of their lives being genderless. That is a novum to both the reader and Genly Ai, who struggles to communicate outside of his expected social norms. Here, I believe, lies the brilliance of Le Guin's work. She's been criticized a lot for the novel's exploration of gender—for not pushing it further, for still portraying gender stereotypes, and so on. Some of these points have a sound basis. While the Gethenians are not exclusively male or female, the female period of their lives brings out softness, timidness, and other verystereotypically feminine characteristics. But by just exploring these issues, I would argue, Le Guin has allowed the readers to take a look at gender as a construct. Even if the author herself has failed to completely deconstruct the world revolving around gender, it's possible to use the novel as an example of a philosophical study of the question.
The novel challenges our idea of society by showing us a picture of another, one so different that we strain to see how it all fits together. That is the essence of novum, of cognitive estrangement. It forces us to question what our own world would be like if the boundaries created by gender/sex were removed. I think the core of this novum is that we believe we do not "see" gender as much as we actually do. Only once it's been removed from the equation do we start to notice what role it played. Genly Ai feels the same. For example, he starts off thinking of Estraven as a man. After all, "he" is a government official, occupying a traditionally male position. "His" characteristics, body language, and everything else label Estraven to be a male in Ai's book. Then, when "he" becomes female during kemmer, Ai feels like he needs to re-calibrate the way he interacts with Estraven. The question Le Guin raises is, why? It's the same person, after all.