How does science fiction as a literary 'genre' distinguish itself from fantasy?
Science Fiction is usually about the future. The stories and novels usually involve space ships, future technology and futuristic concepts. I have found that "fantasy" is more about faeries, witches, wizards, Fae, and goblins. In the Fantasy genre the stories usually are epic, often covering several volumes and a great deal of time. This of course is not always true, but generally speaking fantasy is about the past cultures. Many of my students love fantasy, and stories about dragons, wizards etc., but will not even pick up a Sci-fi book. I think Orson Scott Card said that SciFi is based on real world laws (i.e. physics), where Fantasy the writer can create their own law in their own universe. It doesn't preclude mixing, but one will have to be the dominate rule. A lot of fantasy does seem to also focus on magic or powers, where SciFi has a lot of technology, especially space travel.
As a literary genre, Science Fiction distinguishes itself from Fantasy in the elements of technology and science as opposed to magic and the supernatural. Most of both genres have similar patterns with a human component, different worlds, creatures or aliens and a quest.
With Fantasy, supernatural forces, divine intervention and/or magic are major elements that make up the genre. They are linked to the realm of that particular story line.
Science Fiction removes the magic and the supernatural with science and technology. Whereas a wizard may heal the sick through touch and a chant, in science fiction, a scientist will find a cure for an illness through research and a quest for an elusive agent.
In my opinion, this distinction is increasingly becoming blurred. In the 1980s, Gene Wolfe began his Solar Cycle novels which at first feel like a fantasy novel and then we realize that they are set in a future with robots and space travel. China Mieville's Bas Lag novels are more recent examples that attempt to bridge the gap between fantasy and science fiction to create something new. At best, when distinguishing between these two genres, we can speak to tendencies. However, at the book store, you'll find fantasy and sci-fi shelved together, to the point that many fans refer to them both under the broad acronym, SFF (or science fiction / fantasy).
Several years back, one of my students submitted a story to Marian Zimmer Bradley's magazine which he thought was fantasy. Instead of a plain rejection she responded by defining the two genres as she saw them. The distinctions between the two seemed to clarify the difference between the two for me. Essentially she said that Science Fiction does not change the world but deals with the effects upon the world by men and their actions while Fantasy creates new world with new inhabitants and new rules.
So Robert Heinlein's space stories are science fiction because they deal with characters that are human in a world run by human rules that men have changed through science, but Tolkien's works create new worlds and non human characters who may or may not have human qualities which classifies it as fantasy. Simply, if it is robots or science gone wild in a human world, it is science fiction. It is fantasy if it is set in a world that contains elements created only in the author's mind.