I definitely think there are elements of fantasy in this excellent short story, but overall, I think this story cannot be called an example of the fantasy genre, because of its ostensible lack of paraphernalia that we normally associate with fantasy texts.
Let us consider the elements which are rather fantastical in this short story. First of all, let us note the way that the exact time and the exact location are both unspecified. We are presented with a world unhinged from reality in this sense. Secondly, consider the fantastical description of Ship-Trap Island with its "giant rocks with razor edges... like sea monsters with wide-open jaws." Thirdly, consider the way that the island is extremely isolated and operates away from reality and laws of normal civlisation. In this domain, General Zaroff is like a despotic ruler from some fantasy story, where his dreams can become reality.
However, let us also note the complete absence of any of the ingredients that are normally included in fantasy stories. There are no swords and sorcery, no magicians or little people with big feet and no mysterious dark forces that are supernaturally evil. Zaroff is presented as a man who is losing his grip on reality through his moral code rather than a force of pure evil, and the reality of the struggle between him and Rainsford is stressed throughout.