Ivan gets killed by a knife-spring trap. Zaroff's favorite dog is killed when it falls into a pit of spikes. Zaroff is killed by Rainsford.

These moments of violence take place outside the active narrative perspective, as pointed out above. The violence is more factual than direct; distanced from Rainsford and the audience by a physical distance described in the narrative. 

Let's not forget the crew members of the Spanish ship San Lucar that had the "bad luck" (as Zaroff explains) to go on the rocks. He was keeping them in his training school in the cellar. One of these men would have been the quarry when Zaroff complained that he was bored again because the sailors offered no challenge. This mockery makes the violence even more grim.

eNotes has some great resources that you can use to understand the story better.

Zaroff is injured by one of the traps, and his bodyguard and one of his dogs are killed. Gunshots are heard just before Rainsford arrives on the island, presumably from the gun Zaroff uses to hunt his human prey.

There is not really anything in the way of direct physical violence in the sense of one person hitting or stabbing another.  However, there is indirect violence (the traps) and implied violence.  At the very end of the story, Rainsford and Zaroff are in a standoff against one another.  It is implied that one will live and the other will die.  It turns out that Rainsford lives, implying that he has killed Zaroff.