By using the hunting term "at bay," Rainsford means that his life is in jeopardy as long as Zaroff is alive, even if for the moment he has the upper hand. Nothing short of killing Zaroff and eliminating his rival in a definitive way will assure Rainsford his safety and get him off the island. Until then, he is held as a fox cornered in a hunt, with the hounds closing in, and his only recourse of action is attack.
Rainsfords says these words in the last scene when he ambushes Zaroff where he least expects - in his own bedroom back at the castle. Understood is that fact that Rainsford then "finishes him off" since he sleeps in Zaroff's bed that very night. The fact that Rainsford mentions that he slept very well also suggests that perhaps his own conscience (as Zaroff's naturally is) has been hardened through this harrowing "win all/ lose all" experience.