This is a great question. The context is important to understand what is happening as well as to know what Rainsford's state of mind was.
In this context, Rainsford is running away. To put it another way, he is being hunted. He knows that General Zaroff is getting closer, because he hears the dogs. He realizes that he has to do something. Inaction would lead to his death. So, he decides to climb a tree. As he does so, he sees the general and Ivan with the dogs.
In a quick act, he makes a trap with his knife. Here is the quote:
They would be on him any minute now. His mind worked frantically. He thought of a native trick he had learned in Uganda. He slid down the tree. He caught hold of a springy young sapling and to it he fastened his hunting knife, with the blade pointing down the trail; with a bit of wild grapevine he tied back the sapling. Then he ran for his life. The hounds raised their voices as they hit the fresh scent. Rainsford knew now how an animal at bay feels.
Rainsford waits to see what would happen. Since the dogs are now quiet, he realizes that his trap has worked. Ivan is now dead. From this perspective, we can say that Rainsford's state of mind is one of relief. He has a small respite from being hunted. That said, he knows that the general is still alive and desirous of hunting him. So, his feeling of relief is not complete. I would say that his state of mind is one of relief mixed with fear. But I should also say that he is remarkably cool-headed in the process.