I need three arguments against BIRT: Hugo's novel makes victims of us all.
First, I would consider the idea that not every character in the book is a victim. Some characters likely would not consider themselves victims because of the purpose with which they lived life. Secondly, keep in mind that one key theme of this novel is redemption and rebirth. This means that other characters who may start out as victims, through the experience of redemption, are reborn as conquerors.
I think your three arguments could come from three key characters who match one of the above descriptions. Jean Valjean, of course, is one such character. Not only is he reborn after being victimized by the justice system, but he goes on to rescue others.
Marius is never really a victim at all (certainly not by circumstances). He comes from wealth and chooses poverty for himself. In this way, Marius discovers a world in which he is unprovided for. He develops independence and strength.
Javert is another character who is arguably not a victim. He is a character of decision and purpose. I would not even consider his suicide an act done by someone who felt victimized. Rather, this may be his final solution to his unwillingness to forgive himself in the end.