O'Brien does seem insane and irrational to us. We are like Winston, we have the capacity to think, and it seems O'Brien does too, but he is choosing to believe in Big Brother.
O'Brien feels the need to explain to Winston the reason for Winston being there:
Shall I tell you why we have brought you here? To cure you! To make you sane! Will you understand, Winston, that no one whom we bring to this place ever leaves our hands uncured? We are not interested in those stupid crimes that you have committed. The Party is not interested in the overt act: the thought is all we care about. We do not merely destroy our enemies, we change them.
Further on O'Brien notes:
I enjoy talking to you. Your mind appeals to me. It resembles my own mind except that you happen to be insane.
These two quotes prove something to me. Because I understand this book to be a dystopian novel and because O'Brien calls Winston insane and himself sane, I believe O'Brien is insane and irrational. Also, because his face lights up at the opportunity to torture, that is irrational because no one should love the opportunity to hurt another, that is not human.