His society touts it as quite an honor, as a very important position. And it really is: he's the only one that holds the keys to the way life used to be. But imagine being the only one in an entire society that actually feels anything. Feeling joy is seriously diminished if you can't share that joy with someone else, and feeling misery is that much harder if you are the only one who is allowed to bear it. So in that sense, it is a punishment.