We are given the answer to this question at the beginning of Chapter 12, when Bernard is trying desperately to continue exploiting the savage to boost his own credentials in his society. Note the selected audience that has been invited to this "party" that is indicated in the rest of the chapter. However, what is clear is that John is fed up of the way that Bernard is using him, and he dislikes the way that he has been dragged around without any consultation or permission requested. Consider what the dialogue between John and Bernard reveals about his reasons for not appearing at the party:
"I asked them on purpose to meet you."
"You ought to have asked me first whether I wanted to meet them"
"But you always came before, John."
"That's precisely why I don't want to come again."
Here we clearly see that Bernard has exhausted John's patience and now John wants more control of his life and does not want to be carted around like an exhibition any more to be gawped at by strangers who treat him as a curiosity rather than a human.