Goldstein was using Newspeak during the 2 minutes Hate. How could he be using Newspeak when Newspeak was invented by the Party?
Goldstein was originally a member of the party but he became an enemy after he disagreed with other party members around human rights issues. He is not simply an invention of the party but is a real person who has been effectively used, through the media and through careful manipulation of the masses, as the focus of hartred and the epitome of wrong-thinking.
He speaks the language of the party because he once belonged to it; however, his language quickly "deteriorates" into more complex language as his thoughts require it. This speech is used to make him look ridiculous and to heighten the isolation and hatred the common party members feel towards him.
It is also necessary that he speak the language of the party so that the masses can feel some connection with what he says initially because this heightens their sense of betrayal when he starts speaking a language that is beyond their comprehension later. Consequently, the masses feel more alienation and more hatred than if he would have simply been a stranger citing strange words they did not understand.
Great point. I think it might speak to the overall importance of newspeak to the party. Even when creating a false hope for anti-party thought, they can't bring themselves to abandon newspeak. One reason is that newspeak is essential to the party's control over people's minds. A Goldstein speaking in real English, even if he is a product of the party, might have the unintended consequence of helping to break the power of Newspeak.