All Joe really had to do was speak to make Pip ashamed, but Joe took it further than that. I think Joe was so nervous to be in the presence of Miss Havisham that he lost common sense for a little while. When she spoke to Joe, Joe would respond as if he was talking to Pip. So each utterance out of his mouth would start with something like this, "Well Pip, you certainly know ..." and then he would fill in the blank.
This may have been the beginning of a long time wherein Pip finds great shame in his upbringing, but particularly in his old mentor Joe.