When Pip returns home at his sister’s funeral, his relationship with Joe is still awkward and Pip leaves the next day.
Pip makes it clear that he is ashamed of Joe. When he gets his expectations and heads off to London, he does so without looking back. When Joe visits, he is embarrassed and even calls Pip “sir” because he feels the awkwardness.
Pip is happy to have him leave. When Pip returns home for his sister’s funeral, he still holds himself above his family. He stays at the Blue Boar instead of at home, for one thing. He attends the funeral, but stays afterwards. Things were very formal.
But after dinner, when I made him take his pipe, and when I had loitered with him about the forge, and when we sat down together on the great block of stone outside it, we got on better. (ch 35)
Joe and Pip have lost the closeness they had when Pip was a child. At that time they were both equals. Now Pip has placed himself above Joe, and Joe is very much aware of it even if Pip does not acknowledge the hurt he is causing.