Pip's statement that Biddy has done "an injustice" and "an injury" to him is ironic because she has not done him wrong at all. Biddy has only spoken the truth, and in reality, it is Pip that has been unjust and injurious to her and Joe.
After the death of Joe's wife, who is Pip's sister, Pip promises pompously that he is not "going to leave poor Joe alone," and will come to see him frequently. Biddy does not respond to his declaration, leading Pip to feel hurt and to ask the reason for her silence. Biddy asks him, with a "clear and honest eye,"
"Are you quite sure, then, that you WILL come to see him often?"
Pip is offended that she should doubt him, and reflects upon
"what an unkindness, what an injury, what an injustice Biddy had done (him)"
Joe, despite his rough country manners has always treated everyone, and especially Pip, with deep love and respect, but since Pip has come into his "expecations" and moved up in the world, he has become ashamed of his uncle and has acted like he is very much better than him; evidence of this uncomfortable change on the part of Pip is shown by the fact that Joe and Biddy feel they must call Pip, the boy Joe has cared for, "Mr. Pip." Biddy has reason to doubt that Pip will keep his word and do right by Joe, and Pip, though he is initially hurt, admits as much when he says,
"...the mists...if they disclosed to me as I suspect they did, that I should not come back, and that Biddy was quite right, all I can say is - they were quite right too" (Chapter 35).