In Chapter 58, Pip goes back to Joe's house to find Biddy and ask her to marry him. He is very shocked to find that Biddy has just married Joe -- it is their wedding day. That is what is new with them.
I think that what he begs them to do is to forgive him. He asks that when they have kids they should not tell the kids that Pip was ungrateful and ungenerous. In other words, he is begging them to forgive him for all the time when he never went to see them and never got in touch with them because he did not think that they were good enough for him.
In Ch.58, Pip returns to his native village a much reformed and repentant man hoping to propose to Biddy and confident that she will accept his proposal. But when he arrives at his childhood home he is surprised to find that it is Joe and Biddy's wedding day. They have just been married and they are husband and wife! So once again Pip's 'expectations' have not been fulfilled:
I looked at both of them, from one to the other, and then --
`It's my wedding-day,' cried Biddy, in a burst of happiness, `and I am married to Joe!'
After Pip had recovered from this pleasant shock he begs Joe and Biddy to forgive him all his ungrateful behaviour and begs them both that in future when they have a son of their own they should not tell their son how thankless and ungenerous he has been. Both Joe and Biddly of course refuse to do just that:
`But I must say more. Dear Joe, I hope you will have children to love, and that some little fellow will sit in this chimney corner of a winter night, who may remind you of another little fellow gone out of it for ever. Don't tell him, Joe, that I was thankless; don't tell him, Biddy.
In Ch.59, Pip has decided to remain a bachelor for the rest of his life consequent to his failure of his 'expectations' in getting married either to Estella or Biddy and he begs Biddy to allow him to adopt their son Pip. Biddy refuses saying that he must get married and have children of his own:
`Biddy,' said I, when I talked with her after dinner, as her little girl lay sleeping in her lap, `you must give Pip to me, one of these days; or lend him, at all events.'`No, no,' said Biddy, gently. `You must marry.'`So Herbert and Clara say, but I don't think I shall, Biddy. I have so settled down in their home, that it's not at all likely. I am already quite an old bachelor.'