In Chapter 58, Pip goes back to Joe's house to find Biddy and ask her to marry him.  He is very shocked to find that Biddy has just married Joe -- it is their wedding day.  That is what is new with them.

I think that what he begs them to do is to forgive him.  He asks that when they have kids they should not tell the kids that Pip was ungrateful and ungenerous.  In other words, he is begging them to forgive him for all the time when he never went to see them and never got in touch with them because he did not think that they were good enough for him.