Chapter 43 is when Pip decides to visit Miss Havisham and Estella one more time before he plans to go abroad with Magwitch to get him out of the country. One of the key literary aspects of this chapter is how day is personified as a beggar:
...I set off by the early morning coach before it was yet creeping on, halting and whimpering and shivering, and wrapped in patches of cloud and rags of mist, like a beggar.
I wonder if this is some kind of pathetic fallacy, whereby nature reflects the mood of the action, by suggesting that Pip, himself, feels like some kind of beggar going to Satis House again and pleading his hopeless case for Estella. This is an impression that is only heightened by the rest of the Chapter as Pip spars with Drummle over Estella.
You are correct with your quote. At the end of the Chapter, as Pip, depressed and sad at the discovery that Drummle has seen Estella, prepares himself to see Estella and Miss Havisham:
...I washed the weather and the journey from my face and hands, and went out to the memorable old house that it would have been so much the better for me never to have entered, never to have seen.
Pip is trying to make himself presentable, but also to "wash" the journey from his face and the weather too, suggesting he is trying to make himself cheery in contrast to the dank, dreary and depressing weather. Yet, with sinking heart, he begins to rue the day that he first went to Satis House and met Estella, realising how damaging that relationship has been to him. It is important to note that we see a Pip who is becoming gradually more and more self-aware about his relationship with Estella, and is maturing, able to reflect on his life and identify mistakes.