In my opinion, the incident with the bank notes shows us something about Joe's character. Specifically, it shows us that Joe is a very honest man.
When Joe gets home with Pip, he thinks that Pip has been given a shilling. But instead, he finds out that the shilling was wrapped in two bank notes -- one pound each. A pound was worth 20 shillings. So if Joe was excited about Pip getting a shilling, imagine how much two pounds was worth.
But instead of keeping this windfall, Joe runs back right away to try to give the money back. When he finds out the mystery man is no longer at the pub, he leaves a message for the guy in case he comes back.
This shows me that Joe is very honest -- he does not want to keep money if the man didn't mean for Pip to have it.
In Chapter Ten, Pip is given some money by a strange man in the Three Jolly Bargemen. Pip thinks it is a shilling coin wrapped inside some paper, but when he removes it from the paper, he sees that it is two pound notes. This is a considerable sum of money, which shocks Pip, Joe, and Mrs. Joe. In fact, Joe immediately returns to the Three Jolly Bargemen with the intention of returning the money to its owner.
By reacting in this way, Joe demonstrates that he is not accustomed to dealing with large and unexpected amounts of money. He has clearly lived a modest life. For Joe, giving Pip two pound notes is certainly a mistake and Joe's honesty shines through when he returns to the inn to give back the money.