Boo has spent years watching the kids playing, getting into mischief, growing up, changing, and dealing with life. Through his window, he has seen so much of their lives that he feels like he knows them. He is completely ostracized from all other social contacts, so his indirect relationship with Scout and Jem must mean a whole lot to him. He has no other people in his life that bring him any form of happiness; it is just Jem and Scout. When he places gifts in the tree for them, that is his attempt to try to show them that he cares for them and wants to be friends. And at the end of the book, he has risked life and limb to save Jem. So, imagine you have developed a very close relationship with someone, but never really been near them. The first time you actually come into contact with them, they are in pain and passed out from a broken arm. When Boo wants to pet Jem, it is just him expressing his love and concern for him. It is also taking a very much desired and deserved chance to make the moment real; he is really there, Jem is really there, and he is at last close to the two people that have made life better for him. And it is just brief moment that "his hand came down lightly on Jem's hair" before Scout senses that Boo wants to go back home to where he is more comfortable. But being able to see that Jem is okay, and to touch him in comfort for that brief moment, probably meant quite a bit to a man who has probably never been comforted or cared for in any kind way in his entire life.
I hope that those thoughts help; it's a touching chapter, and a touching moment when Jem shows just how concerned he is for Jem, and just how much he cares for his welfare.