What does Jem do that shocks Dill and Scout?
There are several times in To Kill a Mockingbird where Jem shocks Scout and Dill. One of them comes in Chapter 1 when Jem finally accepts Dill's dare and rushes to the Radley house where he
... threw open the gate and sped to the side of the house, slapped it with his palm and ran past us... Dill and I followed on his heels.
Jem does it again in Chapter 4 when he rolls Scout--who is curled up in an old tire--right up to the steps of the Radley house.
Jem saves another surprise for Scout and Dill--and a group of neighbors as well--when he shows up at the Radley front gate in his underwear. Jem had to discard his pants earlier when he got them caught in the Radley fence and had apparently forgotten this when he joins the crowd.