Empathy is having the feeling or experience of sharing someone else's feelings. This is a significant theme in the book. It is often described as "putting yourself in someone else's shoes." In sharing someone's feelings, you share his or her perspective on things or on the world in general.
When Jem is "sentenced" to read to Mrs. Dubose, Scout goes with him. Seeing Mrs. Dubose lying in bed, looking sickly, Scout notes that she feels sorry for her. Feeling pity is more of a description of sympathy, but this is related to empathy:
In the corner of the room was a brass bed, and in the bed was Mrs. Dubose. I wondered if Jem’s activities had put her there, and for a moment I felt sorry for her. (Chapter 11)
At the beginning of Chapter 12, Scout is unhappy that Dill has not come for the summer. She reminisces and realizes that Dill is an important part of her life, especially during the summer. She says, "summer was the swiftness with which Dill would reach up and kiss me when Jem was not looking, the longings we sometimes felt each other feel." They feel the same longings. This is literally what empathy means. They feel what the other is feeling.
Near the end of the book, Scout walks Boo Radley to his home. She stands on his porch and contemplates Boo's perspective. She means this literally since she is standing on his porch looking at the street. But she also means this figuratively in that she realizes how important it is to consider someone else's perspective, to imagine how he sees the world, and how he might feel (empathy) about how he is treated:
Atticus was right. One time he said you never really know a man until you stand in his shoes and walk around in them. Just standing on the Radley porch was enough.
For Atticus, empathy is about feeling and "knowing" what another person is going through. He does this religiously, even with Bob Ewell. (Chapter 23)