Ponyboy feels that Darry doesn't love him or respect him as much as he does Soda Pop. Ponyboy is wrong, but the feelings are understandable. Darry is hard on Pony, and Ponyboy feels that Darry is always criticizing him. In general, Pony feels that Darry has only negative things to say about him. The time that Darry hit Ponyboy doesn't help things either. It's hard to fault Darry though. He's been put into a situation that he should not be in. The brother's parents are dead, and Darry has been placed into the position of parent and caregiver. That's a tall order for a teenager, and Darry doesn't know what he is doing.
After the church fire, Ponyboy realizes that his thoughts about Darry were wrong. Ponyboy sees exactly how much Darry cares for him and the rest of the Greasers, and Pony realizes why Darry was so hard on him. It wasn't because Darry was being mean for no reason. It was because Darry saw a lot of potential in Pony and wanted to push him hard to bring out that potential.
"In that second what Soda and Dally and Two-Bit had been trying to tell me came through. Darry did care about me, maybe as much as he cared about Soda, and because he cared he was trying too hard to make something of me. When he yelled "Pony, where have you been all this time?" he meant "Pony, you've scared me to death. Please be careful, because I couldn't stand it if anything happened to you."
Ponyboy feels he can never do anything right in his brothers eyes. Darry wants and expects Ponyboy to put 100% effort in everything he does and only wants the best for him.