Ponyboy and Johnny spend five days at the church. After the episode with the Socs at the fountain, which leaves Cherry Valance's boyfriend Bob Sheldon dead, Ponyboy and Johnny flee to an abandoned church, which Dally knows about, on Jay Mountain. Dally gives them a gun and advises them to buy a week's worth of food when they get there. Ponyboy remembers that he hadn't been to church since Sodapop was caught clowning around during a sermon, and the brothers had never gone back. He also says that this church gave him a "creepy feeling."

On the first morning, Johnny leaves the sleeping Ponyboy to go to the store. He comes back with a week's supply of baloney, bread, cigarettes, and matches. He also buys a paperback copy of the novel Gone With the Wind. Both the novel and a poem Ponyboy recites, "Nothing Gold Can Stay," are symbolic of the boys' loss of innocence.

On the fifth day, Dally shows up bringing Ponyboy a letter from Sodapop and then taking the boys for hamburgers at a Dairy Queen. Johnny tells Dally that he wants to return to town and give himself up. Dally tries to talk him out of it, but their discussion is interrupted when they return to find the church in flames (probably because the boys have left a burning cigarette) with a group of school children standing around outside. Some of the children have gone into the church, so the three greasers go into rescue mode with unfortunate results for Johnny.

After the fight with the Socs, which leaves Bob dead after Johnny stabs him in self-defense, the boys are terrified. The boys go to Darry for advice and Darry tells them about the church and how to get there. The boys spend several days in the church. During their time in the church, the boys decide to cut and bleach their hair, in order to try to make it easier to avoid the cops, who must (they think) be on the lookout for them.

On the fifth day, Darry comes out to see the boys and they leave the church to go get lunch with him. When they return, they see that the church is on fire and they work hard to rescue the kids they believe are trapped inside.

So, the short answer is, they were in the church for five days.