Sammy refers to the regular customers in the A&P as sheep because to him, they are like a herd of sheep. They are timid and do what they are told. They flock, to his mind, into one indiscriminate herd. Calling them sheep shows contempt and helps Sammy make a distinction between them and the teenage girl he calls Queenie and her two companions. Sammy observes that
The sheep pushing their carts down the aisle—the girls were walking against the usual traffic (not that we have one-way signs or anything)—were pretty hilarious. You could see them, when Queenie's white shoulders dawned on them, kind of jerk, or hop, or hiccup, but their eyes snapped back to their own baskets and on they pushed.
The "sheep" in the quote above are a contrast to Queenie. She stands out, but they avoid trouble and mind their own business, even though it is clear they notice with surprise—a "jerk, or hop, or hiccup"—that the girls are dressed inappropriately for the store.
A short time later, Lengel, the manager, tells Queenie that she and the other girls have to come into the store with their shoulders covered, as this is store policy. Queenie argues back, but Lengel insists it is the policy. At this point, Sammy notes that the other customers, sensing a scene, stay away from the girls and all move together into the cash register line run by Sammy's friend Stokesie. Sammy once again contrasts Queenie's bold behavior to the compliant behavior of the regular customers:
All this while, the customers had been showing up with their carts but, you know, sheep, seeing a scene, they had all bunched up on Stokesie.
At this point, Sammy feels he has to make a decision, either to be one of the sheep or side with the girls, whom he admires. He knows he is one of the sheep. He, too, is a working class townie who does what he is told, not an affluent summer vacationer. However, he rebels against a life of being a sheep. He quits in solidarity with the girls and walks out. Once outside, he notes,
I could see Lengel in my place in the slot, checking the sheep through.
As he watches, Sammy realizes that his life has suddenly gotten much harder: he has made a bold decision, but it is easier to go through life as a sheep.