George did not remove the lead balls from the bag around his neck because, as he mentions in the story, he's gotten used to them. It's one of the tragedies of that society that people's imposed handicaps are so familiar that they seem like part of them. He also doesn’t remove them because there would be a fine and penalty.

George does not remove some of the pellets from the fourty seven pound bag of bird-shot around his neck while he is at home because the thought really doesn't occur to him. Even if the thought did occur to him it would be driven from his mind almost immediately by the mental handicap he was given by Diana Moon Glampers, the Handicapper General. The reason the thought never occurs to him is because this is the society in which they live. The rules of the society are simple, everyone must be equal in every way. In order to make George equally strong as every other person, he must bear the burden of the 47 pound bag. These various handicaps are cultural norms in this society, like shaking hands to greet someone or saying, "Excuse me," when you burp, the handicaps are automatic and George even says when Hazel offers to let their equality slide so he can rest that he doesn't even notice the bag anymore because it's become such a part of him.