Ah, great question.
Shirley Jackson's great short story reveals several related traits or qualities that it is important to have.
The first of these is that a leader should be aware of the consequences of his or her actions. Miss Strangeworth lacks this quality. She thinks that her actions are well-meant. She thinks she's helping the people in her town, or at least trying to. However, look at what happens when she interacts with the Cranes. Her well-intended advice ends up making Helen Crane feeling "ashamed" about how her daughter is developing, and probably fearing that she (the daughter) is slow.
A second quality a leader needs is self-awareness. Miss Strangeworth lacks this quality too. Miss Strangeworth pays attention to failings and weaknesses in others. She sees the signs of age in Mrs. Harper, but she does not see them in herself--doesn't realize that her judgment may be off, or, more simply, that her hearing might be failing her, as a neighborhood boy thinks later in the story.
A third quality it would be ideal for a leader to have is modesty, and Miss Strangeworth lacks this quality too. Yes, her "grandfather built the first house on Pleasant street," as she says, and her family played a major role in developing the town. However, no one elected Miss Strangeworth, or trained her for her position of moral guardian. Even if her family founded the town, does that make her wise enough to guide it? No.