What quotes from The Giver show that the Elders have power?
In The Giver (Lowry), it is often just implicit that the Elders have all the power, but there are some quotes that do make it explicit. I'll provide a few.
In the scene in Chapter 6 in which Jonas is anxiously awaiting his assignment for the Ceremony of Twelve, how Jonas' parents were matched is explained this way:
Even the Matching of Spouses was given such weighty consideration that sometimes an adult who applied to receive a spouse waited months or even years before a Match was approved of and announced (48).
So, we know that people cannot be married without the approval of the Elders. Furthermore, who has children and which children they get is controlled by the Elders, as we are told in this passage about Jonas' getting Jonas and Lily to raise:
Their match, which like all other Matches had been monitored by the Committee of Elders for three years before they could apply for children, had always been a successful one (48).
In addition to deciding who can wed and whether or not people can raise children, the Elders decide what everyone's job is going to be, which is what Jonas is waiting to hear at the Ceremony of Twelve, and we learn that the "initial speech...was made by the Chief Elder" (51). In that speech, she explicitly states it it is the Elders who have made the assignment decisions, "paying tribute to the hard work of her committee, which had performed the observations so meticulously all year" (52).
While we do not know right away about the Elders, it becomes clear very quickly in the story that there is someone or some ones making all of the rules we read about, and once we learn about the Elders, it starts to become clear that they are the ones with all the power in the community. Giving up the memories of a community can render a community powerless.