The bureaucracy includes federal agencies that implement public policy. The bureaucracy includes the 15 cabinet positions (such as Secretary of State, Secretary of Defense, Secretary of Labor, etc.), independent government agencies such as the Environmental Protection Agency (EPA), government regulatory agencies such as the Federal Reserve, and government agencies such as the U.S. Postal Service. Most of these agencies are controlled by the executive branch, though some (such as the Library of Congress) are controlled by the Congress. Technically, all federal agencies serve under the President and are subordinate to him or her.
The Congress can exercise checks on the bureaucracy by carrying out monitoring and review of the bureaucracy through its implied powers, given in the Constitution. For example, the Congress can vote to change or regulate an agency, as it did in the 1980s over the EPA. Congressional oversight is intended to protect the public from poor management or waste, make sure agencies are running efficiently, and protect people's rights. Congress monitors the executive branch through its Government Accountability Office (GAO). In addition, when a President nominates cabinet members, they must be confirmed by the Senate with a simple majority. In very rare cases, the Congress can impeach cabinet members.
The Supreme Court can also declare regulations by specific agencies unconstitutional or take legal actions against bureaucratic agencies. For example, the Supreme Court recently (in 2016) blocked the EPA's Clean Power Plan intended to limit greenhouse gases until courts could review the law after 29 states brought a lawsuit against it.
The main checks on the bureaucracy come from the elected parts of government. Specifically, both the Congress and the president have ways to check the exercise of bureaucratic power.
The president can check the bureaucracies by appointing high level bureaucrats. These appointees are supposed to control the bureaucrats below them and keep them from exercising too much power. At the same time, the president makes the departments' budget requests. The president can stop requesting money for agencies and programs he does not like. This prospect keeps bureaucrats from just doing whatever they want.
The Congress has the power of oversight over bureaucracies and, most importantly, the power of the purse. If the Congress does not like what an agency is doing, it can call bureaucrats before hearings and scold them. The bureaucrats need to pay attention to what Congress says because Congress is the body that determines how much money each agency actually gets. Congress can also change the laws under which the agencies work. This is another check on the bureaucracy because Congress can make new laws affecting agencies that act in ways Congress does not like.
In these ways, the elected parts of government have a lot of control over the bureaucracies. When bureaucrats exercise "too much" power it is because the elected bodies allow them to do so.