First of all, please note that (in the way we usually use the term) the president does not have a check and balance relationship with the bureaucracy. Checks and balances are supposed to work between branches of the government. The bureaucracy is part of the president's own branch and so the president does not check or balance it.
The most effective check or balance on the bureaucracy is the legislative branch's power of oversight and budget. The legislature has the power to set the budget for any bureaucratic agency. It also has the power to summon officials from the agencies to appear at Congressional hearings to explain their actions (and often to be chastised by every member of the committee). These powers are used quite often as a way for the Congress (particularly when the President is of the other party) to affect what the bureaucracy can do.