The term “bureaucracy” when used in reference to the federal government usually refers to the departments, bureaus, agencies and other federal organizations staffed with experienced people who were not elected to office. The federal bureaucracy began back in 1789 with about fifty employees in the Department of State, the Department of the Treasury, the War Department, and the Justice Department. Constitutionally, there is no Article or Section that deals specifically with the creation of a government bureaucracy, but there are parts of the Constitution that speak indirectly of the formation of a bureaucracy. Article II, Section 2 states “The President … may require the Opinion, in writing, of the principal Officer in each of the executive Departments…” It also states that the president will have the power to appoint “…all other Officers of the United States, whose Appointments are not herein otherwise provided for, and which shall be established by Law…”
Most importantly, there is the “Necessary and Proper Clause” found in Article I, Section 8 that gives Congress the power “To make all Laws which shall be necessary and proper for carrying into Execution the foregoing Powers, and all other Powers vested by this Constitution in the Government of the United States, or in any Department or Officer thereof.” This is sometimes referred to as the “elastic clause”. In essence, this gives Congress the power to create all the agencies, bureaus and other government organizations that make up the bureaucracy that are necessary to keep the government functioning.
The idea of a "specific" place in the structure of American Constitutional government is an interesting one. I think that much of it largely depends on how one perceives the role of government, in general. The Constitution provides specific elements that need to be achieved and the role of government becomes one realm where there is disagreement as to how these ideals and realities should be achieved. We could probably trace this back to the emergence of the first political parties, the Federalists and the Republicans. The former believed in the strength of the central government. They probably would have seen the bureaucracy as one that was necessary in order to bring forth the Constitutional premises into reality. They were in opposition to the Republican party, which favored a more localized reality of government, where government bureaucracy was lessened in favor of laissez faire principles that called for a lesser role of federal government. The Federalists viewed central government and its bureaucracy as an expression of national freedom and identity of the best and strongest leading a nation. The Republicans viewed it as a reflection of inefficiency and intrusion, something to be limited and constrained. To a great extent, we can see this brought out in the belief systems of today, where the role of government in achieving Constitutional ideas is constantly debated. For the most part, I think that we have understood the need for a bureaucracy to be present in that our social order and the demands it exacts are ones that require it. There are those who believe that this bureaucracy is too much and should be curtailed immediately. At the same time, there is a perception that extending this bureaucracy might not be a good thing, but the overall perception of place of the bureaucracy is probably more dependent on political preference which can be traced back to the fundamental belief of one's view of government.