Constitutionalism is an idea that is very similar to the idea of limited government.
In a system that believes in the idea of constitutionalism, the government cannot simply do whatever it likes. Instead, it is limited in what it is allowed to do. The limitations on the government are imposed by a set of rules that are meant to delineate what the government may and may not do. These rules (whether written, as in the US, or unwritten as in Great Britain) are generally known as the country's constitution.
Under constitutionalism, then, a government is bound to abide by a set of rules that are known to everyone. This limits what the government may do and requires that it abide by the rule of law.