Between the start of the "age of discovery" in the mid-1400s and the independence of most of the last major European colonies after World War II, European powers like Spain, Portugal, France, Great Britain, and the Netherlands developed a wide variety of methods for controlling their colonies. The first European empires, like Spain, simply expanded the feudal method of rule (where land and the labor of people living on the land was given to loyal members of the noble class and/or the church in exchange for their support of the monarch) to the New World. However, this quickly proved to be an inefficient way of governing wide-ranging overseas territories and generating revenue for governments back in Europe. Over time, three major methods developed for European countries to govern their overseas territories.
The first of these was company rule. Joint-stock companies were some of the first modern economic institutions to develop in Europe. Where a single individual couldn't afford to invest in the labor, equipment, and other necessities of a business venture (such as a trip to the East Indies to get spices to sell back in Europe for a profit), several people with liquid capital (that is, capital in the form of money or some other form that can be easily transferred between people and places, unlike land and other older, more fixed forms of property that dominated the feudal era) would invest in a company, sharing the liabilities and the profits. This meant that joint-stock companies could engage in larger and more risky (and profitable) ventures than old feudal landlords could. Therefore, they were key to the establishment of European colonies around the world. The first way that European powers learned to govern their colonies was simply to give a joint-stock company a monopoly over colonial operations in a particular area and allow it to act like a sovereign power. An early example of this was the Massachusetts Bay Company that helped settle English colonists in New England in 1630. While the Plymouth Colony had been established ten years earlier by religious dissidents fleeing persecution, it was the Massachusetts Bay Company that firmly established New England as a colonial society by making the venture profitable. Companies like the Massachusetts Bay Company usually had a "charter" from the monarch of the country whose colony they governed. The most famous and probably the most important instance of company rule was probably the East India Company, by which the British gradually established control over India between the early seventeenth and the mid-nineteenth centuries.
As colonial ventures became even larger, however, they eventually exceeded the scope of what company rule could do. Just as companies represented a higher level of collective cooperation by the European ruling class than the individual ventures of feudal lords had, it became necessary to have a higher level of coordination and cooperation than companies had made possible. This was because colonial rule was becoming larger and more complicated. European settlers, especially where they were present in large numbers (as in North America and parts of Africa), needed to be governed. Colonies needed to be protected from rival countries, as was dramatically shown in the decades following Dutch independence from Spain in the 1560s, when the Dutch seized most Portuguese colonies other than Brazil (Portugal was under Spanish rule at the time). Moreover, where native people resisted colonial rule, they needed to be put down. In this regard, India was again a bellwether for the rest of the modern world. In 1857–58, the so-called "Sepoy Mutiny" (or the India Rebellion of 1857) nearly threw the East India Company out of India after the hired native soldiers turned on their colonial masters. In the wake of the rebellion, Britain replaced company rule in India with the British Raj, which was direct rule over India by the British government itself. This was the next level of ruling class collaboration after the advent of the joint-stock company. While the joint-stock company brought together more resources and individuals for greater and riskier colonial ventures than feudal lords, the modern state was able to establish greater levels of collaboration among the members of the European ruling class than companies had been able to facilitate and thus carry out colonial ventures of which the companies had been incapable. Where companies had dominated the colonial order in the seventeenth and eighteenth centuries, the state came to dominate colonialism in the nineteenth and twentieth centuries.
This came in two forms—which are the other two of the three forms of colonial rule, besides company rule, that I'm going to talk about here. The British Raj was an example of what is called "direct rule." This is where a European country would simply conquer a colonial territory and administer it as though it were part of the European mainland. The British also established direct rule in parts of Africa (most notably Kenya) and East Asia (e.g., Hong Kong and Singapore). The major country to use direct rule, however, was France, which created "overseas departments" in almost all of its colonies. A "department" was the French term for a province. Creating overseas departments was not at all unlike the way that, as the United States expanded westward, it encouraged territories to become states. Also like the United States, only Europeans living in France's overseas departments, not native people who had already lived there before the French arrived, were considered citizens. In 1924, the US extended full citizenship to indigenous peoples living in its conquered territories (and to the descendants of former slaves in 1964). Similarly, in the 1950s and 1960s, just as the civil rights movement was heating up in the US and many European colonies were gaining independence, France tried to do the same thing, offering to make all inhabitants of its overseas departments French citizens if they would remain a part of France. However, the colonies opted for independence and, in 1960, 14 of France's overseas departments in Africa became independent countries. France would not, however, allow for the same arrangement in Algeria, which had a large number of "Pieds-Noirs" (European settlers, called "black feet" in French because their feet were on the soil of Africa). This led to a protracted war, one of the bloodiest independence struggles in the modern world, and the collapse of the fourth French republic.
The case of Algeria demonstrates one of the key problems of direct rule: it brings problems in the colonies "home" to Europe. If European colonies really were part of European countries, that could create major problems for those countries. At the bare minimum, it meant developing a huge civil service and military bureaucracy to administer the colony, as the British did in India. At most, as was the case in Algeria, it could effectively plunge a colonial power into civil war. Many European countries, especially Great Britain, decided to reserve direct rule only for their most important colonies (such as India). For the remainder, Britain especially, as well as other major powers, used what was called "indirect rule," the third major form of colonial governance that I want to cover here. Here, local rulers would continue to rule their own people on behalf of the colonial power, which would often station some military and civil service personnel to "advise" the locals. This was the relationship that Britain had with many of its African territories, including Sudan, Zanzibar, and the so-called "fleas in the Queen's blanket" in Southern Africa—Bechuanaland (Botswana), Basutoland (Lesotho), and Swaziland. This was also the way that Britain governed much of the Middle East prior to World War I. Because they had an internal government, many of the colonies that had been governed by "indirect rule" gained independence with the least violence or chaos and have had some of the most prosperous postcolonial futures.
These, then, are the three most important ways that European powers governed their colonies: company rule, direct rule, and indirect rule. For more information, you might read Immanuel Maurice Wallerstein's The Modern World System and Martin Meredith's The State of Africa: A History of the Continent Since Independence.