The Great Compromise solved the problem of representation in Congress during the Constitutional Convention. There were two competing plans to decide representation in Congress. The first, the Virginia Plan, was to provide Congressional representation according to a state's population. This would give the more populous states such as Virginia more power in the new nation. Not surprisingly, Madison was a Virginian. The New Jersey plan called for a unicameral legislature, where each state received one vote regardless of population. For example, this would give Rhode Island the same power as Virginia in Congress. This was also the same arrangement that had existed under the Articles of Confederation.

Roger Sherman of Connecticut came up with a solution that historians now refer to as the Great Compromise. Under the Compromise, Congress would have two houses. The first house, the House of Representatives, would be decided by population. This would incorporate the elements of the Virginia Plan. The second house, the Senate, would have equal representation regardless of a state's population. This plan made both the large and small states content.

While the Great Compromise decided the structure of the legislative branch, there were still issues as to how to decide population. Many Southern states had large slave populations yet wanted power in Congress. The Constitutional Convention had to decide how to count which people decided population and the manner by which the people were to be counted. All of this made it rather remarkable that the United States reached a compromise early in its existence that allowed states with different cultures to coexist within the same government.

The Founding Fathers met in Philadelphia in 1787 to amend the Articles of Confederation. The Articles of Confederation had been the basis of the first national government of the United States, but it was weak and ineffective. The delegates decided to scrap it entirely and write a new constitution. The delegates in Philadelphia considered the Virginia Plan and the New Jersey Plan before finally accepting the Great Compromise.

One key problem with the government under the Articles was its structure. There were no executive or judicial branches. There was only a unicameral legislature in which each state had an equal say—regardless of its population.

James Madison's Virginia Plan called for executive, judicial, and legislative branches. The legislature would be bicameral. Madison's Plan would have favored the more populous states.

Smaller states had enjoyed a great deal of power—equal to the more populous states—under the Articles, and they were not willing to accept lesser status in a new government. They countered with the New Jersey Plan: a unicameral legislature where states had an equal voice.

Deadlock threatened until Roger Sherman (1721–1793) came up with the Great (or Connecticut) Compromise. His solution was a bicameral legislature with a lower chamber (House of Representatives) and an upper chamber (Senate). The House members would be chosen by popular vote and more populous states would have more seats. The Senate would be chosen by state legislatures and each state would have equal representation.

Without the Great Compromise, the Constitutional Convention might have ended with great acrimony.

When the Constitutional Convention was being held there was a significant debate about the legislative branch. The large states wanted representation in Congress to be based on the population of a state. This would give large states more representation in Congress. The small states objected to this because they felt their voices wouldn’t be heard because the large states would have many more representatives than the small states would have. Thus, the small states wanted equal representation in Congress.

Roger Sherman developed a compromise that would satisfy both sides. There would be a legislature with two houses. The House of Representatives would have unequal representation. Representation would be based on the population of a state. Large states would have more representatives than small states. On the other hand, the Senate would have equal representation. Each state, regardless of size, would have two senators.

The Great Compromise resolved the issue of how representation would be based for the legislative branch of government.