Political scientists generally say that there were three party systems during this time.
The first party system pitted the Federalists against the Democratic-Republicans. This started during the Washington administration as Alexander Hamilton and Thomas Jefferson started de facto political parties that became more formal in the early 1800s. This system lasted until 1812 or 1816, depending on how you define it.
The second party system is known as the Era of Good Feelings. After the Federalist Party died off, the Democrats dominated American politics and there was no other serious party. This lasted until the 1824 election made Jacksonians angry at the supporters of John Adams.
This split in the Democratic Party led to the third party system. This had the Democrats being opposed by the Whigs. This system lasted until the Civil War, although it was falling apart by the late '50s as the Whigs broke up and were succeeded by the Free Soil Party and, eventually, by the Republicans.