After the Federalist Party died out, we entered a time period called the Era of Good Feelings, in part, because there was only one political party in existence. This was the Republican Party (previously known as the Democratic-Republican Party - the ancestor to today's Democratic Party). In 1828 Andrew Jackson was elected president and he boldly referred to himself as a Democrat, and his party as the Democratic Party. This upset a lot of people.
Back then the word democratic conjured up an image that was different from today. Back then democratic, to some people, implied a lack of organization, chaos, even mob rule. Not a direction that a country should be heading. Therefore, to ward off a perceived potential collapse of the national government, a second political party was formed - the above mentioned Whig Party. This marked the end of the Era of Good Feelings and the existence of a 2-party system in this country. The Whig Party was successful enough to elect two presidents (William Henry Harrison in 1840 and Zachary Taylor in 1848).
The Whigs were soon replaced by today's Republican Party (1854).
The first party system pretty much ended when the Federalist party died in the early 1810s. The Federalists died out because they were anti-democratic and because they looked like they were unpatriotic when they opposed the War of 1812 (see Hartford Convention).
The second party system started to evolve in the 1820s after the "Era of Good Feelings." The catalyst for this was Andrew Jackson. He made enough people angry that a loose coalition broke away from the Democratic Party and founded the Whig Party. The Whigs were generally more of a nationalist (as opposed to state-centered) and modernizing (industrial) party.