By 1860, one could see that the United States was in uncharted territory in terms of its ability to compromise over slavery. Henry Clay, a leader in creating both the Compromise of 1820 and the Compromise of 1850, was dead. There seemed to be no moderating force in Congress who would take his place. The rhetoric around slavery, heated at times, was reaching a new level during the 1850s. Harriet Beecher Stowe's Uncle Tom's Cabin was one of the most popular books of the era; it depicted slavery unfavorably. Playhouses who acted out the book had large passionate crowds-—in one instance, an audience member shot one of the dogs who was chasing the actress playing Eliza.
Many revolts and political disputes also became more influential. John Brown tried to incite a slave revolt in 1859; he was later hung for this. To slaveholders, Brown was a villain. To abolitionists, he was a martyr. Violence was starting to creep into political life as well. Preston Sumner, a Northern congressman, was beaten by Charles Brooks, a Southern Congressman, when Sumner attacked slavery. Slaveowners around the South then sent Brooks new canes with which to attack more abolitionists. When Stephen Douglas stated that the territories of Kansas and Nebraska could vote on the decision to allow slavery, it led to what would be known as Bleeding Kansas as slaveholders and abolitionists moved into the territory in order to influence the vote. In the process they fought each other, burning towns and killing men, women, and children.
By 1860 there was too much money in slavery to allow for a peaceful compromise such as gradual abolition. The value of the slaves in the South was worth more than the banks and railroads of the entire nation. World demand for Southern cotton was quite high. There was not enough money in the federal treasury for the government to buy all the slaves and set them free. Even if there was nationwide support for abolition, many Southerners were worried about their fate if the slaves were set free. Many feared a slave revolt—John Brown's attempted insurrection at Harper's Ferry only made this feeling stronger.
By 1860 heated rhetoric in both the private and public spheres made the issue of slavery one that made compromise nearly impossible. The amount of wealth tied up in owning people also made plantation owners less likely to accept a compromise. Fear of upending the racial status quo made most of the South reluctant to change the status of enslaved Africans. By 1860, it was apparent that the nation was reaching a turning point. The election of Lincoln in 1860 without his winning a single Southern state made compromise impossible.
Most modern scholarship suggests that Southerners did not chafe at the use of federal power, but that it would no longer be used to further their agenda. They had no problem in using federal power to force slavery on the territories in violation of popular sovereignty (as they did with the so-called Lecompton Constitution in Kansas), in subjecting Congress to "gag rules" concerning the debate over slavery, and indeed enforcing fugitive slave legislation in violation of local personal liberty laws in the North. The idea that "states rights" were actually an issue is more related to Lost Cause ideology than modern historical work. For a good synthesis of the recent historiography of the political crises of mid-century, see William Freehling's Road to Disunion or Allan Tulloch's History of the Civil War Era.
The sectional differences between the north and south were kept at bay by the compromises of 1820 and 1850, and were transitory at best. While both compromises were aimed at maintaining the balance of power in the Congress, the underlying issue of slavery intensified throughout the 1850's polarizing the north and south. The causes responsible for the disintegration of compromise were:
1. the continued rise of the Abolitionist movement
2. the anger from both sides regarding the Fugitive Slave Act; the north because it was passed, the south because many in the north ignored it
3. the publication of Uncle Tom's Cabin
4. the deadly result of the Kansas-Nebraska Act 1854
5. the new 'Republican' party whose platform was to prohibit slavery into the west
6. the decision by the Supreme Court in Dred Scott v. Sanford 1857 which ruled that slaves were property, nullifying the Missouri Compromise and any other ruling regarding slavery
7. John Brown's raid on the Harper's Ferry arsenal in 1859
The balance of political power in Washington had been dominated by the "Free State, Slave State" balancing act that started right from the founding of the country (Look up the acceptance of Vermont and Kentucky as state numbers 14 and 15, one slave, one free, one North, one South.) As the country expanded, it was necessary to compromise on the issue, as in 1820 and 1850, but by 1860 it became increasingly difficult to maintain the balancing act.
As others have posted, it wasn't just about free and slave states, it was about the differing cultures that developed in the North and South and the different ways by which they wished to be governed.
Of the posts above, Post 3 is the best. It is right to look at the importance of social and economic differences between the two sections of the country. We should also note that the reason that things got worse by 1860 was because there had been fairly constant friction over the past 40 years with regard to slavery. This led to a complete breakdown in trust between the two sections by 1860.
Basically, compromise failed because Southerners perceived, with the rise of free soil ideology and the Republican Party, that they would no longer be able to use the federal government to promote the spread of slavery in the West. It was ultimately an unwillingness to compromise on the part of the South, which became manifest after 1850, that caused the split, not anything inherent about a two party system. The Republican Party itself was the product of a series of compromises, between abolitionists and Free-Soilers in particular.
The cultural and sociological divide between the industrial North and the agraian South led to the divisiveness of ideologies in the 1860s. Certainly, there was some economic envy of the free labor and wealth that was possessed by plantation owners who lived virtual aristocratic lives. On the part of Southerners, there was much resentment of the controls that the Federal government, located in the North, wished to place upon them.
The two-party system does not seem to invite compromise. At that point in our political history we had more parties involved, and more compromise possible or at least attempted. Two ideologically opposed parties cannot compromise, apparently. That is our problem now.