It's hard to remember all the things that gave me new insights into these events since it's been so long since I was first learning about it.
Perhaps the one that sticks best in my mind is the description I read about a riot that happened in Boston at an abolitionist meeting. This was a riot by people who wanted to kill the abolitionists, including William Lloyd Garrison. This really brought home to me how much many Northerners hated abolitionists.
A second thing that gave me insight into this period is a set of letters from an ancestor of mine who fought for the North in the Civil War. His letters make it clear that he was not fighting to help African Americans or to end slavery.
Finally, I guess I'd mention Lincoln's famous letter to Horace Greeley in which he says the he would save the Union, if he could, without freeing any slaves. It made clear that what he cared about was the Union, not the slaves.
Between them, these sources all made me realize how little many Northerners cared about the slaves themselves.