Nullification was a problem for President Jackson largely because he had to keep his political party (and the nation) together during the time that South Carolina was trying to nullify the tariff and was even talking about seceding. Jackson was a Democrat from the South. Many Democrats in those days were Southerners as well. This meant that Jackson's party loyalties and his regional loyalties would have been pulling him towards the side of nullification.
Nullification would have been a problem for any president who had to deal with it. For Jackson, it was even more of a problem because his political party and the people who formed much of his political base (Southerners) were generally sympathetic to the idea of nullification even as Jackson opposed it.