Andrew Jackson's legacy has become more and more controversial in recent years. In fact, there has been a movement to remove him from the $20 bill.
There are at least two major controversies regarding Jackson. His stance on slavery and the impact that he had on Native Americans would be the major two.
Jackson, like many southerners of his time, owned slaves. He was known to be a particularly cruel slave owner and vocal proponent of slavery. Jackson also disapproved of efforts to outlaw slavery in the new Western territories.
Jackson's philosophy of "Indian removal" was ultimately responsible for the displacement of tens of thousands of Native Americans. Jackson wanted to expand parts of the Southeastern United States for white settlement and sought to remove Native Americans from the land. Many tribes negotiated an unfair deal to move and some tribes refused to go. One of these refusals ended in the "Trail of Tears," the forced removal thousands of Cherokee off of their lands, resulting in the deaths of at least 4,000.