I do not believe that the premise of this question is accurate. The Democrats and the Whigs did not really agree on Indian Removal. Instead, the Democrats were in favor of Indian Removal while the Whigs opposed it.
Since I am saying that the question is not right, let me provide evidence for my contention. First, in this link, we see that
The Whig coalition included … religious groups alienated by Jackson’s Indian removal policies…
As another piece of evidence, this link says the exact same thing. Finally, a textbook that I often use, (Carnes and Garraty, The American Nation, 14th Edition) has a table on p. 262 that outlines the differences between the Democrats and the Whigs. It says that the Democrats supported Indian Removal while the Whigs opposed it.
So why did each side feel as it did? The Democrats were for Indian Removal because they were the party of the common man. They represented many people who lived on the frontiers and who felt threatened by Indians or who simply wanted the Indians’ lands. Therefore, the Democrats had a political motive to support Indian Removal. Meanwhile, as the quote above suggests, some Whigs were more moralistic and felt that it was wrong to force the Indians to move. In addition, the Whigs were strongly opposed to anything that President Jackson did. They felt that he ruled autocratically and would have been especially unhappy with the way in which he defied the Supreme Court over Indian Removal. Thus, the Whigs’ opposition to Indian Removal was somewhat moral and somewhat pragmatic/political.