One side of this controversy wanted the Bank because they thought it helped to stabilize the US economy. It prevented, for example, banks from lending money too easily, creating "bubbles" which could then pop and cause "panics." (Sort of like how we had a housing bubble because of easy lending which popped in 2007-8 and helped lead to our current problems.)
Those who did not like the Bank generally thought of it as a tool of the elites. They felt that it was being used by the elites to keep the common people down. This is why President Jackson was so opposed to the Bank.