One important point to note here is that, at the beginning of 1789, it was not the United States that owed this money. Instead, it was the individual states. Under the Articles of Confederation, they were the ones who could borrow money and they did so to help finance the Revolutionary War. The first thing the US did to lower the debt was for the national government to assume all of the state debts. That way, the stronger national government could be in charge of getting them paid off. This was very controversial.
Once the state debts were assumed, Alexander Hamilton proposed three main ways to reduce the debt. Two were relatively direct while one was indirect. First, Hamilton wanted there to be a great deal of trade with foreign countries. This trade would be subject to tariffs and the tariffs would give the government income to use to pay the debt. Second, he wanted an excise tax on liquor to get more money to pay the debt. (Of course, these revenues could be used for other purposes as well). Finally, and less directly, he wanted a national bank. He wanted this as a way to make the country generally more prosperous so that tax revenues would increase.