The differences of opinion sprang mainly from people's feelings about the exact way in which government was dangerous. Both sides felt that government was dangerous, but they felt that different parts of government were the danger.
To the antifederalists, the danger was far-away, big government. They feared that the US government would be distant and oppressive like the British government. To the federalists, the danger was from state governments that were dominated by one "faction." They feared that such governments would be run by the common people in ways that would be democratic but which would destroy the economy.
Although the federalists were anti-democratic, it seems today as if they did the right thing because the system that they created has worked well.