The real reason the taxes angered the colonists is because the colonists did not want to pay any taxes that they did not vote on themselves. This is related to the "no taxation without representation" argument.
The colonists made other arguments to try to get the taxes repealed. They argued, for example, that the British government had the right to tax trade but not things within the colonies. They argued that the British had the right to tax as a way of regulating trade, but not simply as a way to raise money.
But really what was behind it all was that the colonists did not want to have to pay any taxes that were not voted on by their own assemblies. They wanted to have control over what taxes they would have to pay. Because they did not, they were unhappy.