This is a direct conflict with our modern ideas of tolerance, but not with the ideas held by the majority of the colonists this question refers to.
Most of the colonists who came to the colonies for "religious tolerance" were not really interested in tolerance for all. Instead, they were interested in tolerance for (or, more accurately, dominance for) their sect. For example, the Puritans who left England were fleeing religious and political persecution. But they were not seeking a place where people of all religions could worship freely. Rather, they were looking for a place where they could run the community according to Puritan rules.
This is a direct conflict with modern ideas of tolerance. But such ideas were not widely accepted in colonial times. Therefore, it was not in conflict with the idea of tolerance held by most people at that time.