There are a few different factors that went into this. Perhaps the two most important are the following:
The type of agriculture done in the various colonies. The plantation economy of the South needed unfree labor to succeed. The work was hard and free people were generally not interested in working that hard for someone else when there was cheap land to be had elsewhere where they could work for themselves. The other colonies generally had small farms that had less need for slaves.
The climate of the different colonies. This plays into the previous factor. The climate and land of the North was not as conducive to large-scale agriculture. Because of this, the New England colonies in particular were dominated by small family farms where slaves were not needed.