The middle colonies—Pennsylvania, New York, New Jersey, and Delaware—were diverse for several reasons. First, they were initially settled by groups of different nationalities. For example, the English settled Pennsylvania, while New York, New Jersey, and Delaware were initially settled by the Dutch. Each group brought with it its distinct language and culture. The strong Dutch presence is a main difference that marks these colonies from the overwhelmingly British settlements to the north and the south.
Beyond that, these colonies opened themselves, whether willingly or reluctantly, to groups of different nationalities and ethnicities. For example, William Penn, a Quaker, started Pennsylvania as a colony based on religious freedom. Sensitive to the persecution that the Quakers had suffered in England, he wanted to open the new territory to other persecuted groups. He invited in persecuted Anabaptists from German speaking states and Switzerland, bringing in a culture that became known as the Pennsylvania Dutch. Penn also promoted peaceful relationships with the native groups in the area.
In addition, more than merely the Dutch colonized Dutch holdings. Swedes and Finns, for example, also settled in New Jersey. Further, while the Dutch at first tried, unlike William Penn, to keep all but people of the Dutch Reformed church from New York (at the time called New Amsterdam), people of other religions came and stayed. For example, a group of Jewish refugees arrived and settled permanently in the colony in 1654.
It should also be noted that Native American groups were a strong part of the cultural mix in all these colonies, especially in the early years of colonization. Although, as Charles Mann argues in his book 1491, many natives had already been wiped out by European diseases brought by traders and missionaries by the time colonies were established, many natives still survived and interacted robustly with the Europeans.