Actually, some of the Pilgrims recognized the merits of the religious tolerance that they experienced in Holland.  While they valued their own Christian, Calvinist beliefs, they did not insist that all others in Plymouth Colony follow them.  I found an excellent resource on this topic, which I am attaching the link to below.  Jeremy Bangs, a historian on the Pilgrims, notes that they had a "conscious construction of a society with separation of church and state.” He even referred to a "1645 proposal by the Plymouth Colony leaders" in regards to other religions and other types of Christianity.  In it, they proposed "that Jews, Catholics, Unitarians and many other sects be accepted in the Plymouth Colony."

Sometimes the Pilgrims are confused with Puritan society in America, which was rigid in regards to religion.  Often citizens of Puritan strongholds were required to attend church. 

Many of the passengers of the Mayflower (but not all) are considered pilgrims because they left their home mainly for religious reasons.  Those passengers who did seek religious freedom were separatists and disagreed with the Church of England and the Catholic Church in many ways.  They believed that both were corrupt.  They believed in a different approach to Christianity, which included the "personal study of the Scriptures."  

Deciding that they were unable to practice their religion as they chose, a group of these separatists moved first to Amsterdam and then to Leiden in Holland.  In Leiden, many of the pilgrims worked hard as weavers.  A group of French Huguenots had also settled in Leiden to seek religious freedom, and some of the pilgrims attended church with them.  

In Holland, the work was difficult.  While they were able to practice their religion with more freedom, the separatists still felt concerned.  Some of their children began to conform to Dutch culture.  Others left to become soldiers.  They felt that their traditions as Englishmen and Englishwomen were fading.  Also, war between the Dutch and the Spanish loomed.  They feared a repeat of the Spanish Inquisition in Holland, where surely they would be a target (for defying the Roman Catholic Church).  Because of these and other reasons, some decided to sail to the New World.

That question can be answered in several ways. Often people just say that the Pilgrims came to North America seeking religious freedom. True, they wanted to worship as they pleased but the answer is not as simple as that. 

The Pilgrims had previously gained religious freedom from the Church of England by leaving in 1607 and settling in the Netherlands. However, even though they lived under fairly lenient conditions in the Netherlands, by 1620, the economic hardships and fear of losing their English language and culture, caused them to decide to try a voyage to the New World. They set off for an area near the Hudson River believed to be where there was already a Virginia colony established. They were unable to reach that destination due to storms and instead settled on a calm looking harbor in the Cape Cod area. 

Therefore, although the Pilgrims wanted religious freedom for themselves, they did not leave England for North American directly for that purpose. In between, they settled in the Netherlands for over a decade where they did have religious freedom but poor living conditions. It was the hope for a better economy and living conditions that caused them to settle in North America where they could continue the religious freedom they had in the Netherlands.

The pilgrims went to North America by way of the Netherlands in order to escape persecution from the government and Church of England.  Their desire to purify (hence the name Puritan) the C of E of their resemblance to Roman Catholicism made church leaders feel a need to silence their movement.