The greatest hardship the earliest colonists faced was fear. They were outnumbered in a new land. Upon first arrival, the first colonists had to find sources of food and water. In the case of the Pilgrims, they found Native American caches of food. In the case of Jamestown, they found water, but they located their colony in a tidal swamp that was a breeding ground for mosquitoes. As a result, many of the original colonists of Jamestown died of mosquito-borne diseases, while many of the earliest settlers of Plymouth died from malnutrition and exposure to the elements.
There was also the hardship of ensuring proper governance. Jamestown looked to John Smith in order to build a stockade and plant valuable tobacco. The Pilgrims created the Mayflower Compact, the first attempt at self-governance in the Americas. Without proper governance, there would be no way to ensure law and order, and no colony would survive the stresses of being in a new land without some sort of security.
There was also worry about the Native Americans in the area. The Native Americans initially outnumbered all of the colonists. There were fears of these people. Initially the colonists got along and traded with the Native Americans; it was not until the colonies sought to expand and take more land that conflict occurred.