As far as Europeans are concerned, Leif Erickson, a Viking, is often credited with being the first to discover America. He and some of his his extended family arrived in Newfoundland, now part of Canada, around the year 1,000. They built houses and settled for a few years but did not stay, despite liking the land. It is believed that they had too many troubles with the local Native Americans.
There are several factors, however, that set Columbus's voyage apart from Erickson's and lead people commonly to credit Columbus with the discovery. First, he arrived after the invention of the printing press in Europe, so news of his voyage could be widely publicized. In contrast, there is no evidence that people in Europe in 1000 were aware of what Leif Erickson had done.
Further, by the end of the fifteenth century, the Europeans had a much more decisive technological advantage over the Native Americans. For example, they had guns, which, if crude by our standards, allowed them to dominate the local populations.
By the time Columbus arrived in the New World, Europeans had the means to easily learn about the voyage and to recognize its potential, so from that point on, they ventured to the continent, eventually establishing permanent settlements.