This quote is attributed to John Winthrop, an English Puritan immigrant to the Massachusetts Bay Colony in 1630, and governor of that same colony. It has been noted that other politicians (for example, Ronald Reagan) reuse this quote frequently. It is a powerful image, for everyone can see—and judge—a city built on top of a hill.
John Winthrop said these words in the context of a speech entitled "A Model of Christian Charity." For him, the Massachusetts colony had to work, in order to show the glory of God and, especially, the righteousness of Puritan believers. He believed that other Puritans would be looking to the new colony as a model. Therefore, it was up to Winthrop and his fellow colonists to ensure that everything was exactly as it should be. Their lives would be an example that others could follow. On the other hand, if they failed, the results would be disastrous for the Puritan cause.
That is what it means when he says "We shall be a city on a hill, the eyes of all people are upon us." Everyone is watching to see if they succeed or fail. In Winthrop's view, success is the only option.