The two dates given in the other answer are the dates on which Delaware and Pennsylvania ratified the U.S. Constitution, which is the date most often used for when a state joined. For example, the 50 state quarters use the ratification date.
Delaware's position as first to ratify primarily reflects the fact that Delaware was strongly in favor of having a strong central government coupled with equal representation in the US Senate for all states. Since that is the scheme that won out in the Constitutional Convention, Delaware was eager to ratify.
Pennsylvania was basically the governmental heart of the country at that point, and is where the Declaration of Independence and the Constitution were written. That Pennsylvania supported the Constitution is clear from the fact that the reporter for the ratifying convention only bothered to record what the proponents said.
1. Delaware - Friday December 7, 1787.
2. Pennsylvania - Wednesday December 12, 1787. However the first thirteen states are considered to be members of the United States from, Thursday July 4, 1776 itself.
In that case, Vermont - Friday March 4, 1791 and Kentucky - June 1, 1792 would be the first two to join the Unisted States.
Vermont: "green mountain" refers to the long ridge of mountains running north-south through the state.
Kentucky: named after the Kentucky river which runs through the state. "Kentucky" comes from the Wyandot (Iroquoian) word kentahteh "tomorrow, the coming day", a word that originally applied to the Iroquoian possessions on the Ohio but later to the lands to the south of the Ohio. Those territories represented the land where they (the Wyandots) would live "tomorrow, in the future." A good translation of the word as it came to apply to the country of Kentucky would then be "The Land of Tomorrow".