As various people have already pointed out, economically undeveloped countries tend to have in common a lack of economic development (hope that doesn't sound tautological!). Among the things that make them different are their cultures, their histories, their social values, their geographical traits, their climates, their natural resources, their religious orientations, and a variety of other such factors. All these factors (and many others) can influence whether and/or how much a country develops economically. Levels of education, political systems, and economic systems are also crucial. The two Koreas share (or used to share) much the same culture and much the same ethnic identity, but South Korea is now one of the most economically advanced countries in the world, while North Korea is one of the least advanced by far. Contrasting political and economic systems seem to have accounted for this huge difference. I always find this photograph startling: http://bigthink.com/ideas/21270
One point in common is that there are limited resources for and access to food, education and sanitation. One diverse thing is the government and religious systems that are in place in various developing countries. Another thing they hold in common is the difficulties they face in opposing exploitation of their physical and human resources. Another diverse thing is their languages and cultures.
One thing that developing countries usually have in common is their systematic oppression of women and girls. These countries often have cultural traditions and laws that place women in extremely inferior roles, and subject them to a life of less education, less health care, and less freedom.
I have to say I respectfully disagree with 9's statements; the US has been around for less than 300 years, but is far more developed than many countries that have existed much longer, such as India, Turkey, Thailand, Russia, and Poland, all of which are on the IMF's list of developing and emerging countries.
The United States is hardly the only example of a historically young but highly developed nation; Canada, Australia, and New Zealand are also examples.
This is such a complex question that it would be nearly impossible to produce a short list of deciding factors as to what causes one nation to develop and another not to, but length of existence does not seem to correlate with development well at all.
Here is a link to the IMF report, which makes for intersting reading:
It is kind of obvious, but developing countries lack the advantage that developed countries such as the US and European countries have of being developed for a long period of time. We are fortunate enough to live in places where we have had stability and a good basis for economic growth for a number of years. Developing countries lack this stability, and the instability that often characterises developing countries means that they face many more challenges than developed countries.
I think that another factor is that developing countries rarely have much infrastructure. This creates a chicken-or-the-egg scenario, as only countries with a stable economic base can afford to build roads, bridges, railways, water treament plants, schools, hospitals, etc.; however a country lacking those things is unlikely to produce sufficient exportable goods to be able to pay for them. It's difficult to sell product if you lack transportation. It's hard for people to be productive if they don't have access to education, safe drinking water, and so on.
Some countries have short circuited this problem because they have mineral wealth; for instance, OPEC countries have developed rapidly because their oil reserves are of great value to the rest of the world, and the oil companies (some of which have annual profits that rival the GDPs of third world countries!) are wealthy enough to move in and build the necessary infrastructure for their own use, giving the entire country a jump start on the development process.
The one thing that developing nations all have in common is the fact that they are not (or not yet) developed economically. That is to say that they generally have low income levels and do not have modern, diversified economies.
Within this definition, there is plenty of leeway for diversity. First of all, there are major differences in income levels between "developing countries." There are countries like Mexico which are typically labeled as "developing" but which have income levels much higher than those of other "developing" countries such as Benin. Another source of diversity is political. Some developing countries are relatively democratic (the Philippines, for example) while others, like Myanmar, are highly repressive.
The only real commonality between these countries is that they are not as rich as the "first world." Outside of that, there is plenty of room for diversity.