Another way to look at this from the perspective of your question is that colonization of Africa was relatively temporary, while the Spaniards were in Latin America for centuries.  There is also the factor of disease to consider, which, along with Spanish policies regarding religious conversion made the colonization of Latin America more inherently genocidal.  This was true in both the cultural and physical sense.  Latin America more universally speaks Spanish and practices Catholicism, where that is less true for European colonies in Africa (though still to some extent).

Hard to argue that the effect in Africa hasn't been more protracted and ugly, with more lasting and divisive colonial legacies at play.

I agree with the previous answer when it says that Africa was harmed more than Latin America.  However, if your question says Latin America was harmed more...

The only way I could argue this is to say that African countries (outside of South Africa and what is now Zimbabwe) were not actually colonized by large numbers of Europeans who stayed and made themselves into a "local" elite.  As the first answer says, in Latin America, you tend to have elites running society today who are more ethnically European.  These elites took over back in colonial days and transformed their societies.

By contrast, Africa was never really Europeanized in this way.  The Europeans came and ruled and then left.  So African culture was left intact, relatively speaking and Africans now rule Africans.  I don't know that this has done them any good, but that is how I would argue the case.

Modern imperialism in the form of international corporate/banking control is equally damaging everywhere.  In historic terms, it would be hard to make a case for this question's assumption; Africa is without doubt the most wrecked part of the world today, and that is in large part due to colonialism.  The boundaries of African countries were drawn up by European powers with no regard to ethnic, cultural, religious or linguistic reference.  The very end of colonialism in Africa was done in such haste after WW II that almost no African country had any real ability to run itself or its political, economic, military or social issues.  This was not because of some mythical inability of Africans, but because the European rulers had never really integrated Africans in the processes of governing or in the running of the economic life of the nations.  The Congo is the most glaring example, in some respects, but no more really than Angola or many others.

Colonialism's main negative impact on South America is that the countries of the continent were mostly under Spain during colonialism, and therefore have several major problems inherent from that. One, they have Spanish-based law, meaning the accused must prove their innocence in criminal cases, infinitely more difficult than defending themselves while the State must prove their guilt.  Two, the legacy of military control and rule by an elite minority, usually of lighter-skinned, more ethnically Spanish people.  That goes along with the third, the extreme inequality of wealth, with a small minority of rich and powerful families ruling in a business-consortium manner, with a large mass of poor peasants and a small middle class.  The fourth is more subtle, the tendency of Spanish-based cultures to attempt to redress grievances by revolutionary activities.  This is in part due to the elitist nature of Latin American society and government and resistance to the needs of the people, but in part is a legacy of the Napoleonic Wars.  Spain's military was unable to resist Napoleon's forces, but with British help they waged a very successful guerrilla war.  The problem is that this left Spain (and Latin America) with a tradition of guerrilla warfare when dealing with any major governmental problem which frustrated a major portion of the population.

There is no doubt that colonialism left S. America with a legacy of serious problems, and that modern "globalization" is wrecking the economies of the continent.  But in neither case are those effects really worse on that continent than in Asia, certainly not worse than in Africa.