It is a fairly persuasive point being addressed in the question. I do think that one of the disadvantages would be that there is a change in the Constitutional structure of the election of the President. This is rooted in the idea that the Constitution works best when it is not changed and elminating the Electoral College removes a portion of it that has been in place for some time. I think that another potential disadvantage of its abolishment would be that smaller states end up losing a voice in the process. The need for candidates to ensure that smaller states are represented in the quest for 270 electoral votes is preserved with the Electoral College as opposed to strictly popular vote. The final reason would be more of a hypothetical situation. The construction of the Electoral College was designed to ensure that there is some buffer between the people and the candidate. If an "evil genius" was able to convince the people to vote for their candidacy, it would be up to the individual who comprise the Electoral College to prevent this from happening. I think that these are reasons enough to explain why it should not be abolished.
The main disadvantage here would be that the states would come to matter less in our political system. This would be particularly true of the little states that do not have all that many voters in them.
In the Electoral College, the small states are overrepresnted. This is because they have the same number of Senators as the large states (and the number of votes a state gets is equal to the number of members they have in Congress). Because of this, there is some advantage to be gained by campaigning in small states -- you can get more electoral votes there than those states are really "worth" in terms of numbers of people.
The idea is that if there were no Electoral College, candidates would ignore these small states and spend all their time in the big states where all the people are.
As the link that I have included says
We would probably see elections dominated by the most populous regions of the country or by several large metropolitan areas.