First of all, we must recognize that many people would say that it is right for governors to have the right to pardon convicted criminals. It is seen as a traditional power and one that allows for clemency in deserving cases.
However, it is clearly possible to argue that governors should not have this right. The main reason for this is that it is (arguably) undemocratic and unfair. In a democracy, we do not give tremendous powers to any one person. We would never give a governor the right to unilaterally pronounce that someone is guilty of a crime. The Constitution takes that power away from Congress. If governors should not be able to convict people of crimes, why should they be able to exonerate people of crimes? From this point of view, it makes no sense. It is too likely to lead to the pardons of people who are politically connected or who have sympathetic stories even if those people are no more deserving than others.
In these ways, letting governors pardon people is both unfair and incompatible with democracy.