The ethical perspective you are probably adopting when you say that you cannot see why eating human flesh is wrong is that of Utilitarianism. Utilitarian philosophy sees harm to others as the primary determining factor in whether an action is moral or not. In this system, it is axiomatic that if you are alone on a desert island, you cannot do anything morally wrong, since you cannot harm anyone.
It is important to realize that, though Utilitarianism is often taken for granted by modern liberals, most people throughout history have not defined morality in this way. For instance, religious people in many different traditions would invoke the idea of divine command theory. They would say that God created human beings and did not do so in order for them to eat each other. Cannibalism is wrong because it is repugnant to God.
There are other arguments which do not require belief in God, however. Deontological ethics holds that actions can be morally wrong regardless of their consequences. Many deontologists, and philosophers in related schools, are concerned with the purity of the subject. They would say that a dead person whose flesh is eaten cannot be harmed, but it does you harm to eat it. A consequentialist argument could also be made along these lines. If you eat human flesh, you will lose respect for humanity, and this will influence the way in which you treat people. A deontologist, however, would find this assertion unnecessary. They would say that the inherent disrespect is enough.