You can find the answer to this question in Book V. At that point in the story, Athena has gone to Zeus and asked him to make Calypso let Odysseus go. Zeus agrees with Athena and orders Calypso to release Odysseus. Calypso is angry, but has to obey.
But first, she asks Odysseus if he wouldn't rather stay with her and become immortal. Odysseus tells her that he realizes it would be wonderful but he just can't do it. The only reason he gives is that he wants to go home and can't think of anything else but going home.