Odysseus' reasons for wanting to leave Calypso to return to his wife Penelope are only partially expressed by Odysseus' own words. 

In Homer’s The Odyssey, the story’s hero, Odysseus, struggles for 20 years to make it home to Ithaca.

Late in his journey, after losing all of his men, Odysseus finds himself stuck on the island of the immortal sea-nymph Calypso. Calypso takes great interest in Odysseus and wants to keep him for herself.

Meanwhile, back in Ithaca, Penelope’s suitors continue to harass her and plot against Telemachus.

Against this backdrop, Athena addresses Zeus at a gathering of the gods, pleading on behalf of the stranded Odysseus. Zeus then orders Hermes, the messenger, to fly to Calypso and instruct her to set Odysseus free.

As the above post noted, Odysseus himself does not have a lot to say about it beyond confirming that he wishes to return to Penelope even though she is mortal and less “comely” than Calypso.

However, Odysseus' own words are not the only evidence of his desire to return home. It is also worth noting how the poet describes Odysseus’ behavior just prior to learning that Calypso will release him:

Him she found sitting on the shore, and his eyes were never dry of tears, and his sweet life was ebbing away, as he longed mournfully for his return, for the nymph was no longer pleasing in his sight. 

Although these are the poet’s words and not those of Odysseus, they still show how Odysseus is affected by his absence from Ithaca and Penelope. Despite living with the love of a beautiful immortal, he still wishes to return to his home and wife. Although he doesn't explicitly say so to Calypso, he no longer finds her "pleasing in sight." Of course, this also implies that Odysseus has the usual male weakness for beautiful women, because he did find her pleasing earlier in the story. 

You can find the answer to this question in Book V.  Odysseus has been living with Calypso on her island, but now Zeus has ordered Calypso to let Odysseus go.  Odysseus has been very sad about being unable to go home.

When she receives the order, Calypso goes and talks to Odysseus.  She asks him if she is not as beautiful as his wife.  Odysseus says she is no less beautiful than Penelope.  However, he says he just really wants to get home.  He does not give any answer other than that.

I am quite aware that my wife Penelope is nothing like so tall
or so beautiful as yourself. She is only a woman, whereas you
are an immortal. Nevertheless, I want to get home, and can think
of nothing else.