Prospero, in Act I scene 2, finally tells Miranda the truth about his own identity and past and who she really is. He reveals to her that he was not born on the island and that both of them have a different home, one where Prospero enjoys a position of some power and prestige, as he discloses to Miranda in the following lines:

Thy mother was a piece of virtue, and
She said thou wast my daughter; and thy father
Was Duke of Milan; and thou his only heir
And princess no worse issued.

He goes on to explain that, although he once was the Duke of Milan, he managed to lose his title when he cloistered himself away from the world to study his magic arts and gave the power he had to his brother, Antonio, to rule and manage his estate. Although Prospero did not yield him the title, what Antonio did was to use the powers that he was given to create allies and support for himself so that he was eventually able to oust his brother and exile him from his former kingdom and territory, and seize the title that was his brother's for his own. This is why Prospero ended up on the island with his daughter and she has experienced such a different childhood from the one that she would have enjoyed had her father not been exiled.