The nature of Caliban and Prospero's relationship looks very different according to whose perspective one adopts. From Prospero's point of view, he came to a wild island, which was until recently ruled by the foul witch, Sycorax. Prospero civilized the island and treated its inhabitants kindly, even Caliban who was
A freckled whelp hag-born—not honour'd with
A human shape.
Caliban repaid Prospero's kindness by attempting to rape Miranda, Prospero's daughter, since which time Prospero has kept Caliban as his slave.
Caliban would agree that he has been enslaved by Prospero, but apart from this, their relationship looks very different from his perspective:
This island's mine, by Sycorax my mother,
Which thou takest from me. When thou camest first,
Thou strokedst me and madest much of me, wouldst give me
Water with berries in't, and teach me how
To name the bigger light, and how the less,
That burn by day and night: and then I loved thee
And show'd thee all the qualities o' the isle ...
According to this version of events, Caliban is the rightful ruler of the island, while Prospero is a usurper who gained his trust by trickery. Then, when he had learned all that Caliban had to teach him about the island, he betrayed and enslaved his former ally.
The latter narrative has gained favor with critics following the rise of Postcolonial theory, since Prospero has all the characteristics of an imperialist, colonizing the island and justifying his depredations by saying that he is more civilized.
None of this makes any difference to the fact that, in practical terms, Caliban is Prospero's slave, and their relationship is one of mutual hatred and distrust. The only question is whether Caliban deserves his slavery or not. Shakespeare's contemporaries, seeing Caliban as a brutal monster, might well have said he did, while a modern audience may be more inclined to the belief that no one deserves slavery.