Richard Rodriguez opens Hunger of Memory with a reference to Shakespeare’s The Tempest: “I have taken Caliban’s advice. I have stolen their books. I will have some run of this isle.” He alludes to the recommendation that the creature gives Stephano and Trinculo about how to kill Prospero and take over the magical island.

After being exiled to this island, Prospero used his books on sorcery to subdue nature on the island and to control its sole inhabitant, the half-monster Caliban. When a second shipwreck strands more people on the island, Caliban seizes the opportunity to shake off Prospero’s shackles by advising castaways Stephano and Trinculo about how to overthrow him.

…Remember
First to possess his books, for without them
He’s but a sot, as I am...

Caliban knows that the secret to Prospero’s power is his books. Similarly, Rodriguez realizes that the way to gain power in American society is through books. He began life as a disadvantaged minority, but he became a celebrated writer. By creating books—which are the source of power—he ascends to a commanding social position.

Once upon a time, I was a “socially disadvantaged” child. An enchantedly happy child. Mine was a childhood of intense family closeness. And extreme public alienation. Thirty years later I write this book as a middle-class American man. Assimilated. Dark-skinned. To be seen at a Belgravia dinner party. Or in New York. Exotic in a tuxedo.

Rodriguez’s odyssey hearkens back to the theme of colonial domination that many 20th century scholars explore in their readings of The Tempest. Prospero is seen as the invading European who takes over and enslaves the native Caliban. His books represent the culture of the interloper that eradicates and invalidates the culture of the natives. As a Mexican-American man who becomes a writer, Rodriguez reverses the traditional power structure by seizing the source of the dominant culture’s power.

Richard Rodriguez’s allusion to the character Caliban refers to his original characterization in William Shakespeare’s The Tempest, but it also involves Caliban's later appropriation by postcolonialist, nonwhite writers, beginning with Aimé Césaire of Martinique. Rodriguez’s reference to “books” encapsulates the dilemma of the colonized, who must seek their liberation using the same things that had oppressed them.

In The Tempest, Caliban—whose mother, Sycorax, was a powerful sorceress—believes himself to be the rightful ruler of the island that Prospero has taken over by even more powerful magic. Prospero has enslaved Caliban and constantly denigrates him, calling him a monster and accusing him of sexually harassing his daughter. Caliban, finding the shipwrecked sailors, enlists their help in gaining access to Prospero’s books, which he sees as a crucial source of the magician’s power. The books symbolize both the knowledge and materials that the colonizer brings to his colonies but withholds from the colonized people.

In Césaire's revision of Shakespeare's play, which is titled A Tempest, Césaire aimed to rescue Caliban from the reputation of monstrous savagery that Shakespeare had imparted. This 1969 play marked a turning point in focusing on race and colonialism as elements of Renaissance drama, especially Shakespeare’s works. Césaire includes language more generally as an element of the “books,” with Caliban speaking in the African language Swahili and refusing to speak English. This aspect of identity as related to bilingualism is also a feature of Rodriguez’s allusion to Caliban.

Richard Rodriguez's 1982 book The Hunger of Memory reflects on his educational journey from a Spanish-speaking son of Mexican immigrants to a PhD candidate who ultimately rejected a career in academia.  Rodriguez's allusion to Caliban comes from Shakespeare's 1610 play The Tempest. Caliban, enslaved by Prospero, wants to steal Prospero's books because he believes them to be full of magic, hence power—and a way to free himself from enslavement.

Rodriguez alludes to the magical power of books, generally, as agents of transformation in his life.  By "their" books in your quotation, Rodriguez means "Anglo" culture. He believed, at this point in his intellectual development, that it offered a power that he would not find in his own family's traditions, including language and culture. In retrospect, Rodriguez observes that in immersing himself in the English language he has sacrificed his Mexican American culture and Spanish language.  He is never truly bilingual. In choosing to steal the "magic" contained in these books as a ticket to a form of freedom, Rodriguez loses a major part of himself.

This is a literary allusion (reference) to the character Caliban in Shakespeare's play, "The Tempest." Caliban is a monster who has been enslaved by Prospero the magician who landed on Caliban's island when he and his daughter were put out to sea to die. Caliban resents the power Prospero holds over him, so when other men are shipwrecked on the island, Caliban tries to convince them to help him kill Prospero so he can steal his books and magic.

As a major theme of "Hunger" is education and the strong desire for it, the allusion the author uses makes a great deal of sense.

Check the links below for more information. Good luck!