In my opinion, the thing that is the most likely answer to this is Prospero's clothes.  We see, in Act I, Scene 2 of the play that he has something that he calls his magic garment.  In the stage directions, Shakespeare refers to it as his mantle.  A mantle is another word for something like a cloak.

Later, in Act V, Scene 1, we see Prospero wearing his magic robes.

I think that these are more of a symbol of his magic than his book is.  However, when he gives up doing magic he does say that he will drown his book.  So you might say that is the symbol.