The Ghost first uses the term in Act I, Scene 5, that he is "forbid to tell the secrets of my prison-house." Thus, the motif of the prison-house world becomes central to Hamlet's dilemma. The Ghost is trapped in the prison of purgatory as he was slain before confessing his sins. Hamlet is in a prison where everyone watches him: Claudius watches Ophelia and Hamlet, Polonious watches Hamlet as well as watching his daughter, Ophelia, and Hamlet, and Polonious also watches Gertrude and Hamlet, which leads to his death. Rosencrantz and Guildenstern are paid to watch Hamlet. Denmark becomes Hamlet's prison.
However, if you look at the prison metaphor a bit more closely, a prison must have a warden. Who keeps Hamlet confined to his prison-house world? I would suggest Hamlet, in his inability to act, becomes his own warden as well as a prisoner. Interesting thought to explore!
A good question. The prison that Hamlet speaks of is a good example of how the inner and the outer align in this play. Though he is its prince, he feels trapped in Denmark, due to the death of his father and the way the unpleasant reality about his mother and uncle are shoved in his face. It seems like he can't escape their ugly reality. At the same time, the prison is evoked through language as his life and his duty. He would escape both—he talks of suicide—but feels that he cannot.