The prison outside Mr. Jaggers's office is Newgate Prison, an infamous site in London located on the corner of Newgate Street and Old Bailey, just inside the city of London. In the setting of Great Expectations Newgate housed many prisoners, especially those who awaited trial for serious crimes or their hanging outside the prison walls.
It is this ominous place that Pip tours with Mr. Wemmick, who walks among the inmates as though he were tending a garden. Afterwards, Pip reflects,
I consumed the whole time in thinking how strange it was that I should be encompassed by all this taint of prison and crime; that, in my childhood out on our lonely marshes on a winter evening I should have first encountered it; that, it should have reappeared on two occasions, starting out like a stain that was faded but not gone; that, it should in this new way pervade my fortune and advancement.
Pip's guilt returns to him with this visit to Newgate. In fact, Magwitch, Miss Havisham, Mr. Jaggers, Pip, and Estella are all connected by this "taint of prison and crime" these chains of guilt and corruption in their roles as corruptor and the corrupted, betrayers and the betrayed. Truly, the guilt of one of the characters in Dickens's novel, tinges another with Newgate as symbolic of this corruption and betrayal.