In "Bartleby", Melville practically hits readers over the head with the references to "walls." The story takes place on Wall Street. The office is surrounded by walls on all sides. The office is separated by partition walls. In the prison at the end, the narrator finds Bartelby sitting with "his face towards a high wall."
Walls are a symbol of separation, and in this story represent Bartleby's separation from both reality and other people. He is suffering from psychological walls that keep him isolated, uncomfortable with human contact. He has removed himself to such a degree that the only thing keeping him tethered to Earth is his work, which he does with maniac precision and efficiency. Until he loses his taste for that as well, and walls himself off from all human pursuits. He would "prefer not to" engage in them, and instead sinks into his mind.
In a larger sense, Bartleby's story and the walls that Melville mention are symbolic of the time period and the new industrial environment of the cities. Walls were being erected everywhere, and business and industry were the main concern. In this world of walls, how are people to get to know one another, to take comfort in personal relationships? This is emphasized through the narrator's reflections on his employees. He knows them only through behaviorally inspired nicknames, and not a real individuals.