The deliberate withholding or release of information is a powerful storytelling tool; it can heighten or ease tension, or alter the reader's perspective on events. It is fundamental to the "twist" ending. In the case of Bartleby, the information about the dead letter office serves to gives some context and explanation for Bartleby's odd behavior.
Bartleby was hired by the narrator, a lawyer, and initially did a good job, but after rejecting a request he seems to enter a downward spiral. First he stops working, then seems to be living in the office, and later refuses to eat, and eventually dies in prison. The narrator is perplexed, but curious and sympathetic to Bartleby, and is surprised at this sympathy considering that Bartleby had given him every reason to dismiss his actions as insane. Throughout the story, there is no explanation for Bartleby's behavior other than his repeated, placid statement that he "would prefer not to" do whatever is being asked of him.
The revelation that Bartleby worked in a dead letter office, regardless of whether it is true or a rumor, serves to explain his behavior. By placing this information at the end of the story, it allows the reader to reflect on their observations about Bartleby. Had the information been provided up front, the story would have amounted to little more than an extended obituary and epitaph. Delaying the information also allows us to sympathize with the narrator's perspective, since we are equally confused about Bartleby's behavior.
Bartleby's melancholy attitude, inspired by some real-life tragedy blown to extreme proportions, is an element of Romantic literature that this story has in common with some of Poe's works from the same period.