One of the major points that Thoreau is trying to make in this essay is that government (the State) is not a good thing. It is best to have a government that does not interfere in the lives of the people any more than is absolutely necessary. Thoreau goes so far as to say that the best government would not govern at all. This is the point he is trying to make in the passage that you mention.
After being let out of jail, Thoreau wants to get away from the government. He wants to go out and live his life without government interference. He emphasizes this by telling us what he did after being released. Basically, he went about his previous errands as if the state did not exist. He went and got the shoes that were being mended. Then he went out with the berry pickers and went off to a hill two miles away where "the State was nowhere to be seen."
By telling us this, Thoreau is showing that the state is not, or should not be, relevant. He is showing that he can get away from the state completely and that this is what he wants to do in his life.
Henry David Thoreau was on his way to see the shoemaker when he was arrested for not paying the Massachusetts poll tax. He turned down the chance to pay it again. The sheriff locked him in a cell with another prisoner in the Middlesex County Jail, which stood near the center of Thoreau’s hometown of Concord. This experience was a new and different one for Thoreau, and he made the most of it. It gave him the opportunity to think deeply about what it really means to take a personal stand against the government, and how this is something that each individual must decide for himself. He was passionate about his choice not to give the state any money for this general fund. But someone else paid his bill overnight – probably his Aunt Maria, who lived about a block away from the jail – and Thoreau was released the next day. Life goes on. He could pick up his repaired shoes and continue along with his daily routine, as he had planned it before being interrupted by the sheriff. “The only obligation which I have a right to assume is to do at any time what I think right,” he says in the fourth paragraph of his essay. He did what he needed to do, and that was that. He was not going to let any aspect of government impact his life any more than it already had.