The quote you are referring to comes from page four of Walden and reads, in total, like this: "Any nobleness begins at once to refine a man's features, any meanness or sensuality to imbrute them."
In order to understand what Thoreau is trying to say we need to have a good idea of the terminology being used in the statement. The tricky word here is "imbrute." This word means "to make a brute out of something...to make something brutal." Keeping that in mind, it would appear that Thoreau is trying to say that when a person acts in kind, intelligent, thoughtful ways his physical features are refined. To refine, in this sense, would mean "to make more detailed or better." So, behave well as a noble person would and it will reflect in your face.
Contrast that with what happens if you are a mean person who is governed by his/her passions. According to Thoreau, these types of behaviors will also take a toll on your features by making you look more like a brute. Brute, here, would be used to mean "like an animal." So, behave badly and it will reflect in your face as well by making you look like a cruel sub-human.