In Civil Disobedience, Thoreau is arguing that people have a duty to refuse to participate in evil.  He says that people do not have the duty to actually go out and fix evils that occur in the world.  However, they do have the duty to abstain from actually helping to make the evil happen.

Thoreau says that there are very few people in a society who actually perform this duty.  That is where the passage that you are asking about comes in.  He says that the vast majority of people do not put any thought at all into serving the state.  They do what the government tells them to do without ever thinking about whether they are doing right or doing wrong.  These are the people who will go off and fight for the government as it participates in a war to extend slave territory (Thoreau was, famously, opposed to the Mexican-American War because he felt that it was being fought for the benefit of slaveholders). 

Thoreau argues that there are only a few people who actually think.  These people are the true heroes because they will think about what the government is doing and they will resist it when it is doing evil.  They will do their duty and that will make them true “men.”  As Thoreau says,

A very few, as heroes, patriots, martyrs, reformers in the great sense, and men, serve the state with their consciences also, and so necessarily resist it for the most part…

Thus, what Thoreau is saying in this passage is that most people do not think enough about what is right and wrong and just blindly follow what the government does.  He wants fewer people to do that and more to engage in civil disobedience.