Thoreau is expressing his belief in the importance of the individual. He had some big problems with the concept of majority rule. According to him, majority rule has nothing to do with right vs. wrong because 99% can be wrong. It also has nothing to do with fairness because it is not truly a fair society where 49% of the population is unhappy. Thoreau said that the only reason majority rule is in place is because of physical strength...which is no reason to make major decisions. Literally, if it came down to it, the majority could beat up the minority. The "majority of one" you refer to is Thoreau's way of saying it is the thoughts and beliefs of you that truly matter. He is expressing the idea of Individualism defeating conformity.