The background information needed to help you understand a quote is also called the context of the quote. In order to determine this, you should go back to the text itself and re-read what is going on when this was spoken.
Atticus here is discussing Tom Robinson's case (sparked by the Mrs. Dubose situation) with Scout. Scout is worried about the way the town, including Mrs. Dubose, speaks of Atticus. She wonders why everyone seems to think Atticus is wrong if he believes what he is doing is right. Atticus basically says here that despite the fact that everyone ("majority rule") seems to disagree, his own conscience believes in Tom's innocence and that is why he has agreed to fight the case.
The bigger lesson Atticus is teaching Scout is very similar to the old poster you may have seen hung in your elementary or middle school classrooms which reads: "What is right is not always popular; what is popular is not always right." Just because the majority of the people think something is wrong, he is teaching Scout to follow her conscience before she follows others.