In this statement, Atticus is explaining that he does not understand the racisim and prejudice the townspeople feel. Generally, Atticus is well liked and gets along with the other townspeople. Before the trial of Tom Robinson, Atticus was usually spoken to (or about) with respect and even reverence by the townspeople. Their racist and prejudice ideas lead to the conviction of an innocent man. Atticus is able to show that he is innocent in court, but the townspeople are unwilling to accept it. Atticus makes this statement to express his confusion and disgust at their attitude. Clearly, he does not share their ideas. We see his defense of Tom and his caring attitude toward Tom's family. The townspeople find this behavior confusing and confrontational. Atticus cannot understand how rational, normal people can become so irrational when dealing with a simple matter of skin color.
Throughout the narrative of To Kill a Mockingbird, Atticus Finch displays complete common sense and logic. So, when people go against reason, he is baffled. In Chapter 9, he terms the unreasoning bias of the townspeople "Maycomb's usual disease"; for, when there is "anything involving a Negro," people go "stark raving mad."
Unlike others in Maycomb, Atticus Finch follows a fundamental ethical pattern that does not make exceptions: his belief in the rights of all people under the judicial system of the United States. That is, all people, regardless of color or creed are allowed equal rights under the law. However, the unwritten social code of Maycomb does not adhere to this code, and it is this code that baffles the reasonable Atticus Finch.
In Chapter 20, Atticus professes his belief, as well as declaring that he is no idealist, but simply a believer in the integrity of America's courts. However, the integrity of Maycomb's court is, indeed, compromised.