Atticus has been forced to defend Tom Robinson; Judge Taylor had pointed to Atticus and said, "You're it." The judge knows that no other attorney would defend Tom as strenuously as Atticus. And Atticus knows that Tom deserves a fair trial, believing him innocent of the charges against raping a white woman. Atticus knows that it is one assignment that he can not turn down. He worries that Jem and Scout will become bitter, but more importantly, Atticus is afraid that they will catch Maycomb's "usual disease"--the racist hatred of the black man.
Why reasonable people go stark-raving mad when anything involving a Negro comes up...
Atticus explains that he would not be able to "hold my head up in town" if he didn't take the case, and that he could never expect Jem and Scout to trust him again. Trust--both with the townspeople and his children--are very important to Atticus. He knows he cannot win the case, but he must try.