Judge Taylor is very informal and laid back in his courtroom. At times, he appears uninvolved or distracted but this is not the case. Judge Taylor is very aware of everything that goes on in his courtroom even though he appears to be dozing at times. Scout remarks that the judge looked weary after getting control of the courtroom. (This is following Bob Ewell's line of testimony in which he accuses Tom of raping Mayella. After Bob's crass language and accusation, the judge's usually calm and informal courtroom erupts: 

So serene was Judge Taylor’s court, that he had few occasions to use his gavel, but he hammered fully five minutes. Atticus was on his feet at the bench saying something to him, Mr. Heck Tate as first officer of the county stood in the middle aisle quelling the packed courtroom. Behind us, there was an angry muffled groan from the colored people. 

So, at this point, the usually calm judge actually expresses some weariness. Perhaps at this point he realizes the significance of racism in this trial. It is also possible that the judge is simply weary because he has to deal with an ignorant witness such as Bob Ewell. It is no secret that the judge is fair and therefore sympathetic to the difficulties Atticus and Tom face in this trial. The judge's weariness could also be an expression of that sympathy; in other words, after the courtroom erupts, the judge's weary look might be an acknowledgment of just how difficult Atticus' task will be and how the trial will create division in Maycomb. 

As for the comment that the judge is getting older and no longer kisses his wife much, this goes back to an early comment in which Scout recalls the judge chewing on a cigar. She asks Atticus how Mrs. Taylor could stand to kiss him and Atticus playfully responds that they don't kiss much. The judge does "show his age," according to Scout, and this goes along with his expression of weariness: a recognition that this will be a difficult and controversial trial.