This is a great question, as it shows that you have a eye for detail. Mention of the Idler's Club is found in chapter sixteen. In this chapter we see that the trial of Tom Robinson is about to begin. People from all over the place begin to stream into Maycomb, as if they were going to be entertained.
Within this context, Scout finds herself among the Idler's Club. She describes them as old men who wear khakis with suspenders. They do nothing all day. Since they have a lot of time on their hands, they go to the courthouse and watch. They have become experts in the law. According to Atticus, they know as much as the Chief Justice.
Here is the quote that describes this club.
This was a group of white-shirted, khaki-trousered, suspendered old men who had spent their lives doing nothing and passed their twilight days doing same on pine benches under the live oaks on the square. Attentive critics of courthouse business, Atticus said they knew as much law as the Chief Justice, from long years of observation.