Short Answer: Walter is invited to lunch after Jem breaks up a fight between Scout and Walter. When he learns that Walter is the son of Mr. Cunningham, a client of Atticus, he invites the boy to share their meal with Scout and him because Walter's father is an acquaintance of Atticus that Atticus respects.
After Scout's well-intentioned attempts to inform the new teacher about the economic status of other students result in Miss Caroline's feeling that the little girl is trying to undermine her authority, Scout is rendered a few strikes on the hand. Astonished, hurt, and embarrassed by this action that ignites laughter from the class, Scout vows revenge upon Walter Cunningham as the cause of her humiliation. Once outside in the schoolyard, then, Scout attacks Walter and rubs his nose in the dirt until her brother sees her and orders her to stop. When Jem asks her why she has attacked Walter, Scout explains what has happened and stamps her feet at Walter as though to chase off a stray dog or cat.
However, much in the manner of his father, Jem "examine[s] Walter with an air of speculation"; then, he asks Walter if his father is Mr. Walter Cunningham from Old Sarum. After Walter nods in assent, Jem smiles and invites him to their house for "dinner" [in rural Alabama there are people who refer to the noon meal as such]. Jem explains, "Our daddy's a friend of your daddy's. Scout here, she's crazy--she won't fight you any more."