Great question. I can't give a definitive answer. But here's what I know. Kieou-he-yu is a character inThe Analects,"Classified/Ordered Sayings," and Khoung-tseu is a name for Confucius. Thoreau was "thoroughly" interested in Hindu philosophy as it adhered more closely to his transcendentalism, certainly more than the Unitarian variety of Christianity that he experienced. You can find examples inWaldenwhere he is openly critical of the Unitarian church. He included this (and other references to Eastern philosophy) because some of the concepts of those philosophies fit his transcendental ideas about being a part of nature. Although Confucianism and Buddhism are different entities, the former influenced the latter, the Buddhist idea of all things are connected sounds like it could appear in Thoreau's own words.
As for the anecdote, it appears in a context wherein Thoreau is criticizing people who live for the news. (How even more appropriate this would be in today's world.) They are so deluded by outside news that they have no sense of themselves, passively taking in just the surface of reality. He then says the philosopher calls all news gossip. Following the Khoung-tseu anecdote, Thoreau criticizes the preacher who comes down on the tired farmers at the end of the week, rather than preaching to them to rest and be rejuvenated for the next week.
Sandwiched between these two sections, my guess is that Kieou-he-yu's plea to Khoung-tseu (Confucius) to help him diminish his faults represents someone who is introspective, humble and thoughtful; not concerned with "the news," even the news from Confucius himself. If Kieou-tseu were a preacher, he would tell his farmers/parishioners to rest and contemplate the significance of their work with nature - rather than bombard them (as the news does) with chastisements or trivial gossip.