In Walden, Thoreau alludes to classical sources, the Bible, and great English authors. It would be as common among the educated audiences that Thoreau was addressing to know Greek literature, the Bible, and Shakespeare as it is today to understand such as allusions as "we're not in Kansas anymore."
An example of Biblical allusion in Walden is "it would be like keeping new wine in old bottles." This is a slightly altered reference to Jesus's statement in three of the Gospels that it ruins both the wine and wineskin to try to reuse an old wineskin. This is a metaphor in which Jesus tries to explain why he has abandoned old rituals, such as fasting. In Thoreau's case, he is saying people should literally wear their old clothes until they feel so inwardly changed that it would seem strange to keep the old clothes on. Like Jesus, he means that transformation should be inward, not a matter of outward ritual.
An example of an allusion to Shakespeare is "the winter of man's discontent." This is a reference to the opening lines of Richard III, which read:
Now is the winter of our discontent
made glorious summer by this sun of York
As can be seen, Thoreau alters this quote slightly. He doesn't supply the second half of it, as he can assume his audience will know it. Thoreau is, on one level, tongue-in-cheek in his use of this reference. Shakespeare is writing metaphorically, saying a new and better day has come with the reign of Edward IV. Thoreau, however, is literalizing the allusion, using it in the context of the winter snow melting and spring coming, which means he can begin his sojourn at Walden Pond. However, he is also speaking in metaphor, saying his own discontent is melting like the snow as spring rouses him to new action. A better day is coming for him, too, as he embraces a life of simplicity.
However, as Richard III ends tragically, one of the undertones of the allusion is that venture could also end tragically: Thoreau doesn't quite know what he is getting himself into. This allusion might keep his audience curious.
Finally, an example of an allusion to Greek mythology comes when Thoreau likens his new, tiny home by Walden Pond to a place fit for the gods, saying:
This was an airy and unplastered cabin, fit to entertain a travelling god, and where a goddess might trail her garments.
In this way, he is exalting simplicity to a heavenly status.