Foreshadowing is used to give the reader clues at what will happen, whether or not he consciously realizes it. It keeps the reader interested if he is looking for foreshadowing, but even if he's not he will realize an event was foreshadowed earlier in the story when he comes across the important event at which it hinted.
I would like to add a bit and to take it all a little deeper: we are talking about the psychology of reading, an analysis of which is part of the "Reader-Response" school of criticism. One such critic is Stephen Booth, who notes that it is human nature to love to feel superior to someone less clever than ourselves, even if that someone is the "self" who was reading just a few pages ago!
That sense of satisfaction, of superiority, of being "in the know," carries us forward, even if we are still rather ignorant and still falling for the author's tricks. This is the reason that most of us love to be tricked by a story full of twists and turns and ironic reversals,
Although this makes us all sound rather egotistical, I tend to agree with him. Otherwise, why would we stay with a story that continually attempts to trick us by giving us "leading" clues in one direction, all the while providing true foreshadowing that we can go back and notice, to help us read more intelligently from here on?
So, I agree that foreshadowing keeps the reader engaged, but there is also an element of trickery involved (otherwise, the author would just say outright rather than foreshadow), which appeals to our innate desires to master a difficult process we're in while also finding ways to applaud ourselves at our own superiority. Ultimately, we engage in a kind of conspiracy with the author, don't we?
Foreshadowing accomplishes a couple of things. It keeps the reader intrigued. They keep reading because the clues the author leaves is like a treasure hunt. We have to be very clever to find them. Also when we get to the end, we have that "Aha. I knew it moment," "Or the wait a minute, I gotta go back and find the clues moment." Either way we are more involved as readers.
All the answers you have so far are correct. The only thing I'd like to add is that effective foreshadowing keeps readers engaged. Let's face it, one of the reasons we all kind of "grow out of" certain works of literature is because they become too predictable. When the foreshadowing is subtle and effective, readers are more likely to be intrigued by and connected to the work. Effective and mature use of foreshadowing is delightful and intriguing to any reader.
- Accomplished authors use foreshadowing in order to ensure the narrative of verisimilitude. When, for instance, an author writes a short story that contains a surprise ending, the readers are, indeed, taken aback. But, if the author is accomplished in his/her art, the readers can return to the exposition and discover the most subtle of hints of the action that occurs in the denouement. In O. Henry's classic story, "The Gift of the Magi," for instance, while readers realize that Della has only $1.84 with which to buy a Christmas present because she and her husband are poor, and while they comprehend the love that Della has for her husband, they become very engrossed in Della's impetuous decision to sacrifice her luxurious hair in order to purchase a present for her husband. All the while that the readers follow Della's actions, they do not consider that Jim 's love is equally as unselfish and that he is committing similar acts. However, with the denouement in which the couple discovers that they have relinquished the possession for which the other has bought a present, the readers' surprise should not be as great since O. Henry lays hints that each spouse had only one valuable possession, and in their impoverished state, the necessity of selling this one possession is implied if there is to be a present for the other.
- In a sense, the use of foreshadowing creates a certain intrigue for the reader as the suggestions and hints pique a reader's desire to continue the narrative in order to discover what does happen, as well as to ascertain the author's point in writing, or the theme.
I can think of at least two reasons that authors use foreshadowing. The first is that it creates suspense. If an event is foreshadowed, the readers want to read further to see how it develops. An example might be from Romeo and Juliet, in which the ending is told to the readers: "A pair of star-crossed lovers take their lives." We know that things will not end well for Romeo and Juliet, but we don't know how or why. We read the play to find out these answers. Think of horror movies in which eerie music precedes a terrifying event. The music gives us clues that the situation will soon change for the worse, but we are not sure how. We stayed glued to the screen to find out.
Secondly, the author prepares us for the ending so that it does not seem to be random or arbitrary. The ending, even if it surprises us, should make us feel as if had we been more alert as readers, we might have been able to guess it. These types of endings make the best stories. In other words, we can't just have our hero get hit by a truck, coming out of nowhere. We need to feel that the author has a plan, that he or she began the story with the ending in sight. In this way, we derive much pleasure from the ending, even if the ending is a tragedy. In the short story "Most Dangerous Game," by Connell, for instance, the opening dialogue between Rainsford and Whitney foreshadows a reversal in the roles of the hunter and hunted. While we might be able to guess that Rainsford will become the hunted so that he can feel the fear that Whitney claims the hunted feel, we are surprised by the second reversal in which Zaroff becomes the prey. So, a certain degree of surprise and predictability (the later through foreshadowing) is necessary for a story to be entertaining.
I think authors use foreshadowing for a variety of reasons. Many use it to develop plot, or help create or enforce a theme, like you said. But I also think that authors can/do use it to help keep a reader interested. Or at the end, when they discover the outcome, to allow them an "Aha!" moment where they realize what was being foreshadowed earlier in the story.
Thanks you :D