An allegory, according to the eNotes site under Guide to Literary Terms, is:
an extended metaphor in which a person, abstract idea, or event stands for itself and for something else. It usually involves moral or spiritual concepts which are more significant than the actual narrative.
What this means is that an image represented by artists and authors is represented in both a literal (taking something for what it actually is) and a figurative (not used in a literal sense) sense.
An example of an allegory is when an author uses the image of a Grim Reaper to represent a literal figure (an entity clothed in black with a sickle) and a figurative meaning (one which represents death).
Another examples of allegories fall under the literary genre of fables. All fables are meant to teach lessons. They use literal imagery to teach a figurative message.
Take for example the fable "The Dog and His Shadow". A dog holding a piece of meat sees his own reflection in water. Thinking that the "other" dog (his own reflection) possess a larger piece of meat the dog drops his on meat and tries to get the bigger piece of meat that his shadow possesses. In the end, the dog loses the real piece of meat and the meat of the reflection (which does not exist). The moral of the story is not to be greedy. This is an example of a simple allegory.
As for examples of things which are not considered allegorical, it would be anything taken for face value which one could not give any alternate or figurative meaning to. This is actually hard given many people regard things with two meanings naturally. For some, a tree losing its leaves does not simply reflect a tree losing its leaves. Instead, it represents the coming of fall or the end of summer.