As you know, an allegory is something that stands for something else. Christians believe they are all "pilgrims" of a sort, on a journey through a world that is not their home. Thus, in The Pilgrim's Progress, the Pilgrim is on a journey as well, following the Bible's advice to "seek first the kingdon of God and its righteousness." Heaven is the Christian's home, but he cannot reach heaven because of sin. He must rid himself of his sin before he can enter heaven. Since man is not capable of ridding himself of sin, Christians believe this is why Christ died on the cross, to take away the sins of the world. Once sins are taken away, a Christian's "burden" is gone because Christ bears the burden for the sins of mankind.
In The Pilgrim's Progress, the Pilgrim, Christian, is traveling to the Celestial City (Heaven). Along the way, he encounters many obstacles, just like a Christian does in this world. He also encounters many people, all of whom he tries to convince to go with him, but they are tied to this world by many things. The names of the people are also symbolic: Obstinate, Atheist, Prejudice, and Ill-will. These people cannot or will not go to the Celestial City. Christian carries his sin with him for awhile (his burden), but he finally is able to rid himself of it. Also, he encouters monsters on the way, such as Apollyon, who is Satan. A character called Evangelist points the way to the "gate" and Christian meets Piety, Prudence, and Charity who teach him important things about religion.
If you read the overview here on eNotes, you can find out additional information. This work is supposed to represent what it is like for a Christian to live in this world while trying to get to the next.