Chaucer used a number of familiar stories and familiar characters in this work, but tweaked them to make them seen new and put a couple of unique spins on the whole process of story-telling.
The first innovation was writing in English. Up until The Canterbury Tales, all serious Literature was done in Latin or French. Chaucer, however, wanted his stories to be understood by the common people and not just the educated elite. Chaucer knew both the common and the wealthy, so he had a unique perspective from which to begin.
His second innovation was the "frame tale." The stories are extremely diverse, but Chaucer unifies them with the idea of travellers on a journey telling stories to pass the time. This was a practice that was common among all travellers (and still is common around camp fires and long car trips.) Chaucer took the idea and applied it to his literature, something no one else had ever thought to do.
So, Chaucer's inspiration was likely two-fold: take a common practice and use it to frame a series of new and familiar stories to paint a picture of life in the 14th century.