There are two places in Mary Shelley's Frankenstein where a worm is spoken of. Both of these quotes appear in chapter four of the novel.
Darkness had no effect upon my fancy; and a churchyard was to me merely the receptacle of bodies deprived of life, which, from being the seat of beauty and strength, had become food for the worm.
I saw how the fine form of man was degraded and wasted; I beheld the corruption of death succeed to the blooming cheek of life; I saw how the worm inherited the wonders of the eye and brain.
In the first quote, Victor is explaining why he is able to take bodies from their graves in order to complete his desire to reanimate life. For Victor, the bodies are only food for the worms.
Very soon after the statement regarding the worms' existence being linked to feasting on bodies, Victor seems to become very jealous of the worms. For him, it seems that the worms feasting upon the deceased bodies brings them more then nourishment. By the worms feasting upon the deceased, they are able to inherit (or absorb) the "wonders of the eye and brain." Victor is jealous of the fact that the worms possess the deceased, and through this, they possess all the knowledge the bodies contain. Therefore, by Victor taking the bodies from the worms, he is able to possess the "wonders of the eye and brain."