Really, the extended metaphor in this poem runs through the whole of it, not only these few lines. The "chambered nautilus" is, itself, a metaphor. Holmes is comparing the human soul to the inhabitant of a shell who, upon outgrowing it, finds another which fits him better and thinks no more of his former home, now too small and constricting for him.

In the section you particularly mention, the speaker calls upon his soul--or, rather, he says that he hears a voice within the "caves" of his "thought" calling to his soul--to leave its "low-vaulted past." That is, the soul should, like a sea creature, decide deliberately to grow and increase in size and understanding until its current home seems too small for it, and seek a new home which fits it better. Note the secondary connection drawn throughout the poem between shells and "temples," with church imagery such as "vaulted" and "dome" underlying the fact that the soul is here under discussion, rather than the physical body. The speaker's ultimate hope is that the soul will continue to grow until such time as every "shell" is too small for it--that is, when a person can finally leave its last "outgrown shell" behind it, he or she will have departed her body and ascended to heaven in a spiritually developed state.