One of the most important aspects of English Romantic poetry is the opening up of subject matter to other times and people. While 18th cent. poets (e.g., Pope, Swift, Johnson) focused on either classical Greece and Rome or London city life, the Romantics were much wider in time and space. Part of this involves looking at the medieval period--or at least their conception of it. Hence, poems like Coleridge's "Ancient Mariner" have quite a medieval flavor (in terms of imagery, diction, subject matter) to them. This continues into the Victorian Era (e.g., Tennyson). Of course, this doesn't mean that the Romantics believed in what the medievals did--such as Catholicism or scholastic philosophy. Far from it.
As the e-notes article cited below states, Gothic architecture was also important to the Romantics. This is tied into the idea of the sublime, which is a subject too wide for this post. In addition, one could note that, as in Wordsworth's "Tintern Abbey," it is the ruins of Gothic cathedrals that hold interest.
Finally, in terms of form and versification, the Romantics got away from the heroic couplets of Dryden and Pope. You could say that their use of something like ballad meter is a medieval element in the sense of getting back to the roots of English poetry.
The English Romantic poetry historically begins with the 1798 Lyrical Ballads, a collection combining William Wordsworth and S.T. Coleridge, continues into the works of Keats, Shelley, Byron and Scott. In Victorian poetry, Tennyson, Arnold and the Pre-Raphaelites are to be seen in terms of a legacy of Romanticism. Medievalism happens to be one of the defining features of romantic poetry.
1. Medievalism is appealing to the Romantics in terms of their propensity to go back to the past and revive it in certain ways.
2. Mysticism is another draw of the medieval as far as the Romantics are concerned. The Romantic interest in the so-called addition of strangeness to beauty is relevant over here.
3. The mythological interest of the Romantics also draws them to the middle ages.
4. There is an attraction with the medieval literary structures, especially the long narrative ballad as we see in the poetry of Coleridge--Kubla Khan and Christabel or Byron.