The "conversation poems" are so named because they all represent an attempt by the poet to converse on the poet's examination of nature and the role of poetry, as examined through the lens of a particular life experience.
The term is typically applied to a particular set of poems by Samuel Taylor Coleridge, of which "The Eolian Harp" is one. It can also refer to poems similar in style to these, like William Wordsworth's "Lines Composed a Few Miles Above Tintern Abbey."
In "The Eolian Harp," Coleridge meditates on the experience of sitting outdoors with his then-fiancee (later wife) Sara, watching the evening fall. He becomes caught up in the image of the harp and its ability to bewitch or transport the human soul through music. Sara, however, intervenes with a look, reminding the poet of his responsibility to his faith in Christ.
In "Lines," Wordsworth returns to a spot above Tintern Abbey after five years' absence. He notes "the landscape with the quiet of the sky," including the farmland that surrounds him, the greenness of spring, the cottages and hedgerows. It's a scene that Wordsworth says he has returned to many times while "in lonely rooms, and 'mid the din/Of towns and cities," providing him with "tranquil restoration." Not only has this scene helped Wordsworth feel better, he says, it has also helped him transcend human cares and "see into the life of things."
Now that he's seeing the same scene again in real life, Wordsworth feels it is full not only of "present pleasure," but also of beauty and meaning that will sustain him in the future. Even though he's changed, nature has not. Nature thus gives Wordsworth a link to that which is eternal and meaningful in his soul.
In both poems, the poets find themselves momentarily transported by the rapture of nature and natural images, only to return by the end to some sense of their ordinary lives - a sense that is nevertheless changed by the rapture of oneness with Nature.
Because the poems walk the reader through this experience, they may offer a more effective way to share the experience than other genres would. For instance, an essay that began "I was sitting out in nature one day, and I was thinking about how I was part of everything...." inserts a voice of separation between the reader, the speaker, and the event that the poem does not.
Coleridge and Wordsworth aren't merely trying to explain what they were thinking or feeling while sitting around in nature; they're trying to share that experience with the reader, so as to lead the reader to a deeper understanding of the joy and sublimity they feel in their relationship with nature and poetry. The poets may have chosen this approach because they struggle to articulate those feelings in any other way than by sharing the experience - note how neither poem contains any sort of definitive "oh yes, this is definitely what I learned from my analysis and will apply to my future life." Rather, both poets come away from their meditation with a profound sense of peace and purpose, but not one that can be translated easily into a moral or a to-do list item.