In general, these designations of being the poet of the "eye and ear" refer to Wordsworth's tendency to describe sights and sounds of events in nature with such profundity that it is as if he's describing thought. Shelley and other subsequent critics have said this of Wordsworth. Wordsworth was (and is) known to have described common, ordinary things (especially in nature) in his poetry. If he is described as the "poet of man," it is because one of his goals was to describe such ordinary things in more sensual prose to get at the root of fundamental human experience. In describing such things, he would of course describe the sights and sounds but he would give these ordinary sense impressions extra-ordinary importance.
For example, in "The Solitary Reaper," the speaker (Wordsworth) describes a moment when he saw a Scottish girl singing by herself. The girl was unaware of being watched and was therefore singing without reservation. The speaker of the poem describes it as natural as the song of a nightingale. Her accent is too deep for him to decipher. Yet, he is so overwhelmed by this sight and these sounds that he considers any possible deeper significance they might have.
Perhaps the plaintive numbers flow
For old, unhappy, far-off things,
And battles long ago:
Or is it some more humble lay,
Familiar matter of to-day?
Some natural sorrow, loss, or pain,
That has been, and may be again? (18-24)
Wordsworth has taken relatively ordinary sights and sounds and transformed them into vehicles for the imagination.
As much as Wordsworth speaks about the imagination and transcendent experience in life, he often does this by using descriptions of the senses. (Also remember that "seeing" and "knowing" are often used interchangeably by many poets and philosophers.)
In "Lines Composed a Few Miles Above Tintern Abbey," Wordsworth makes this claim overtly.
Therefore am I still
A lover of the meadows and the woods,
And mountains; and of all that we behold
From this green earth; of all the mighty world
Of eye, and ear,-both what they half create,
and what perceive; well pleased to recognise
In nature and the language of the sense,
The anchor of my purest thoughts, the nurse,
The guide, the guardian of my heart, and soul
Of all my moral being. (102-111)
He says that he is a lover of "all that we behold . . . Of eye, and ear,-both what they half create, / and what perceive." Here, he makes a connection between sense experience and the imagination (creative thought and/or perception). In other words, he notes how the world we behold (experience) is "half created" by our sense experiences and half created by our own perceptions. He tried to show this in his poetry.
Wordsworth is the "poet of man" because he describes normal, everyday experiences in nature. He tried to capture rural, rustic life rather than writing about kings and warriors in a lofty prose.