How does the memory of what the speaker saw affect him later?
The short answer to this is that the memory of what he sees when he wanders around like a cloud helps make him feel good later on. You can find this in the last stanza of the poem.
The poet says that now, if he's lying around feeling bad, all he has to do is to think about what he saw. Then his heart fills with pleasure and dances with the daffodils.
This poem always reminds me (because of its meaning) of the song "My Favorite Things" from the Sound of Music. The speaker is saying that when he feels bad, all he has to do is remember what he saw (instead of his favorite things) and then he won't feel so bad... And I apologize if you've never heard of this song...
The delightful dance of the daffodils as seen in tandem with the dance of the waves alongside and a gleeful beauty of the entire landscape of nature--all these visual images of wandering come back to the poet when he is in a sorrowful and rather hollow mood seated on his couch, suffering from alienation and boredom of loneliness. It is this memory that energizes him and becomes the 'bliss of solitude' thereby completely changing his attitude toward his loneliness.
This importance of memory is not just comforting and therapeutic, it has a special value in Wordsworth's idea of poetry and its composition. As he says in his Preface to the Lyrical Ballads, poetry is composed of 'emotions recollected in tranquility'. The function of memory is paramount to poetical composition in Wordsworth.
The speaker in the poem beings his poem with a lonely and melancholy tone. He goes on to say how majestically the waves of daffodils had been and how beautiful they were. The sight had been one of the most beautiful ones he had ever seen.
Later when he begins to feel moody or blue, he thinks back on the screen of the daffodils and it brings him some pleasure and joy. The whole serenity and the magnificent beauty had stayed with him and he uses it as a happy memory to ward off the solemn feelings.