For many years Wordsworth's sister Dorothy kept a journal in which she recorded the various people and places that she and William (and at times Coleridge) saw and visited. In the April 15, 1802 entry of Dorothy's journal is her description of "a few daffodils close to the water-side" that Wordsworth, some years later, went on to write this famous poem about. To read the entire entry, consult the notes in the back of the Penguin Classics edition of Wordsworth's Selected Poems (pub. 1994). In terms of taking the "idea" of the poem from his sister, I have never heard any evidence to support that idea.