You can find your answer in chapter 4 of the book. The man in the yellow suit asks Winnie's grandmother whether she has heard the elf music before.
In chapter 4, the man in the yellow suit first approaches Winnie. He asks her whether she has lived in her part of the woods for long. Next, he tells Winnie that he is actually looking for a family.
For her part, Winnie tells him that she has lived in the woods "forever." Upon hearing this answer, the man in the yellow suit becomes thoughtful. Winnie encourages him to inquire with her father about the family he is looking for.
The man in the yellow suit answers that he will. However, before he can do that, he is interrupted by Winnie's grandmother. The latter demands to know who Winnie is speaking to.
Winnie answers that she is speaking to a man who is looking for someone. The man in the yellow suit then turns his attention to Winnie's grandmother. He tries to get information out of her. However, the old lady is suspicious of the man's motives and refuses to answer his questions.
While they are speaking, beautiful music plays. It is ethereal in nature and sounds like it is coming from a music box. Winnie's grandmother exclaims that the music comes from the elves. She then takes Winnie by the hand, and both enter the house.
The man in the yellow suit, of course, is left standing by himself. From the music, he believes that he has found the family he has been searching for.