You can find your answer in chapter 4 of the book. The man in the yellow suit asks Winnie's grandmother whether she has heard the elf music before.

In chapter 4, the man in the yellow suit first approaches Winnie. He asks her whether she has lived in her part of the woods for long. Next, he tells Winnie that he is actually looking for a family.

For her part, Winnie tells him that she has lived in the woods "forever." Upon hearing this answer, the man in the yellow suit becomes thoughtful. Winnie encourages him to inquire with her father about the family he is looking for.

The man in the yellow suit answers that he will. However, before he can do that, he is interrupted by Winnie's grandmother. The latter demands to know who Winnie is speaking to.

Winnie answers that she is speaking to a man who is looking for someone. The man in the yellow suit then turns his attention to Winnie's grandmother. He tries to get information out of her. However, the old lady is suspicious of the man's motives and refuses to answer his questions.

While they are speaking, beautiful music plays. It is ethereal in nature and sounds like it is coming from a music box. Winnie's grandmother exclaims that the music comes from the elves. She then takes Winnie by the hand, and both enter the house.

The man in the yellow suit, of course, is left standing by himself. From the music, he believes that he has found the family he has been searching for.

"Wait!" said the man at the gate. He had stiffened, and his voice was eager. "You've heard that music before, you say?"

The above quote can be found in chapter four of Natalie Babbitt's Tuck Everlasting.  Winnie has been playing in the yard, and the sun is nearly set, so Winnie is getting ready to catch some fireflies.  Unfortunately, she is interrupted by the arrival of the man in the yellow suit. He asks Winnie all kinds of questions about how long she and her family have lived in the area.  Eventually, Winnie's grandmother comes out and is immediately suspicious of the man.  A few moments later, the trio hears some music coming from the woods.  Winnie's grandmother is very excited, because she believes that she is hearing magic elf music . . . again.  

"That's it! That's the elf music I told you about. Why, it's been ages since I heard it last."

The man in the yellow suit is very interested in the fact that Winnie's grandmother has heard the music before, and he double checks to make sure that he heard her correctly.