I think that there are two things that you can point to that made him think this.
First, he realizes that in his day, people are generally not happy. He knows that, in the past, people were happy. He tries to figure out what is different now and the main thing he can think of is that there used to be books and now there are not.
Second, there is the example of the old lady who burns herself along with her books. Montag thinks that, if she was willing to do that, there must be something important and compelling about books.