Specifically, in Fahrenheit 451, Montag turns to books because he is unhappy. He wants some of what Clarisse has.
Montag is smart enough to see that television is simple-minded, that his marriage is without love, that he is incapable of looking at nature and finding something of value in it. He is smart enough to know that something is wrong with a society in which technicians that pump people's stomachs after overdoses of sleeping pills, do it so often that they are nonchalant about it.
Montag simply wants more. Clarisse and her family show him what conversation can be like. She shows him the interest in nature and love that she has. And he wants the same things.
He turns to books in the hope that they can change his existence. And, eventually, they do.
Books are banned because they contain ideas that might make the general population unhappy or dissatisfied with society. This futuristic Utopia is based on keeping people entertained and their minds numb to any unpleasantness. All people want to do is watch the 4th wall, interactive TV and be amused. Books sometimes have unhappy endings. Mustn't make people sad! Sometimes books discuss subjects that are not the norm. Mustn't give people ideas! Books make people think, and the last thing the government wants is to have a thinking society. Therefore, books are evil and must be destroyed. Think of all the escapism that books provide today's reader, and the insight into different life styles. 1984 is the ultimate form of censorship.
The book that he has that is the last of its kind (as far as he knows, but there's really no way to actually know) is the Bible.
I think that Montag turns to books because he thinks that they must have the answers. He explains this some to Faber. He talks about how he knows that something is missing. He knows that people in his time are worse off than people were in the old days. So he wonders what has caused the change. The only thing he can think of is that they used to have books and now they don't.
So because that is the difference he can think of, he thinks that maybe books can help him out.