Bradbury uses the Bible as a symbol of human culture and accomplishment.
Montag is obsessed with the Bible. His first contact with Faber is a phone call.
"Professor Faber, I have a rather odd question to ask. How many copies of the Bible are left in this country?"
"I don't know what you're talking about! "
"I want to know if there are any copies left at all." (part 2)
Montag is concerned because he is afraid that his copy is the last copy of the Bible that exists. Mildred even accuses him of caring more about the Bible than her, and in some ways she is right. He does care more about it. She is just a parody of a wife. The Bible is real history and human culture, and more important than her, him, or any one person.
Montag tries to memorize as much of the Bible as he can, no matter the risk, so that it won’t be lost forever. He no longer cares about himself.
Montag explains to Faber why the Bible and the other books are so important to him.
"I don't know. We have everything we need to be happy, but we aren't happy. Something's missing. I looked around. The only thing I positively knew was gone was the books I'd burned in ten or twelve years. So I thought books might help." (Part II)
Faber calls him a hopeless romantic, but he realizes Montag is serious. He has looked at the world and seen the absence of culture. When we lose our books, and our religion, we lose our humanity.
When Montag does find the underground book people, he becomes part of the Bible. Montag memorizes Ecclesiastes, which is significant because it literally means "teacher." When it is time to rebuild civilization, Montag will be a part of it.