His fever symbolizes the "heat" of discovery that something is wrong with what he is doing. It is very much like a conscience--his own Jiminy Cricket sitting on his shoulder saying, "Pinnochio, are you sure you want to skip school today?"
Montag is suddenly coming to the realization through his contact with Claire and the witnessing of the old woman choosing to burn with her books that burning these books and destroying these ideas is wrong. He is destroying part of civilization--the freedom of thought and speech that generates new ideas.
His fever comes immediately after destroying the lady, her house, and her books. He has stolen a book from her house and many others. He has been preserving books for a long time now and only just begun to realize how he truly feels about his job.
Unlike other firemen, Montag does not recover even though he does return to work long enough to have his own house reported.