Faber means that people who don't want to contribute to society often end up destroying it.
Montag explains to Faber that Captain Beatty has read a lot of books—because of this, he's knowledgeable, and Montag wants to talk to him. If he does, though, he's putting himself and Faber at risk. Montag thinks back on his past as a fireman and explains to Faber that only a week ago he was pumping kerosene into the places he needed to burn and felt like it was fun. Now his world is radically different.
Faber says, "Those who don't build must burn. It's as old as history and juvenile delinquents." What he means is that human nature isn't static. People either create or they destroy—it's rare for someone just to sit and do nothing. Montag was a person who destroyed; his experiences changed him enough that now he's someone who creates.
Faber goes on to say that there's "some of it in all of us," showing that the human tendency to create or destroy is universal.