Jean Valjean's parole was, in some ways, as harsh as his prison sentence. As a paroled prisoner, Valjean was required to carry papers that identified him as a parolee and to show them to pretty much everyone he encountered. He had to report regularly to the police so they knew where he was and what he was doing, and they were allowed to harass him for seemingly any reason—the fact that he had been a prisoner was enough.
When the Bishop of Digne takes pity on Valjean, he's the first person to do so. His kindness in giving Valjean his silver inspires Valjean to start his life over. Instead of being shunned everywhere he goes by revealing himself as a parolee, Valjean will simply take another name and become someone who has no criminal past—this was much easier to do in a world without the sort of recordkeeping we have today.
There's just one problem: If he takes on a new name and starts living a new life, he's no longer reporting to the police, which is a violation of his parole.
Javert, meanwhile, has been waiting for Valjean to mess up. He's convinced Valjean is a horrible, irredeemable criminal, and that given the chance Valjean will start committing crimes again. To Javert, Valjean's failing to report for his parole simply confirms this suspicion. He expected Valjean to commit another crime, and violating parole is a crime; therefore, he believes he was right about Valjean all along. Javert thus feels completely justified in tracking Valjean down because he's so convinced Valjean is a terrible person who needs to be back in prison.