The first reason why Jean Valjean makes the decision to move out of their rented home on the Rue Plumet is that he had seen M. Thenardier repeatedly and was certain that "Thenardier was prowling in their neighborhood" (Vol., 4, Bk.9, Ch. 1). The Thenardiers were the innkeepers that Fantine left Cosette with to be taken care of while she went off to a neighboring village to find work to provide for Cosette. The Thenardiers proved to be very cruel to Cosette. M. Thenardier also met Jean Valjean when he came to rescue Cosette. He also knows of Valjean's identity as a convict and once saw Valjean in Paris and recognized him. Not only that, Thenardier is quite willing to turn Valjean in to Inspector Javert. Therefore, seeing Thenardier in the neighborhood is certainly reason for Valjean to be alarmed.

The second reason Valjean not only wants to leave the Rue Plumet but Paris as well is that Paris is becoming more and more politically dangerous. As a result, the police in the city were becoming more and more active and more and more suspicious, as we see in the lines:

Paris was not tranquil: political troubles presented this inconvenient feature, for any one who had anything to conceal in his life, that the police had grown very uneasy and very suspicious. (Vol. 4, Bk. 9, Ch. 1)

Therefore, Valjean makes the decision to leave Paris with Cosette and move to England.