First off, not everyone agrees with this position. Typically, probation is not revoked at the slightest provocation. While different probation officers are, of course, different, they tend to file to revoke probation only for more serious breaking of the rules. For example, if a probationer is required to make restitution, to attend classes, and to do community service, it is unlikely that they would have their probation revoked if they are a little short on restitution one time. This is partly for reasons of humaneness and justice, but it is also partly practical. Governments do not want to have the expense of jailing people for petty offenses.
However, if you were to argue that probation should be revoked at the slightest provocation, you can have two main points. First, you can emphasize that the probationer is still serving their sentence although it is a sentence with some freedom allowed. It may seem wrong to jail someone for a petty infraction of probationary conditions, like being $5 short on a restitution payment. On the other hand, that is not really what the person is being re-jailed for. They are being re-jailed for their original crime. They were given conditional freedom in their sentence to avoid being in jail and they did not take advantage.
Probation: A sentence whereby a convict is released from confinement but is still under court supervision; a testing or a trial period. (West's Encyclopedia of American Law)
Second, the probationer has essentially signed a contract. They have acknowledged that they are serving the remainder of their sentence under conditions granting court supervised freedom and have sworn to do certain things. They knew the consequences of breaking their oath. If they violate their probation, they do so knowing the consequences. When they do so, it is as if they were showing that they do not believe that they need to follow the terms of their continuing sentence.
Thus, while most probationers are not held to strict conditions, it is justifiable to do so.