First, it should be noted that there is a difference between prosecutorial and sentencing discretion. When a police officer, a district attorney, or the president makes a choice to enforce or not enforce a law, that is prosecutorial discretion. There are arguments for and against this kind of discretion, too, but as a practical matter, we do not have sufficient resources to go after all the "bad guys" every time. Given insufficient resources, the police officer on the beat who sees two crimes being committed, for example, must be afforded the discretion to decide which to act upon. On a larger level, the president knows full well there is insufficient funding to find, prosecute, and deport every single unlawful immigrant. So he exercises prosecutorial discretion. The fact that he does so for humanitarian or political reasons tends to mask this reality.
My view on discretion in sentencing is a bit cynical. Today, how we decide how much latitude to allow judges is starting to be based upon economic realities as well. States that instated mandatory sentencing have found, to their dismay, that it costs them a great deal to take care of those who have been sentenced. This has led to some movement away from mandatory sentencing, as well as a decriminalization of various non-violent crimes. On the other hand, as prisons become increasingly privatized, there could be a trend towards some mandatory sentencing again, since there is a profit to be made in prison privatization. Are these good bases upon which to decide on judicial discretion or lack thereof? Probably not, but as the above response suggests, this is a tricky issue and opinions tend to be highly personal.
The answer to this question is, of course, a matter of personal opinion. There is no objective way to tell how much discretion these actors should have. My own view is that they should have “some” discretion, but the line between not enough discretion and too much discretion is impossible to draw.
These actors need to have some discretion. If they have no discretion whatsoever, it is likely that there will be outcomes that are unjust. For example, if a judge does not have the discretion to reduce a sentence for someone who has committed a crime in which there are extenuating circumstances, the sentence might end up being unjustly long. As another example, if a parole board had no discretion in determining which inmates should be paroled, we might see inmates who had not reformed being paroled while those who had been model prisoners remained incarcerated.
However, we cannot allow these actors to have unlimited discretion. When that happens, the actors can use their discretion to make unjust decisions. They can mete out excessive punishments to people they are biased against and they can make lenient decisions in the cases of people to whom they are sympathetic. For example, one could argue that the prosecutor in the case of the police officer in Ferguson, Missouri used his discretion to present a weak case to the grand jury, resulting in the officer not being indicted for killing Michael Brown. Another person might argue that the federal government (led by President Obama) is using its discretion wrongly when it chooses to allow certain groups of illegal immigrants to remain in the country legally.
What we can see, then, is that there are dangers in both excessive discretion and insufficient discretion. However, it is exceedingly difficult (and perhaps impossible) to know exactly how much discretion is the right amount.