Administrative discretion is something that may or may not be exercised by workers in a bureaucracy. Workers in a bureaucracy have administrative discretion if they have the ability to apply their professional judgment to a given situation. They do not have administrative discretion if they are required to simply follow preexisting rules to the letter.
There are clear advantages to administrative discretion. Mainly, such discretion allows for outcomes that are more just and more humane. To take one example, we can see highway patrol officers as members of a bureaucracy. They are, among other things, enforcing speed limits. Now let us say that you are going 75 mph in a 70 mph zone in good weather with no traffic around. Technically, you deserve a ticket, but an officer exercising their administrative discretion might choose to say that you are not really causing any great danger to yourself and others. This would be a more humane outcome that takes all factors into account rather than blindly applying rules.
The problem with this is that it can lead to abuse. Let us say that you apply to your local zoning authority for permission to build your house one foot closer to the lot line than is legal. You are rejected. Now imagine that someone else applies for the same thing only they have friends in the zoning authority and they are allowed to build in the same way you wanted to. This would, at the very least, make you feel like the authority was abusing its discretion in ways that were not really based on professional judgment.
Thus, administrative discretion can make for more humane and just decisions, but it also opens the door to abuse of power.