Let x and y be the two numbers

We want x-y = 8 and xy=P as small as possible

As you wrote, x=8+y

Once you have that, you can substitute into the other equation, and get

P=(y+8)y = y^2 + 8y

Just as you had.

NOW:

the nice thing about this, is you have P written as a function of only 1 variable, and P is a parabola. Parabolas come in two flavors: rightsideup and upsidedown:

"Right side up" parabolas have a minimum. There is a spot on the parabola that is lower than any other. (The "vertex") But they don't have a maximum. That is, you can always go higher and higher.

"Up side down" parabolas have a maximum. There is a spot on the parabola that is higher than any other (the vertext). But they don't have a minimum. You can always go lower and lower.

Our parabola looks like: