The magnetic field around a permanent magnet is the strongest closest to the poles.
One of the ways of imagining a magnetic field is that the magnetic field is represented by magnetic flux lines. These flux lines are imaginary lines which emanate from the north pole of a magnet and go into the south pole, but between the two they don't just travel in a straight line but radiate out in all directions. You can see a good image of this partway down the page of the first reference.
If you think of the flux lines all coming from one point, the north pole and all going to the south pole, but being much less dense in between, these flux lines are the densest at the poles themselves. This density of field lines represents the fact that their magnetic fields are strongest there.
Whilst that was quite a handwavy explanation of the topic, the same result can be acquired if you crunch the maths of Maxwell's equations which describe electromagnetism. If you want to know more, I suggest you look at the second reference.