If the atmospheric pressure outside of the pipe were greater than the fluid pressure inside the pipe, then the fluid in the pipe would not be leaking out. The atmosphere could be leaking into pipe if the pressure difference were great enough. This could be equally bad, but you would not see any fluid leaking out of the pipe. If the two pressures are equal, then fluid still won't be leaking out of the pipe.
Your question seems to suggest that fluid is leaking out of the pipe. Therefore, the pressure inside the pipe is greater than the atmospheric pressure outside of the pipe.
Any fluid (liquids or gasses) tends to flow from areas of higher pressure/concentration to areas of lower pressure/concentration. This is called diffusion. If the fluid is water, then it is called osmosis.
A really easy and fun way to show how atmospheric pressure can hold in a denser fluid is with an empty two liter bottle. Poke 2-3 tiny holes in the bottle using a small needle or thumbtack. Fill the bottle with water and cap it. The outside atmospheric pressure (along with the water's surface tension) should be equal to the water pressure in the bottle. It won't leak out of the holes unless you squeeze the bottle or take the cap off. Taking the cap off allows the air pressure to push down on the remaining water and push it out of the bottle.