Capacitors have two sheets of conductors which are oppositely charged. The capacitance of a parallel plate capacitor is given by the expression e0*A/d, where e0 denotes permittivity, A is the area of the plates and d is the distance between them.
The use of a dielectric material between the plates of the capacitor has three advantages.
First, it becomes easier to keep the capacitor plates very close to each other with their coming in contact. This decreases the d in the expression for capacitance.
Second, under high voltages, air polarizes and becomes a conductor. A dielectric material is able to handle higher voltages than air without undergoing dielectric breakdown and starting to conduct.
Third, the permittivity due to a dielectric is increased from e0 to K*e0, where K is the dielectric constant of the material and is more than 1.
Now looking at the expression for conductance e0*A/ d, we can see that the capacitance due to the use of the dielectric material between the conducting plates is increased as e0 can be made larger to K*e0, and d can be made smaller.