The rubber band in a slingshot is an elastic band which when pulled to extend it beyond its equilibrium length returns to the original length when released.
The behavior of the rubber band can be considered to be like that of a spring. When a spring is stretched or compressed from its equilibrium length it requires a certain amount of work to do so. The force that has to be applied is related to the distortion by the Hooke's Law as F = -kx. The work done is stored in the spring as potential energy which is released when there is no force acting on the spring.
In the case of the rubber band in the slingshot, stretching it by an extra distance requires energy which is stored as potential energy in the band. When the band is released, this energy is converted to kinetic energy of the stone. The kinetic energy KE = (1/2)mv^2, as the mass of the stone remains the same, any extra kinetic energy results in an increase in the velocity. If the stone has a greater kinetic energy, it will go faster.
This explains the stone going faster if the rubber band in the slingshot is stretched an extra distance.