A bicycle that is stationary is not stable and would fall to its side if it is not supported by a bicycle stand. On the other hand a moving bicycle is very stable. As the speed of a bicycle is increased it becomes easier to maintain balance and prevent the bicycle from tipping over.
This is due to the fact that the spinning wheels of a bicycle have an angular momentum. Angular momentum is conserved and unless there is an external torque applied, the angular momentum remains constant. For a moving bicycle to tip over a large torque has to be applied. This would require the cyclist to lean over in one direction by a large extent. Or the angular momentum can be reduced by slowing down the speed at which the bicycle is moving and the rate at which the wheels are rotating. As any cyclist may have noticed it is quite difficult to balance a bicycle moving at a very low speed. That involves maintaining the center of gravity of the bicycle directly over the base of the wheels and if it is moved slightly to one side, the bicycle tips over.