A battery consists of an electrode and a cathode, both often metal, in a electrolyte paste. A chemical reaction between the electrodes and the paste causes a buildup of electrons. Electrons are the particles outside the nucleus of an atom that are lost or gained in chemical reactions. An electrolyte is a solution that conducts electricity because it contains charged particles (ions).
The anode is made from material that has a greater tendency to lose electrons than the cathode. In every day terms, we would say that the material used for the anode corrodes more easily. Electrons can't travel through the electrolyte solution or paste to get from the anode to the cathode. When the anode and cathode are connected by an external conducting wire electrons will travel through it from the anode to the cathode. When you turn on a battery-powered device you are connecting this wire, and when you turn it off you stop the flow of electrons.
Fruits are a source of electrolyte for the chemical reaction. Lemons work well because they are juicy and acidic. Acid is an electrolyte. Potatoes work because they contain a potassium compound that's an electrolyte.
A voltmeter can be used to measure the voltage produced by a fruit battery. Higher voltage can be obtained by using metals that have bigger differences in corrosion resistance and by using fruits that are more acidic. A copper wire and a zinc-coated nail inserted into a lemon and connected to each other with a wire will produce measurable voltage. You can find an inexpensive voltmeter and wires with alligator clips on both ends at RadioShack.
Voltage and current (the flow of electrons) can be compared to a waterfall. When the waterfall has more vertical drop the water going over it has more potential energy. Similarly, a battery has more potential energy, which we call voltage or electrical potential, when there's a bigger difference between the anode and cathode materials' tendency to react. The amount of water going over the waterfall is like current. More current is produced by a battery when it contains more chemicals. You may have noticed that a AAA battery doesn't last as long as a AA. They both produce 1.5 volts when new, but the chemical reaction in the AAA doesn't last as long because it contains smaller quantities of chemicals.