There are two types of substances that are good conductors or electricity. Metals conduct in the solid phase because they contain electrons that are mobile, or free to move around. When a copper wire is used in an electrical circuit it has electrons moving through it whenever the switch is closed. When the switch is open the circuit is broken and the electrons stop moving. These valence electrons aren't really associated with specific nuclei in the solid metal, and end up near different nuclei when they move in response to a potential difference. Think of it as electrons playing musical chairs. When the switch is opened (turned off) it's like the music has stopped.
Ionic substances, also known as salts, conduct electricity when dissolved in water or when in the liquid state (molten). The ions that make up an ionic solid are held in a lattice formation by the attraction of opposite charges. When the ions are free to move around, as when dissolved or molten, they can carry charge from one electrode to another.