The valence of an atom is determined by the number of electrons in the outermost shell of the atom.
The IUPAC definition limits valence to the maximum number of univalent atoms that may combine with the atom, that is the maximum number of valence bonds that is possible for the given element.
The electron configuration of an atom is most stable when it has one that is similar to the atom of the closest noble gas. To gain a complete octet of electrons atoms can either give their outermost electrons to other atoms or accept electrons and incorporate them into their outermost shell. This is the case with the formation of ionic bonds. Or the electrons may be shared between atoms to form covalent bonds.
Most elements can display different valence when they form bonds and this changes the type and number of bonds formed. The only thing constant is that the electron configuration after the formation of bonds is more stable than what it was before the bonds were formed.
Atoms have a tendency to connect up to other atoms, to form molecules which are compounds, or to form structures. Each type of atom has a preference for connecting to a particular number of atoms. A hydrogen atom likes to join up with one other atom, whereas an oxygen has two vacancies for other atoms to join on to it. This, the "number of other atoms it likes to connect with" is the valency.
Carbon has a valency of 4, so in compounds with carbon there are four bonds to other atoms. Nitrogen has a valency of 3, niobium 5, iridium 3, etc.
The inert gases such as helium, neon, krypton, etc all have a valency of zero, ie the atoms don't really connect to any other atoms, and so there aren't a lot of chemical compounds for inert gas elements.
Valency in practice:
H20: (water) H - O - H
CO2: (carbon dioxide) O = C = O (double bonds)
CH4: (methane gas) a carbon atom with four hydrogen atoms connected to it.
In a structure such as a piece of metal or a crystal, the valency results in the atoms aligning in a regular way so each atom attaches to the appropriate number of other atoms.