Pure water consists of just hydrogen and oxygen atoms bonded together to form H2O.
However, water is a very good solvent. That means many different compounds can dissolve in water. Many of these compounds are ionic when dissolved. That means they separate into positive and negative ions. The positive ions are usually from groups one and two on the periodic table and the most common are potassium (K), sodium (Na), calcium (Ca), and magnesium (Mg). Solutions containing the group two metal ions are called "hard" water.
Soap is an organic compound that has one or more acid groups attached to its structure. When you add soap to a solution containing ions one of two things can happen. If the solution contains group one metal ions, the soap dissolves and forms a solution.
However, if the solution contains group two metal ions, they combine with the soap molecules to form an insoluble precipitate. The soap comes out of solution. You may experience this when taking a bath and you see a ring around the tub. The ring is caused by the soap (plus the dirt the soap has removed from your body) combining with the calcium and magnesium and settling out on the sides of the tub.
Water softeners treat hard water by replacing the calcium and magnesium with sodium ions to form a "soft" water.