Soap is usually made from one or more fatty acids. Fatty acids are relatively long chain organic molecules that have both a hydrophobic (water repelling/fat loving) part and a hydrophillic (water loving/fat repelling) part which is the carboxylic acid end group. When the acid group is neutralized by combination with a group one metal ion, the resulting ion will dissolve in water.
The way that the soap works is that the hydrophilic end will interact with the oils and greases that you are trying to remove from your hands/clothes/etc. and dissolve them. At the same time the hydrophilic end makes the molecule water soluble so when you rinse your hands the soap molecule and the dirt are washed down the drain.
Soaps when added to water also reduce the surface tension of water, allowing the formation of the lather that you referred to. There is a certain minimum amount of soap - the critical micelle concentration - that is necessary for the lather to form. That is why if you use soap with really dirty dishes the initial lather disappears, even in soft water.
Now, in hard water there are relatively large amounts of calcium and magnesium ions. Each of these ions can react with two soap molecules, forming an insoluble precipitate. Since the soap has been effectively removed from solution there is not enough molecules left to form the soap lather. The common way hard water is noticed is the ring that is left around the bathtub when bathing in water that is hard.