There are a number of ways in which the federal government does this.
The most obvious way the government does this is through the FDIC. The FDIC protects our savings by insuring them. If we deposit money in a bank which then goes bankrupt, the government will ensure that we get our money back.
However, there is a broader way in which the government does this as well. The government does this by protecting (or trying to protect) the strength of our economy. Our savings and our financial security in general are protected mainly by the strength of our overall economy. The government protects this through such things as fiscal and monetary policy, but also through law enforcement and military power.
In these ways, the government can be said to do many things to protect our savings.