The reason for this is that lenders want to get back an amount of money that is at least as valuable as what they originally lent, plus something extra to compensate them for lending. If the interest rate is not high enough, the money that they get when the loan comes due will not be worth that much.
Think about it this way. If someone lends $100 at 5% interest for one year, they will get $105 at the end of the year. But what if the rate of inflation is 7%? To get money that's as valuable as the $100 that they lent, they would have to get back $107. If they only charge 5% and get back $105, they are actually losing value because they lent the money.
For this reason, the higher the rate of inflation goes, the higher interest rates go. If interest rates did not go up, lenders might actually lose money (in real terms) when they lend.
Interest rates are increased when inflation is higher as a measure to control it. Banks operating in the US are required by law to maintain a certain percentage of reserves with the Federal Reserve. When a bank needs more funds it can borrow it from other banks or from the Federal Reserve. By changing the rate at which it lends funds to banks, the Federal Reserve can influence the overall rate at which banks can get funds which they can then lend to their customers.
If the average interest rate at which banks can get funds to lend goes up so does the rate that they charge from their customers. This reduces the demand for loans as people know that they would have to pay a lot of interest on any amount that they borrow. Also, when the rate at which money is lent increases so does the interest rate that banks offer for deposits. This acts as an incentive for people to deposit more money.
The combined result of a reduction in borrowing and an increase in deposits reduces the amount of money that people have with them for spending. One of the reasons for an increase in inflation is driven by an increased demand. If the demand for the same goods is higher, producers can increase prices and still find customers willing to buy them. When the overall money available in the system goes down there is a reduction in demand. This leads to a reduction in inflation as producers are forced to reduce prices.
This is one of the primary reasons why interest rates are higher when inflation goes up.