Is it correct to say central banks should focus only on controlling inflation rather than on other irrelevant policies?
This depends greatly on what you mean by "other irrelevant policies." In general, the role of a central bank is to regulate the monetary policies of a country. In other words, the central bank is supposed to monitor the money supply that exists in a country at a given time and adjust things as necessary to ensure that the money supply is at the correct level.
This does, certainly, involve controlling inflation. However, inflation is not the only bad thing that can happen if the money supply is at an incorrect level. If the money supply is kept too low, for example, a recession can set in. In times of recession, it might makes sense to try to loosen the money supply to promote growth. This is the opposite of controlling inflation because inflation comes if growth is too rapid.
So inflation is not the only concern of a central bank. The bank has to keep the money supply at a level that prevents inflation, but it must also try to make sure that the money supply is high enough to allow growth.