In paragraph 118 of A Treatise Concerning the Principles of Human Knowledge, Bishop Berkeley turns from the subject of natural science, which he has been discussing in the preceding paragraphs, to mathematics. He admits that mathematics is justly celebrated for its clarity and certainty but says that the discipline cannot be entirely free from error, if that error is one which mathematicians share with the rest of humanity. The premises of mathematics are unusually certain, but these premises relate to quantity and do not extend to any enquiry into higher or more abstract principles that influence all the sciences. If these higher principles contain errors, then the errors will affect every branch of science, including mathematics.
Berkeley says that the principles laid down by mathematicians are true and that the methods of deduction mathematicians apply to these principles are correct. However, there may be certain mistaken principles that apply more widely than to mathematics alone and which are not examined or questioned. Mathematicians, like anyone else, may be deceived by errors which arise from abstract ideas and the existence of objects outside the mind.
Berkeley is obviously correct in asserting that there may be errors in the principles applied by mathematicians (and everyone else) of which we are all unaware. The problem, to which Dr. Johnson famously objected, is that his criticism of abstractions is itself highly abstract. The objection to mathematical certainty is purely theoretical and speculative. One might easily go up to anyone involved in any activity and say: "You may be making a mistake of which neither of us is aware, due to our imperfect understanding of the higher principles which govern the activity in which you are engaged." It is unlikely, however, that they would derive much benefit from this warning.