The answer to this is largely a matter of personal opinion. My own view is that the answer depends on whether you believe that IQ tests have any validity. If you tend to think that they have validity, you will tend to think that they do not lead to racism and ableism. If you think that they do not have validity, you will tend to think that they do lead to these evils.
If IQ tests really do measure things that are relevant to things like the ability to do well in school or in a job, then they cannot be racist or ableist. Let us say that we have a vocational rehabilitation program that admits only people with IQs above 85. If people with lower IQs are less likely to actually succeed in the jobs that the program is preparing its beneficiaries for, it is not ableist to exclude them. However, if people with lower IQs are just as capable of doing the jobs that the program is preparing people for, we cannot justify the use of the tests.
My own view is that IQ tests are not typically valid. They do not measure whether people will do well in school. They do not measure whether people will do well in a job. Therefore, I think that it is racist and ableist to use such tests to determine who gets jobs or who gets services.