Yes, employers do have this right. However, there are some states in which it is illegal to look into an applicant’s credit history except in the case of some sensitive jobs that require the handling of money. In addition, firms must generally ensure that the degree of intrusiveness is commensurate with the level of responsibility that the job entails.
The reason for this is twofold. First, employers have a significant interest in knowing the background of people they might employ. Small businesses, in particular, can be badly harmed by the actions of a bad employee. Therefore, it makes sense to allow such businesses to find out all they can about a person before they hire. Second, employers typically ask applicants to give consent to such background checks. So long as the applicant is informed and has the choice to opt out of the check (and not apply for the job), privacy issues are minimized.
In short then, employers generally have the right to conduct checks, particularly if they are commensurate to the responsibilities of the job and if the applicant gives consent.