For Madison, a republic (and a large one in particular) was the way to keep the minority from overwhelming and abusing a minority. He thought of this in class terms, not racial terms. So the minority he was concerned about was the rich.
To some extent, I think he was right. Having representative government in a large country has helped to prevent the worst excesses of populism. We have not had any serious anti-wealth movements in the US and that may be in part due to the size and political organization of the country.
However, it is possible that the system we have today goes to far in the other direction. You can argue that the big corporations have taken over American government and that the rich are now in position to overwhelm and abuse the rest of us.
I read a book about Madison. Unfortunately, I don't remember the title.
The book said that Madison wanted to barrow money from a European bank to invest in land in Ohio, but Madison could not get the loan because the United States had no court system in which the bank could recover his debt if he defaulted.
Madison's objective: build a court system in the United States.
Madison's strategy: build a republic with a system of courts.
And the rest is history.