The idea here was that Africans were not of the same category of human being as whites were. This is much the same as how women and children were not given freedom and equality in the country.
The founders believed that all people were born with certain rights, but they also believed that people were born in a hierarchy. They believed that certain people, like women, children and blacks, were not as capable as white men were. Therefore, such people needed to be protected by the white men and treated in a paternalistic way.
So, the nation could be founded on the principles of equality and freedom, but it was understood that not all people were capable of being free and equal, just as we do not believe that children are capable of freedom and equality today.