At the time the Constitution was drafted, women and slaves were considered possessions, not people in their own right, and Native Americans were considered only savages to be gotten out of the way, just as colonial powers all over the world considered indigenous peoples to be peoples to conquer, not to be equal. There were African-Americans who were not slaves, and at least theoretically, they would have been held to have the same rights as white males under the Constitution. There are instances of this in history, but sadly, not very many.
Historically, women in most western societies of that time had few if any rights. They were considered the possessions of their husbands, who could do as they pleased with them, including abuse and rape. Spousal rape was not a crime until the 1980s, for example, in many jurisdictions, which meant that it was permissible to rape one's wife, basically because she was a possession, not a separate human being. Women were generally not permitted to hold property in their own names, and any assets they brought to a marriage came under the husband's control. Certainly, the notion of women voting would have been alien to the founders of our nation.
The record is clear that African-American slaves were considered possessions, not people. They were listed as part of one's will, to be handed down in an estate as a form of property. When the southern states sought to increase the count of their state populations, it was agreed that their African-American slaves would count each count as a percentage of a person! In many southern states, it was illegal to teach a slave to read and write, still more evidence of their status as less than people, evidence that no one even contemplated that they would be capable of being citizens participating in a democracy.
It must be understood that if you want to conquer people, such as the Native Americans, and dominate them, you must be able to tell yourself they are not "really human." This form of denial has enabled people to conquer, enslave, and otherwise abuse others. It is so much easier to do if those whom you are decimating are not human beings at all. True equal rights under the Constitution for all were simply not part of the founding fathers' world views at that time and place. And as you can see, even now, there are those who are not completely convinced that various groups should be protected as equal.