Before the end of the Civil War and the establishment of the 13th-15th Amendments to the Constitution, African-Americans were viewed as property. For this reason, they were not afforded the civil liberties that all Americans enjoyed under the Bill of Rights. Since they were not considered citizens, or persons for that matter, slaves did not have legal rights. This is especially true of slaves in the late 17th and 18th Centuries when slave codes were more firmly established.
When you speak of a social contract, you are essentially discussing how you give permission to take away certain freedoms so that the government may protect important natural rights. In other words, while you forfeit your freedom to harm another person, or take his property, in return you are granted safety and security for the government. No such protections existed for slaves in the United States. Slaves could not own guns for protection and could not sue their masters in court for damages. Slaves were not even granted the freedom to move from place to place or legally marry. Because slaves were viewed as property, like a horse or mule, they were not participants in the social contract.