In Chapter 1 of his autobiography, Frederick Douglass says that many people believed that his owner was also his father.  He says that this does not matter -- he would still be a slave.

He goes on to say that slaves who were the children of the master had a harder life.  He says that they were likely to get picked on by the master's wife because she would be mad at them as evidence that her husband was sleeping with the slaves.

He also says that such slaves often get sold because it is too hard for the master to deal with having them around.