The African-Americans who most affected Lincoln's policy toward slaves were actually not citizens. They were slaves themselves, who "voted with their feet," flocking toward Union lines in droves from the earliest days of the war. Because the Union had no real policy toward freed slaves, many were returned to their masters while others, like those who fled to General Benjamin Butler's lines, were designated contraband of war. Many volunteered for service, and were put to work performing labor in the Union war effort. The numbers were significant. By the time of the Emancipation Proclamation, over 10,000 slaves were estimated to have entered Union lines in North Carolina alone, with even larger numbers in South Carolina. As the flood of refugees continued, Congress passed a law forbidding Union generals to return escaped slaves. Ultimately, Lincoln became persuaded (in part through the efforts of abolitionists like Frederick Douglass) that emancipation, and allowing African-Americans to serve in the armed forces, was a matter of military necessity, and could be constitutionally justified as such.