The factors that most likely led Douglass to this conclusion were:

• The fact that he himself had been a slave.  Of course, people who were not slaves could believe that slavery was evil.  But Douglass had first hand experience of how evil slavery was.  This would lead him to believe that America should be punished for slavery.
• The fact that the Civil War was caused in large part by slavery.  An economic depression, for example, would have been less obvious proof of God's anger about slavery because it would not be clearly connected to slavery.  Since the Civil War was largely about slavery, it could be seen as a punishment for slavery.
• The fact that the Civil War was so terrible.  Slavery was a terrible thing.  Douglass might well have believed that it deserved a terrible punishment.

Since slavery was a great evil, it deserved (one can argue) a great punishment.  The Civil War was a great disaster that could logically be seen as a punishment for slavery.