In the poem, Whitman describes various people suffering horrible misery in different forms. However, as he describes them, he doesn't judge, get involved, make commentary, or pass some sort of overall moral or lesson to be learned. This is what he means when he says he is "silent". He simply opens a window for everyone to see what he sees, and lets the reader make their own interpretations and judgments. For example, he describes the misery that exists, "the ranklings of jealousy and unrequited love attempted to be hid," and doesn't comment on how awful that is, or what we should learn from it, or how we should change the misery; his next line is simply, "I see these sights on the earth". No moral imposition, no call to action. He is simply reporting what he sees. That is the silence he is referring to, a silence of judgment or analysis on the events.
Whitman, a great believer of individualism and trusting your own instincts to interpret the world, probably wanted to let the reader trust his or her own instincts, and to feel whatever they felt, without him telling them how to feel about it. This fits the theme of individualism well, a movement that Whitman was a part of.