Whenever you are analysing a quote it is important to look at it in context. In this case, what is important is what has just happened prior to this quote. Elie and all the Jews have been forced to watch the hanging of three prisoners who had been in possession of arms. One of these is just a child. However, although the two adults die quickly, their necks broken by their weight, the child, because he is much lighter, does not die in the same way as he is slowly and excruciatingly suffocated:
And so he remained for more than half an hour, lingering between life and death, writhing before our eyes. And we were forced to look at him at close range. He was still alive when I passed him. His tongue was still red, his eyes not yet extinguished.
As Elie passes him, he hears a man asking, "For God's sake, where is God?" The answer that rises up from within Elie is the quote you have specified. It is important to realise that Elie is not speaking literally. Of course, God cannot be hanged. However, Elie chooses to make a symbol out of the dead boy to indicate the death of his faith. This is the crucial moment for Elie, the straw on the camel's back, that makes it impossible for him to believe in the existence of God in the face of such suffering.