Kemmerich is crying in the second chapter because he knows he is dying. After being wounded in the war, he is in a hospital bed whilst the narrator speaks to him and tries to cheer him up. However, it is clear from both the half-hearted attempts of the narrator to distract him and from the behaviour of those around them that Kemmerich is on his way out and it is only a matter of time before he does die. Note what the narrator says just before Kemmerich starts crying:
I do not reply. It is no use any more. No one can console him. I am wretched with helplessness. This forehead with its hollow temples, this mouth that now seems all teeth, this sharp nose! And the fat, weeping woman at home to whom I must write. If only the letter were sent off already!
When the narrator realises that Kemmerich his crying, he laments his "follish talk" and the "fine mess" that he has made of everything with trying to console him. The narrator, in spite of his fine intentions, is left speechless in the face of death. He is unable to say any words that will make what Kemmerich is facing any easier.