Göth cannot kiss Helen because she is a Jew and he is disgusted by (what he sees as) her racial impurity and inferiority to him. He is a sadistic cruel man who is impervious to human feeling. He is a man without empathy. The author describes him as “a sadist when drunk,” although it would seem that he did not need to be drunk to act sadistically. It was just his nature.
He has fully accepted the notion that the Jews in the prison camp are not human beings and that he can treat them as if they were animals. In fact, he treats them worse than he treats animals. He is much kinder to his dogs than he is to the Jews.
Helen occupies a special role in his twisted psyche. He sees how beautiful she is. He therefore enjoys torturing and tormenting her because in some twisted way, he is testing himself. If he can continue to put her Jewish identity above her physical allure, then he passes his self-test. In other words, if he can put her Jewishness above her physical beauty, he can respect himself as a Nazi. He cannot kiss her because that would be a slippery slope for him. It might make him see her as a woman instead of as a Jew. The author notes,
“Even though Amon enjoyed beating her, her Jewishness always saved her from overt sexual attack. But there were Germans who were not as fastidious on racial matters as Amon.”
He beats her because it satisfies him more than a sexual encounter would. It also fulfills a sadistic need he has to remind her that he holds her life in his hands and could beat her to death at a whim.
In fact, when she witnesses him arbitrarily shoot and kill a woman who was innocently walking by, her fear escalates. She notes that there was nothing special about his victim. She was not fatter or thinner than any of the other prisoners around. She was ordinary. There was no rhyme or reason why Göth killed her. He just did it for his own amusement.
Oskar convinces Helen that Göth enjoys torturing her too much to kill her. He tells her to hold on to that in order to survive until the war ends. Her description of the first beating she received from Göth makes it clear that he is cruel and evil:
"That was the first beating. I said to him... I don't know why I said it; I'd never say it now... Why are you beating me? He said, The reason I'm beating you now is you asked me why I'm beating you."