You really should look for books on this subject. It's one where it really is a good idea to consult people who are truly experts.
I'm not an expert -- just someone who has taken a lot of classes on Japan, read books about it, speak the language fairly well. But my opinion should not be substituted for that of a real expert.
My feeling is that the Japanese atrocities came from a number of sources, but the main two or three that I can think of are:
- Ancient rivalry with and conflicted feelings about China. The Japanse got much of their culture from China yet had surpassed China in many ways. So they felt superior yet they knew they owed a lot to the Chinese.
- The samurai/feudal ethos. This helped them believe that those who were not warriors (or didn't live up to the Japanese view of what warriors should be) were less deserving of respect and didn't need to be treated well. This showed up in their treatment of Allied POWs as well.
- Related to this is the militarism that was rampant in Japan at this time. This was the idea that the military was superior to all civilians and could, therefore, sort of do what it wanted.
One possible reason was the fact that the Nationalist forces had put up a strong defence against the Imperial Japanese Army. In response to the heavy casualties and tough fighting, the Japanese forces chose to take out their frustrations and anger on the local population. This brutal policy of sacking and pillaging Nanking, which had fought to defend itself, could thus be seen as one of retribution. Such atrocities could also have been committed to serve as a warning to force China to submit. Mass rape and deliberate sexual humiliation were used as weapons to break the Chinese will to fight. They would have been a calculated act of terror to intimidate China as a whole. It could perhaps have also been a ploy initiated in Japan to arouse Japanese public opinion as casualty numbers from the war had begun to rise and there appeared, in the eyes of the general public, to be no end to the fighting.