The answer to this varies to some extent between Western and Japanese sources.
In the West, it is generally said that the Japanese "rejected" this demand. It is said that they did so because the Potsdam Declaration would not have guaranteed that the Emperor would continue to rule Japan. It would not have even guaranteed the personal safety of the Emperor, who could have been tried for war crimes. This was unacceptable to the Japanese.
In Japanese history books, the case is made that the Japanese did not actually reject the demand. These books emphasize that the word "reject" was one that came from Western translations of what the Japanese actually said. What the Japanese said can be translated in many ways. They used the word "mokusatsu" which is made up of two characters meaning "silence" and "kill." It can mean "ignore," but it can also mean "no comment." Japanese history books tend to argue that the official Japanese response was a "no comment" rather than a rejection.
Of course, this probably would have made no difference. The bottom line is that the Japanese were not ready to accept unconditional surrender.