Where to start the history books is in some sense arbitrary: No matter where you draw the line, there were probably important events before that, unless you start all the way back at the Big Bang, but that makes your history book an awful lot longer! (Though it is what Stephen Hawking famously tried to do with A Brief History of Time.)
But in general we usually try to start the history of a civilization at a point where that civilization begins to take on some of its distinctiveness. China didn't really become a unified nation until about the 3rd century BC in the Qin Dynasty, and has gone through many fundamental changes since then. But there were important precursors of Chinese civilization long before that, as far back as 2100 BC, and much of modern Chinese culture and language is owed to cultures from that period. People probably didn't even really think of themselves as "Chinese" (rather than local ethnicities such as Han and Zhuang) until even later, perhaps around the unification of China under the Song Dynasty in the 10th century AD. (Even today, people in Tibet and Taiwan often don't think of themselves as "Chinese".) But only by understanding these more ancient cultures can we shed light on the causes that make China what it is today.
On the other hand, so much has changed over time that it might even make sense to say that China didn't become what it is now until 1912 with the formation of the People's Republic of China, just over a century ago. Huge shifts in government and economic policy led to large changes in culture and religion. In this sense, Chinese history is quite "new" indeed.