What a great question! There are so many reasons to study art from civilizations which now only exist in history books. Here are a few to consider.
By studying a society's art, it is possible to determine what a society found valuable. Were they religiously focused? Did they place great value on personal appearance? Did the people of that society strive for a minimalist impact on the world around them? So many questions about how people lived and what they held in high regard can be answered through studying the art a society produced. Since art existed even before efforts to formally educate societies, it can be studied to realize the impact people had on their worlds.
Studying a society's artwork can help you see how various aspects of any particular society worked together to create its unique place in history. Through art, you can draw conclusions about a society's culture, politics, economics, and religious values. You can then evaluate how the society changed over time and how these changes led to different social and political periods.
In a more straightforward way, artwork is the only way for us to have a visual representation of the way people looked long ago. This is sometimes understated in our modern society, where everyone walks around with a camera in hand nearly all day long. But the camera has only been around for a little more than a century; all physical representations which came before this time only exist through the artwork.