If people work hard and create or buy physical property such as houses or jewellery or cars or acquire money, they own that property in perpetuity, and can bequeath it to their heirs. We would be outraged if someone said that other people could have "fair use" of our homes or that our right to bequeath our family heirlooms would expire 70 years after our death. Yet, for the owners and creators of intellectual property, the notions of "fair use" and copyright expiration both limit the rights of creators to dispose of their own intellectual creations as they wish.
Imagine the situation of a novelist who spends three to five years writing a book. This is a huge investment of time and labor, equivalent to the labor of someone who spends the same number of years working in a corporate job. The value of the novel, like the value of anything else, is a function of the amount of labor going into it. Audience members have only the rights to a novel or movie or music that they specifically purchase. The idea that it is "fair" to make copies without the permission of the creator of the work is the same as saying that I should have "fair" use of someone else's house or money.
The negative side of weakening of copyright and advocacy of "fair use" is that it reduces the incentives of people to invest time in creating high quality work that audiences can enjoy. The less creators earn from their work, the faster they must produce it to earn a living, meaning the lower the quality. It is in everyone's interest to protect the rights of creators so that they can earn an adequate living from their work and continue to produce the works we all enjoy.