This is to some extent a matter of opinion and definitely differs depending on who you are and what you do. For most Americans, the Patriot Act has had no tangible impacts. However, groups like the ACLU believe that it could have severe impacts on civil liberties.
In theory, the Patriot Act could give the government a lot more of an ability to spy on people simply by arguing that those people are or could be terrorists. There are some indications that provisions have been used to conduct "fishing expeditions" among groups of people who are suspected of terrorism based only on their religion or ethnicity.
However, there have not been any indications that the Patriot Act is having an impact on the lives of the typical American person. We do not (so far as we know) have the government tapping our phones or looking at what we borrow from the library on a regular basis. We have not heard of cases of people being arrested for non-terrorist crimes on the basis of Patriot Act powers.
Therefore, I would argue, the Patriot Act has had little in the way of real impacts on the civil liberties of the average American.