The most important problem, in my opinion, is that the goods and services provided by a private contractor, as opposed to being provided by the government, are driven by a profit motive rather than by the motivation of giving taxpayers good value for their money. A recent example in Pennsylvania serves to show this. There are private "prisons" for delinquent youth here, prisons that gained hundreds more inmates when the corporation bribed judges into making sure there were plenty of teens sentenced to serve their time there. In this particular instance, injustices were created to boost the coffers of a private corporation. Other examples are less striking, but in general, when profit is the motive, the interests of taxpayers are not being served properly.
Perhaps the biggest problem with privatization is that it creates an incentive for the providers of services to "cut corners" when doing what they are paid to do.
When a government service is privatized, the providers are motivated by profit. This means that they will try as hard as they can to cut costs so as to make as much profit as possible. This can be good, but at times it can lead to the companies providing substandard services. Government agencies themselves are more focused on providing good service as they have no real pressure to save money (which is, of course, not always a good thing). Private companies focus on saving money, which can, at times, be a bad thing.