What are some ways that poor sampling can bias the results of a Nielson ratings study? What steps could be taken to minimize this bias?
The accuracy of the Nielsen ratings have long been disputed among television industry officials and critics. There are a number of criticisms of the Nielsen ratings system that have been voiced over the years, especially with respect to critically-acclaimed series that, according to the ratings system, have failed to develop an audience sufficient in size (and prosperity) to make them attractive targets of advertisers, the key indicator in whether a show remains on-the-air or is cancelled. Just within the past two years, network affiliate managers and corporate officials have questioned whether the sampling sizes are large and diverse enough to determine whether a television show attracts enough viewers.
The A.C. Nielsen Company, which has been the main source of consumer data on television show popularity, traditionally has boxes installed on the televisions of test homes, and/or requests families maintain logs tracking their viewing habits. There are weaknesses in the Nielsen ratings system, however, that may underestimate the popularity of some series. For starters, the evolution in television broadcasting and recording technologies, for example, the use of DVDs, DVRs and streaming services like Netflix, have greatly complicated the task of collecting data on viewing habits. Nielsen data do not reflect television shows that are viewed through such means, often in protracted sittings by busy individuals not able to watch the weekly broadcast but exploiting opportunities to catch up on their favorite shows through the viewing of many episodes back-to-back.
As mentioned above, the issue of adequate sampling size, along with underrepresentation within the ratings system of many groups, including lower-income families and colleges, can skew the data significantly. The Nielsen ratings system, which predated the invention of television and was used for consumer product surveys and radio ratings, was originally only concerned with the three main networks, NBC, CBS, and ABC, plus local stations, usually one per urban area. Today, there are more networks available to viewers than can be reliably tracked, plus the aforementioned alternatives of DVDs and streaming over one’s computer. Collecting and processing data on television viewing habits, consequently, has become a largely meaningless exercise. Ratings are used to gauge competition between shows airing at the same time. When people can record and watch shows later, deciding to view another show in its original broadcast, a ratings system is largely worthless.
If there is a good solution to the problem of inaccurate ratings, it has not yet surfaced. Certainly, the popularity of shows can be monitored through already compiled data on DVD rentals and purchases, but by then the show may have been cancelled due to low ratings. Nielsen can collaborate with Netflix and other streaming services in an effort at capturing more data, but the practical utility of such an exercise is questionable. Because extremely lucrative advertising revenue is at stake, the entertainment industry has much to gain by refining the ratings system. In order to do so, however, Nielsen and the networks would have to exploit the kind of data only found in a broad census of the population. Efforts at improving the accuracy of the system by focusing more on viewer “psychographics,” in effect, personal information on viewers’ lifestyles and interests, may bear fruit, but it is too soon to tell.