CSI Berlin: The Strange Case of the Death of Panels

Kees de Jong

Despite well documented shortcomings, online access panels remain the predominate sampling source for online market research projects. Online research, according to ESOMAR, is now the largest single methodology; by spend, at 20% of all revenues (ESOMAR Industry Report 2009). When one considers the relative cost per interview it must far and away be the largest single source of quantitative data in the world. The same Industry report for 2004 place online research spend at 9.2%% of global research revenues. Revenues have essentially doubled over five years.

As more and more "mainstream" clients and projects come online questions have been asked about the rigour, reliability, projectability and inter-changeability of panel sources. In many cases these have been found wanting. Online Access Panels are simply not random samples of any definable population but sources of convenience samples. The extent to which they are comparable to each other is, at least according to the ARF (Foundations of Quality research), dependent on recruitment sources used. In classical parlance they suffer from frame errors, and very substantial ones at that.