Scanner data and quality adjustment

Insee has launched a pilot experiment which aims at introducing scanner data in the French CPI. Started in 2010 with a small set of companies and a small number of industrial food products, the experiment has now reached a larger scale with a daily transmission of data covering 30% of the market. This experiment gives Insee the occasion to review the quality adjustments in the French CPI. Thus, Insee has chosen a strategy of analysis that is mainly based on the same principle as the one applied for the rest of the French CPI: the sample is drawn yearly in the universe of the products in order to reach a certain level of accuracy in the resulting CPI; a two-steps computation is made : the first step consists in computing micro-aggregates while dealing with possible substitutions that occur at the micro-level by the use of adequate price index formulae and the second step consists in the traditional Laspeyres aggregation. The product is followed until it disappears. It is then replaced by a new product after a quality adjustment.

The paper deals with the quality adjustment applied in scanner data. Besides prices and quantities associated to EAN (barcode), each day, and each shop, Insee has bought a database containing descriptive variables of each sold EAN. This information makes it possible to choose in a proper way, replacement products based on a kind of distance between products. It also makes it possible to estimate in an objective way, quality differences with respect to descriptive variables. We compare, on a subset of 13 product families (food and manufactured goods), the results obtained through different techniques of quality adjustment and discuss the advantages and disadvantages of the various techniques with respect to the numerical differences we get. We find on yogurts, chocolate bars, soft cheese, toilet tissue and fruit juice families that quality adjustment is necessary since a quality-adjusted price index differs significantly from a non quality-adjusted index. Furthermore, all quality correction methods tested in the paper lead to statistically-identical price indices. Nevertheless, some systematic differences exist between classical methods of quality adjustment tested here. These differences are negligible as the accuracy required for the index is not too high (up to 20 times the level reached for an index based on traditional collection). But if the required accuracy is higher, then the differences between the quality adjustment methods may become problematic, even for food product as tested in this paper.