Potential reasons for CPI chain drift bias while using electronic transaction data
Abstract
Scanner data mean electronic transaction data that specify product prices and their expenditures obtained from supermarkets’ IT systems by scanning bar codes (i.e. GTIN or SKU). Scanner data are a relatively new and cheap data source for the calculation of the Consumer Price Index (CPI) and the biggest advantage of scanner data is the full product information they provide already at the lowest level of aggregation. Thus, the digitization of the public sector becomes not only something that is needed but an actual necessity resulting from organisational and economic premises (e.g.: reduction of costs or time related to obtaining data). One of main challenges while using scanner data is the choice of the right price index. The list of potential price indices, which could be used in the scanner data case, is quite wide, i.e. bilateral and multilateral indices are used in practice. One of the most important criterion in selecting index formula for scanner data case is the potential reduction of the chain drift bias. The chain drift occurs if the index differs from unity when prices and quantities revert back to their base level. In the paper we present situations on the market leading to the serious chain drift bias. Our main hypothesis is that lagging consumers’ reaction to price changes is the cause of the chain drift effect. Moreover, the article is an attempt to answer the question whether the correlation of prices and quantities may have an influence on the scale and sign of the bias of the measurement of price dynamics. The study focuses also on the scale of over- and underestimation the target full-window multilateral indices by their corresponding splicing extensions. Finally, the paper verifies a hypothesis that the identity test is a key property in reducing chain drift bias. In order to verify the above research problems, both empirical and simulation studies were carried out. Our main result is the confirmation of earlier suspicions that delayed consumer response and price-quantity correlation are determinants of chain drift bias.
First published online 06 February 2023
Keyword : scanner data, electronic transaction data, digital transformation in public statistics, big data, Consumer Price Index, chain drift, chain indices, multilateral indices, splice indices, PriceIndices package
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Australian Bureau of Statistics. (2016). Making greater use of transactions data to compile the Consumer Price Index (Information Paper 6401.0.60.003).
Białek, J. (2020). Proposition of a hybrid price index formula for the Consumer Price Index measurement. Equilibrium. Quarterly Journal of Economics and Economic Policy, 15(4), 697–716. https://doi.org/10.24136/eq.2020.030
Białek, J. (2021a). PriceIndices: Calculating Bilateral and Multilateral Price Indexes. R package version 0.0.9.
Białek, J. (2021b). PriceIndices – a new R package for bilateral and multilateral price index calculations. Statistika – Statistics and Economy Journal, 36(2), 122–141.
Białek, J., & Beręsewicz, M. (2021). Scanner data in inflation measurement: From raw data to price indices. The Statistical Journal of the IAOS, 37(4), 1315–1336. https://doi.org/10.3233/SJI-210816
Białek, J. (2022a, June). The general class of multilateral indices and its two special cases. In 17th Meeting of the Ottawa Group on Price Indices. Rome, Italy.
Białek, J. (2022b). Improving quality of the scanner CPI: Proposition of new multilateral methods. Quality and Quantity. https://doi.org/10.1007/s11135-022-01506-6
Białek, J., & Bobel, A. (2019, May). Comparison of price index methods for CPI measurement using scanner data [Conference presentation]. 16th Meeting of the Ottawa Group on Price Indices. Rio de Janeiro, Brazil.
Białek, J., & Roszko-Wójtowicz, E. (2021). Dynamics of price level changes in the Visegrad group: Comparative study. Quality and Quantity, 55, 357–384. https://doi.org/10.1007/s11135-020-01021-6
Caves, D. W., Christensen, L. R., & Diewert, W. E. (1982). Multilateral comparisons of output, input, and productivity using superlative index numbers. Economic Journal, 92(365), 73–86. https://doi.org/10.2307/2232257
Chen, T., & Guestrin, C. E. (2016). XGboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). San Francisco, CA, USA, ACM. https://doi.org/10.1145/2939672.2939785
Chessa, A. (2015). Towards a generic price index method for scanner data in the Dutch CPI. In 14th Meeting of the Ottawa Group (pp. 20–22). Tokyo.
Chessa, A. (2016). A new methodology for processing scanner data in the Dutch CPI. Eurostat review of National Accounts and Macroeconomic Indicators, 1, 49–69.
Chessa, A. (2019, May). A comparison of index extension methods for multilateral methods. In 16th Meeting of the Ottawa Group on Price Indices. Rio de Janeiro, Brazil.
Chessa, A., & Griffioen, R. (2016). Comparing scanner data and web scraped data for consumer price indices (Report). Statistics Netherlands.
Chessa, A. G., Verburg, J., & Willenborg, L. (2017). A comparison of price index methods for scanner data. In 15th Meeting of the Ottawa Group (pp. 10–12). Eltville am Rhein.
de Haan, J. (2008). Reducing drift in chained superlative price indexes for highly disaggregated data. In The Economic Measurement Workshop. Centre for Applied Economic Research, University of New South Wales.
de Haan, J. (2015). A framework for large scale use of scanner data in the Dutch CPI. In Report from Ottawa Group 14th Meeting, International Working Group on Price Indices (pp. 1–43). Ottawa Group.
de Haan, J., & Krsinich, F. (2018). Time dummy hedonic and quality-adjusted unit value indexes: Do they really differ? Review of Income and Wealth, 64(4), 757–776. https://doi.org/10.1111/roiw.12304
de Haan, J., & van der Grient, H. A. (2011). Eliminating chain drift in price indexes based on scanner data. Journal of Econometrics, 161(1), 36–46. https://doi.org/10.1016/j.jeconom.2010.09.004
de Haan, J., Hendriks, R., & Scholz, M. (2021). Price measurement using scanner data: Time-product dummy versus time dummy hedonic indexes. Review of Income and Wealth, 67(2), 394–417. https://doi.org/10.1111/roiw.12468
Diewert, W. E., & Fox, K. J. (2018). Substitution bias in multilateral methods for CPI construction using scanner data (UNSW Business School Research Pape No. 2018-13). https://doi.org/10.2139/ssrn.3276457
Dijmărescu, I., Iatagan, M., Hurloiu, I., Geamănu, M., Rusescu, C., & Dijmărescu, A. (2022). Neuromanagement decision making in facial recognition biometric authentication as a mobile payment technology in retail, restaurant, and hotel business models. Oeconomia Copernicana, 13(1), 225–250. https://doi.org/10.24136/oc.2022.007
Divisia, F. (1925). L’indice montaire et la theorie de la monnaie. Revue d’Economique Politique, 40(1), 49–81.
Eltetö, O., & Köves, P. (1964). On a problem of index number computation relating to international comparison. Statisztikai Szemle, 42(10), 507–518.
Eurostat. (2018). Practical guide for processing supermarket scanner data. Harmonised Index of Consumer Prices.
Eurostat. (2020). Practical guide on multilateral methods in the HICP (draft). European Commission. Version September 2020.
Feenstra, R., & Shapiro, M. (2003). High-frequency substitution and the measurement of price indexes. Studies in Income and Wealth, 64, 123–146.
Fisher, I. (1922). The making of index numbers: A study of their varieties, tests, and reliability. Houghton Mifflin.
Geary, R. C. (1958). A note on the comparison of exchange rates and purchasing power between countries. Journal of the Royal Statistical Society. Series A (General), 121(1), 97–99. https://doi.org/10.2307/2342991
Gini, C. (1931). On the circular test of index numbers. Metron, 9(9), 3–24.
Griffioen, A. R., & Ten Bosch, O. (2016, May). On the use of internet data for the Dutch CPI. In The UNECE-ILO Meeting of the Group of Experts on Consumer Price Indices. Geneva, Switzerland.
Guerreiro, V., Walzer, M., & Lamboray, C. (2018). The use of supermarket scanner data in the Luxembourg consumer price index. Economie et Statistiques, 97, 1–17.
Inklaar, R., & Diewert, W. E. (2016). Measuring industry productivity and cross-country convergence. Journal of Econometrics, 191(2), 426–433. https://doi.org/10.1016/j.jeconom.2015.12.013
International Labour Office. (2004). Consumer price index manual: Theory and practice. Geneva.
Ivancic, L., Diewert, W. E., & Fox, K. J. (2011). Scanner data, time aggregation and the construction of price indexes. Journal of Econometrics, 161(1), 24–35. https://doi.org/10.1016/j.jeconom.2010.09.003
Jaro, M. (1989). Advances in record-linkage methodology as applied to matching the 1985 census of Tampa, Florida. Journal of the American Statistical Association, 84(406), 414–420. https://doi.org/10.1080/01621459.1989.10478785
Jevons, W. S. (1865). On the variation of prices and the value of the currency since 1782. Journal of the Statistical Society of London, 28(2), 294–320. https://doi.org/10.2307/2338419
Khamis, S. H. (1972). A new system of index numbers for national and international purposes. Journal of the Royal Statistical Society: Series A (General), 135(1), 96–121. https://doi.org/10.2307/2345041
Krsinich, F. (2014). The FEWS index: Fixed effects with a window splice. Non-revisable quality adjusted price indexes with no characteristic information. In Meeting of the Group of Experts on Consumer Price Indices (pp. 26–28). Geneva, Switzerland.
Lamboray, C. (2017). The Geary Khamis index and the Lehr index: How much do they differ. In 15th Meeting of the Ottawa Group (pp. 10–12).
Laspeyres, K. (1871). IX. Die Berechnung einer mittleren Waarenpreissteigerung. Jahrbücher für Nationalökonomie und Statistik, 16(1), 296–318. https://doi.org/10.1515/jbnst-1871-0124
Małkowska, A., Urbaniec, M., & Kosała, M. (2021). The impact of digital transformation on European countries: Insights from a comparative analysis. Equilibrium. Quarterly Journal of Economics and Economic Policy, 16(2), 325–355. https://doi.org/10.24136/eq.2021.012
Paasche, H. (1874). Über die Preisentwicklung der letzten Jahre nach den Hamburger Börsennotirungen. Jahrbücher für Nationalökonomie und Statistik, 23(2–4), 168–178.
R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria.
Roszko-Wójtowicz, E., & Grzelak, M. M. (2020). Macroeconomic stability and the level of competitiveness in EU member states: A comparative dynamic approach. Oeconomia Copernicana, 11(4), 657–688. https://doi.org/10.24136/oc.2020.027
Törnqvist, L. (1936). The bank of Finland’s consumption price index. Bank of Finland Monthly Bulletin, 16(10), 1–8.
van der Grient, H., & de Haan, J. (2010, May). The use of supermarket scanner data in the Dutch CPI. In Joint ECE/ILO Workshop on Scanner Data. Geneva.
van Loon, K., & Roels, D. (2018, May). Integrating big data in the Belgian CPI. In Meeting of the Group of Experts on Consumer Price Indices. Geneva, Switzerland.
von Auer, L. (2019, May). The nature of chain drift. In 17th Meeting of the Ottawa Group on Price Indices. Rio de Janerio, Brasil.
von der Lippe, P. (2007). Index theory and price statistics. Peter Lang. https://doi.org/10.3726/978-3-653-01120-3
Zhang, L.-C., Johansen, I., & Nyagaard, R. (2019). Tests for price indices in a dynamic item universe. Journal of Official Statistics, 35(3), 683–697. https://doi.org/10.2478/jos-2019-0028