Retirement Security: BLS Should Explore Ways to Improve the Accuracy, Timeliness, and Relevance of Its Cost-of-Living Measurements

In the United States, federal retirement programs typically include cost-of-living adjustments based on a CPI that measures inflation for a subpopulation of workers. This includes Social Security, which provides benefits for more than 60 million older Americans, workers with disabilities, and their families. As the life expectancy of Americans continues to increase, more Americans will be subject to these adjustments, so it is critical for them to be accurate.

GAO was asked to review U.S. and international efforts to measure the cost of living for older populations. This report examines (1) key issues that BLS faces in measuring the cost of living for older Americans; and (2) the experiences of other countries that developed alternate methods of adjusting retirement benefits. GAO reviewed pertinent literature; assessed BLS efforts to measure inflation; conducted case studies in three countries—Australia, New Zealand, and the U.K.—with a variety of CPIs, which GAO selected based on expert referral and document review; and interviewed agency officials and experts.

The U.S. Bureau of Labor Statistics (BLS) faces accuracy, timeliness, and relevancy challenges developing consumer price indexes (CPI) for subpopulations of blue-collar workers and older Americans. For example, the CPI for these workers is used to adjust federal retirement benefits for inflation, including Social Security. BLS has not evaluated the extent to which its existing data are adequate to produce CPIs that reflect what these subpopulations pay, where they shop, and what they purchase. Officials cite budgetary reasons for not having done this, but there may be cost-efficient methods for evaluating the adequacy of these data. Without an evaluation, federal retirement benefits could be subject to adjustment based on potentially inaccurate information. Additionally, BLS has made limited use of certain data already collected by the federal government—such as National Accounts data on U.S. production and consumption—that could be used to increase the accuracy, timeliness, and relevancy of CPI calculations that reflect the mix of goods and services consumers purchase. Without adequately exploring the potential of using these data, BLS may be missing an opportunity to improve its CPIs.

Reports about the retirement systems in the 36 Organisation for Economic Co-operation and Development countries indicate that most use their primary measures of inflation to adjust government retirement benefits. In addition, all three of GAO's case study countries (Australia, New Zealand, and the United Kingdom, or U.K.) have a variety of CPIs, including for subpopulations, and they filled information gaps in their CPIs with National Accounts and other data. For example, Australia and the U.K. use National Accounts data annually to update their calculations of the mix of goods and services consumers buy, thereby making the CPIs more relevant and accurate. All three countries also collaborated with stakeholders—such as other agencies—to implement changes, for example by gathering input on the design of subpopulation CPIs.

GAO recommends that BLS explore cost-efficient ways to evaluate the data currently used to produce subpopulation indexes, and explore the use of National Accounts data to produce more accurate, timely, and relevant CPIs. BLS agreed with the first recommendation but disagreed with the other. GAO continues to believe both recommendations are warranted, as discussed in the report.

In the United States, federal retirement programs often use consumer  price indexes (CPI) to calculate cost-of-living adjustments (COLA) for  retirement benefits to ensure these benefits keep pace with inflation.  These programs include the Federal Employee Retirement System, as  well as Social Security, one of the largest federal programs that paid  benefits to more than 60 million older Americans, workers with disabilities,  and their families as of the end of 2019. Federal retirement programs in  the United States generally use the CPI-W, an index that measures  inflation for a subpopulation of working Americans. CPI-W is one of  several price indexes produced by the Department of Labor’s Bureau of  Labor Statistics (BLS). Although one of BLS’s goals is to create accurate  inflation measurements, some economists have argued that the CPI-W  may overestimate the cost of living in general, while other economists  have argued that it may underestimate the cost of living for older  Americans. A number of legislative proposals have sought to change the  index used for Social Security and other federal retirement benefits. We  previously reported how even a fraction of a percent difference in the  index used to adjust retirement benefits can accumulate over time,  resulting in tangible monthly differences for individual beneficiaries.

As Americans’ life expectancy continues to increase, more retirees are  exposed to the effects of COLAs over longer periods. Similar to the  United States, a number of other countries with aging populations use  CPIs to adjust benefits from their national pension systems. Some  countries have changed to an alternate index to reduce benefits and  improve the financial sustainability of their pension systems, while others  have changed to increase the purchasing power of pension benefits. We  recognize that each country has a unique set of circumstances, including  varying national pension programs and methods for producing national  statistics. Still, the experiences of other countries that have created  alternate indexes to increase understanding about how inflation may vary  by age or income level, or that switched the index to adjust retirement  benefits to ensure these benefits take into account the costs faced by  retirees, may be relevant to the United States.

Given the central role of CPIs in determining the level of federal  retirement payments that may be paid over multiple decades, you asked  us to review the experiences of the United States and other countries in  measuring the cost of living for older populations. This report (1)  examines the challenges BLS faces in measuring the cost of living for  older Americans and actions BLS is taking to address the challenges; and  (2) describes the experiences of other countries that developed alternate  methods of adjusting retirement benefits.

To examine the challenges BLS faces in measuring the cost of living for  older populations and steps BLS has taken to address the challenges, we  interviewed a range of agency officials and other stakeholders and  reviewed pertinent literature:

We interviewed officials from BLS and reviewed agency policies and  federal laws and regulations related to the production of CPIs and  their use in adjusting federal retirement benefits. We also interviewed  a nongeneralizable selection of nine subject-matter experts, including  academic researchers and members of committees that advise BLS  on technical matters. We identified subject-matter experts in pertinent  literature or by referral from other interviewees. Finally, we  interviewed officials from two agencies that use CPIs: the Social  Security Administration (SSA), which uses them to calculate COLAs  for retirement benefits; and the Bureau of Economic Analysis (BEA)  within in the Department of Commerce, which uses CPIs to produce  other national statistics and which also produces a similar measure of  consumer inflation called the Personal Consumption Expenditures  price index.

We also conducted a literature review of research and policy studies  using two data collection methods. First, we conducted a formal  search of multiple databases for studies on the measurement of  inflation for older Americans published from January 2009 to June  2019. From 88 studies, we identified 13 that described or analyzed  issues in producing CPIs for measuring the inflation faced by older  Americans. Second, we identified two additional studies that were  authored or recommended by the subject-matter experts we  interviewed, for a total of 15 studies.

We assessed BLS efforts to measure inflation for older Americans against  (1) standards for internal control that call for federal agencies to develop  statistics using relevant data obtained in a timely manner that faithfully  represent what they purport to represent; and (2) BLS operational and  performance goals to produce accurate products and improve the  timeliness and relevance of its information on price changes in the  economy.

To describe the experiences of other countries that developed alternate  methods of adjusting retirement benefits, we conducted case studies of  Australia, New Zealand, and the United Kingdom. To select these  countries, we interviewed BLS and SSA officials and representatives of  five international organizations that play varying roles with respect to CPIs  and retirement benefits. We reviewed publications by the Organisation for  Economic Co-operation and Development (OECD) and SSA describing  the national pension systems of various countries. We also reviewed  documents from the agencies responsible for producing CPIs and  administering retirement benefits in some of these countries. Based on  our review, we selected countries with a variety of CPIs that all adjusted  national pension benefits based, at least in part, on a CPI, similar to  Social Security retirement benefits in the United States.

We also selected these countries, in part, because they are similar to the  United States with respect to retirement. For example, we selected  countries with similar proportions of people age 65 or older and with a  similar eligibility age for full national pension benefits. Because older  Americans generally devote a substantially larger share of their total  budgets to medical care than the broader population, we also selected  one country with comparable out-of-pocket expenditures on medical  care. We recognize that each country has its own methods for producing  national statistics such as CPIs, as well as retirement systems designed  to reflect that country’s unique historical and political experience. As a  result, information from the case studies is not generalizable. However,  the case studies provide examples of approaches countries took to  address issues similar to those facing the United States in measuring the  cost of living for their retired populations.

We conducted this performance audit from February 2019 to June 2020  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

BLS currently produces a number of different price indexes to estimate  price inflation (see table 1). In line with its strategic plan, BLS aims to  make these estimates as accurate as possible, meaning that they reflect  the average level of price inflation for a selected group of consumers. The  accuracy of a price index can be assessed in multiple ways, such as the  extent to which the index applies appropriate formulas to data that are  complete and drawn from sufficiently large samples covering the relevant  group of people. BLS bases its collection of these data on the population  covered by the Consumer Price Index for All Urban Consumers (CPI-U).  BLS then uses data collected for the CPI-U to produce three other price  indexes. After introducing the CPI-U as its primary, or headline, index,  BLS maintained a separate data collection for the CPI-W from 1978  through 1980 but found little difference between data for CPI-W and CPI- U. According to BLS, as a result of this and budgetary issues, BLS  stopped collecting separate data for the CPI-W in 1981 and began using  CPI-U data to derive the CPI-W.

To create the CPI-U, BLS chooses a sample of outlets (e.g., stores or  internet sites) at which the CPI-U population shops (see fig. 1 for more  information on how BLS creates price indexes). BLS then collects price  data at these outlets for goods and services the CPI-U population buys  and uses the data to develop basic, or elementary, indexes for each good  and service.

BLS combines the elementary indexes into a single, aggregated index by  applying a set of expenditure weights—factors that determine, for  example, whether a change in the price of apples or mobile phone service  has a larger effect on total inflation (see fig. 2). These expenditure  weights reflect the proportion of spending consumers direct to each good  or service. To develop expenditure weights, BLS directs the Census  Bureau to gather data about the relative importance of each purchase  within the target population’s “market basket” of consumer goods and  services. The Census Bureau collects these data in the Consumer  Expenditure Survey, a nationwide household survey conducted by BLS to  determine how consumers spend their money that also contains  demographic data about the households surveyed. BLS uses 2 years of  Consumer Expenditure Survey data to calculate the expenditure weights,  in part so the sample sizes are large enough to produce accurate  weights.

From data collected to produce the CPI-U, BLS derives two  subpopulation indexes—indexes that focus on the spending patterns of a  portion of the population of all urban consumers: the CPI-W and the CPI- E. To produce these subpopulation indexes, BLS adjusts the relative  importance of price changes in each good and service through a process  sometimes referred to as “reweighting,” meaning BLS develops alternate  sets of expenditure weights that reflect the spending patterns of the  subpopulation. For example, since medical care comprises more of the  CPI-E subpopulation’s total expenditures (about 12 percent) than of the  CPI-U population’s total expenditures (about 9 percent), the CPI-E gives  more weight to medical care than the CPI-U.

BLS also creates a “chained” index using the same data for the entire  CPI-U population but changing the formula used to combine indexes for  each good and service, known as elementary or basic indexes, into a  single aggregated index. This formula captures how consumers shift  spending among different types of goods and services as prices change  (see text box). In contrast, the other indexes assume that consumers  keep purchasing various categories of goods and services in the same  proportions over a 2-year period regardless of price changes.

A chained price index uses a formula that is believed by some  economists to better approximate a cost-of-living index by more  accurately accounting for changes in consumption patterns in response  to relative price changes. They contend that such a formula reduces the  potential for overstating inflation relative to the other indexes BLS  produces, which assume consumers keep buying goods and services in  the same proportions no matter their price. Like the other three indexes  BLS produces, the Chained CPI-U reflects consumers’ ability to adapt to  changing prices by choosing among closely related goods and services  as prices change, for example purchasing a different type of apple  because it is on sale. However, unlike the other three indexes, the  Chained CPI-U further reflects consumers’ ability to choose among all  available goods and services as prices change, such as taking a train to  work instead of driving when the price of gasoline rises, and purchasing  headphones to listen to music during the commute.

We previously reported that, were federal retirement benefits to be  indexed to the Chained CPI-U, SSA and other agencies would need to  determine whether to base retirement COLAs on final data that may be  outdated or preliminary data that may be inaccurate. This is because  the data needed to use a superlative index formula only become  available after a significant time lag. This lag delays issuance of final  monthly estimates for the Chained CPI-U by up to 1 year. Additionally,  the chair of a panel convened at the request of BLS to examine issues  in measuring the cost of living cautioned that chained indexes may not  accurately reflect the way people with varying incomes substitute goods  and services. For example, retirees with lower incomes might not have  the same ability as retirees with higher incomes to substitute other  goods and services when the prices of needed medical care or  prescription drugs rise.

BLS receives input on its processes from several sources. For example,  BLS receives advice and recommendations from several advisory  committees that variously focus on technical issues and the needs of  users of BLS statistics. BLS also periodically receives input on its price  indexes through external commissions and panels. For example, in May  1995, the U.S. Senate created the Advisory Commission to Study the  Consumer Price Index, commonly referred to as the “Boskin  Commission,” after its chairman, Michael J. Boskin. In December 1996,  the Boskin Commission released its final report identifying sources of bias  in the production of CPIs that the commission concluded were causing  the indexes to overstate inflation. BLS also receives input on its price  indexes through public comment. For example, in May 2019, the Office of  Management and Budget issued a request for public comments on the  various price indexes produced by BLS and BEA.

While there are a number of federal retirement benefit programs, Social  Security is by far the largest provider of indexed retirement and disability  benefits in the United States, paying out over $1,047 billion in retirement  and disability benefits in 2019. Social Security was established in 1935 to  provide for the general welfare of older Americans by, among other  things, establishing a system of federal old-age benefits, including a  retirement program. To determine a worker’s initial retirement benefit,  Social Security indexes the worker’s earnings to an average wage  index. According to SSA, this ensures that a worker’s future benefit  reflects the general rise in the standard of living that occurred during his  or her working lifetime. Since 1975, Social Security has also indexed  retirement benefits after the initial benefit level has been set to a CPI.  According to SSA, this ensures that benefits are not eroded by inflation  over time. When SSA began indexing benefits, CPI-W was the only  national CPI available, and SSA continues to use the CPI-W to determine  COLAs.

As we have previously reported, the Social Security program faces  financial difficulties that, if not addressed, will affect its long-term  stability. In April 2020, SSA projected that Social Security’s retirement  program trust fund will be unable to pay full benefits in 2034. We have  also reported that, according to projections by SSA and the Congressional Budget Office, use of an alternate index to determine  COLAs would have less effect on Social Security’s long-range finances  than some other options for addressing the program’s finances, such as  changing the taxation of earnings or raising the retirement age. That  said, we found that, according to SSA projections, using an alternate CPI  to calculate COLAs would affect Social Security’s finances in different  ways. Specifically, using the CPI-E would increase expected COLAs and  thus program costs and using the Chained CPI-U would decrease  expected COLAs and thus program costs, while using the CPI-U would  result in little change to either.

National Income and Product Accounts (National Accounts)

Produced by BEA, the National Accounts are a set of statistics on U.S.  production, income, consumption, investment, and saving. Among these  are Gross Domestic Product, a measure of the goods, services, and  structures produced across the economy, and the Personal Consumption  Expenditures index, a measure of consumer inflation similar to CPIs, but  constructed using different methods and data sources and covering  different populations and transactions. Data collected by BEA to produce  the National Accounts differ in a number of ways from those collected by  BLS to produce CPIs. For example, while CPIs focus on the expenditures  of households in urban areas, the National Accounts also include  expenditures on institutional populations, such as individuals living in  nursing homes. Further, while CPI expenditure data are based on the  recollection of consumers, National Accounts expenditure data primarily  reflect the records of the businesses that serve consumers. In other  words, to collect data on the quantity of goods and services consumed,  BLS surveys consumers about how much they bought, whereas BEA  surveys companies about how much they sold.

The National Accounts are produced primarily from data collected by  federal government agencies. These data include both “statistical” data  collected from federal statistical agencies, such as the Census Bureau, as  well as “administrative” data collected by federal agencies as a byproduct  of administering their programs. For example, BEA uses sample data  generated by the Internal Revenue Service in processing tax returns to  estimate corporate profits. BEA supplements these statistical and  administrative data collected by federal agencies with data obtained from  trade associations, businesses, international organizations, and other  sources.

BLS Faces Challenges Developing Consumer Price Indexes, but Has Made Limited Use of Data Collected by the Federal Government That May Help It Improve the Indexes’ Accuracy and Timeliness

BLS Faces Challenges Related to the Accuracy and Timeliness of CPIs, Among Others

BLS faces a number of challenges related to the accuracy and timeliness  of CPIs, as well as challenges related to measuring inflation for older  Americans. Some of these challenges may have implications for federal  retirement benefit adjustments.

According to BLS officials and documentation, BLS is unsure if the data  sources it uses to produce the CPI-U are adequate to produce accurate  subpopulation estimates—specifically, the CPI-E and CPI-W. For the  CPI-E, BLS has not evaluated the adequacy of the CPI-U data it uses to  measure inflation for the 62-and-older subpopulation. Specifically, BLS  has not evaluated the extent to which CPI-U data represent the outlets  where members of this older subpopulation shop, the prices they pay, or  the mix of goods and services they purchase. BLS considers the CPI-E  an experimental index, in part, because of the relatively small sample size  within the Consumer Expenditure Survey used to create the expenditure  weights for this subpopulation, which account for the mix of goods and  services the subpopulation purchases. According to BLS documentation,  the expenditure weights for the CPI-U rely on about 65,000 household  interviews, which are collected quarterly over 2 years. In contrast, the  expenditure weights for subpopulation indexes use about one-third or less  of that: 21,000 interviews for the CPI-E and 16,000 for the CPI-W. For  the CPI-W, BLS has not evaluated the adequacy of using CPI-U data  since 1980, but the relative sample size used to calculate the expenditure  weights for the CPI-W subpopulation has been shrinking in part because  of declining response rates and demographic shifts away from the  occupations included in the CPI-W. For example, occupations in the CPI- W include blue-collar jobs such as clerical, sales, laborer, and  construction jobs. BLS officials and documentation indicate that as a  result of these demographic shifts and the subsequent shrinking sample  size within the Consumer Expenditure Survey, the accuracy of the CPI-W  expenditure weights may be deteriorating.

A core element of BLS’s mission is to provide accurate products.  Moreover, standards of internal control call for agencies to obtain relevant  data from reliable internal and external sources to meet information  requirements for meeting their objectives. For BLS, this could include  obtaining relevant data from reliable sources for producing CPIs. BLS  officials said they have not evaluated the adequacy of the existing data  because it is costly to undertake a full evaluation, but there may be cost- efficient ways to do so. BLS also has not evaluated different methods to  conduct a cost-efficient analysis. Without taking actions to understand  available options for a cost-efficient solution, BLS lacks reasonable  assurance that adjustments to Social Security and other retirement  benefits are based on indexes that reflect what they are intended to  reflect. Specifically, benefits could be subject to adjustment based on  potentially inaccurate information.

Most experts we interviewed identified potentially cost-efficient methods  to evaluate the adequacy of existing data for subpopulation indexes. For  example, five experts we interviewed, including some on BLS advisory  groups, suggested that BLS may be able to use existing data to examine  the adequacy of using Consumer Expenditure Survey data for the CPI-E.  Specifically, one expert suggested that BLS could compare expenditure  patterns for the older subpopulation in the Consumer Expenditure Survey  to those in third-party data. Another expert added that the overall prices  older Americans pay may not be significantly different than the prices the  general population pays. For example, gas stations generally charge the  same price to each customer regardless of age, so this expert said that it  may not be worthwhile for BLS to collect separate price data for older  Americans. Another expert indicated that, while it might not be possible to  link expenditures and demographics (such as age) for all CPI categories  using third-party data, it may be possible for certain categories such as  groceries, which are a sizeable portion of the older population’s  expenditures. Another suggested that to improve subpopulation indexes,  BLS could shift resources from cost savings realized from other ongoing  projects.

BLS officials acknowledged some potentially cost-efficient methods could  exist to evaluate the adequacy of existing data for subpopulation indexes.  For example, they said that a recent change in survey methodology will  enable them to connect demographic information with information on  where people shop beginning in 2019. The ability to make this connection  should allow them to determine whether certain subpopulations shop at  the same or different outlets and could help them determine the adequacy  of their outlet sample selection. According to agency officials, BLS  advisory groups could weigh in on such issues, but BLS has not asked  the advisory groups to do so nor do the advisory groups have any recent  or ongoing research on indexes for subpopulations such as older  Americans. BLS officials added that obtaining transaction and  demographic data from credit card companies could help, but cautioned  that companies may be unwilling to share these data.

BLS is currently undertaking a project to improve how it estimates its  subpopulation indexes, CPI-E and CPI-W, in part by examining changes  to the formulas used to apply expenditure weights. As part of its  justification for the project, BLS expressed concerns about the decrease  in the relative sample size for the CPI-W population in the Consumer  Expenditure Survey and reiterated the importance of the CPI-W in  adjusting federal retirement benefits. This project is a step in the right  direction but does not fully address the question of whether the CPI-U  data are adequate to produce CPI-W and CPI-E.

In 2009, BLS began another project to address measurement error in and  households’ willingness to respond to the Consumer Expenditure Survey,  which is primarily conducted to create expenditure weights for CPIs.  According to agency documents, the survey faces increasing costs and  declining response rates. One particular goal of the project is to reduce  error due to underreporting. For example, BLS is currently testing  replacing a paper record of household expenditures with an online form  with the goal of more accurately capturing expenditures and maintaining  response rates. The project is ongoing and BLS expects to implement  changes in stages through and beyond 2022. According to agency  officials, the project was not designed to address subpopulation indexes,  but instead was designed to address broader issues with the accuracy of  the Consumer Expenditure Survey.

BLS also faces challenges regarding the timeliness and relevance of  CPIs. In particular, most CPIs are published using expenditure data that  can be up to 4 years old, and, in this dynamic economy, as expenditure  data age, they become less relevant to present-day expenditure  patterns. Most of BLS’s price indexes, including the CPI-U, CPI-E, and  CPI-W, rely on 2 years of expenditure data and the data require additional  time to be collected and processed for use, referred to as a lag. For  example, the CPIs produced from January 2014 to December 2015 used  expenditure data from 2011 through 2012. BLS officials said reducing the  lag could enable more timely use of expenditure data for CPIs but would  not be possible without a significant change to the use or design of the  Consumer Expenditure Survey.

Another of BLS’s indexes, the Chained CPI-U, aims to incorporate  current-period expenditure data, which may be most relevant for current- period price changes, but as we reported in 2019, the data are subject to  revision and BLS produces the final, revised Chained CPI-U with a 10 to  12 month delay. BLS officials told us they do not currently have timely  enough expenditure data to produce the Chained CPI-U without this  delay. We found in our 2019 report that if the Chained CPI-U were to be  used to calculate Social Security or other federal retirement benefit  COLAs, it could result in permanent differentials stemming from  measurement error that would have a larger effect on people who receive  benefits longest or have lower incomes.

BLS also faces several other challenges measuring inflation for older  Americans, several of which BLS is examining in the subpopulation  project discussed above.

Large purchases. BLS is examining how to treat large purchases that  are acquired in one time period but used throughout many time periods,  such as owner-occupied housing and durable goods. BLS’s current  approach to owner-occupied housing is to calculate what it would cost to  rent a similar home. In part, because many seniors own their homes, BLS  is considering instead calculating how much it costs to own and occupy  the home (e.g., by including mortgage interest payments but not the  purchase price of the home).

Definition of average. BLS is also examining whether a subpopulation  index should represent the average expenditures of all households (as its  CPIs currently do) or the expenditures of an average household. The  current approach of representing the average expenditures of all  households is simpler because the index can be constructed from  information on average expenditures. The alternate approach of  representing expenditures of an average household is more complicated  because it gives each household equal weight, and requires first  constructing a price index for each household, then an averaging of those  indexes. According to BLS, the current approach tends to give more  relative weight to the purchasing behavior of higher-income households,  whereas the alternate approach may be more appropriate for a  subpopulation index, such as the one used to adjust Social Security  benefits. For example, taking the average of all expenditures tends to  reflect the more expensive purchases typically made by higher-income  households. In contrast, measuring the average household’s  expenditures may better represent expenditures made by a particular  subpopulation, such as recipients of federal benefits programs like Social  Security.

User needs. BLS is also examining how to define the subpopulation of  interest to meet the needs of its users, such as the Social Security  Administration. Specifically, CPI-E is based on households headed by  someone age 62 or older and the CPI-W is based on households with  particular occupations, and BLS is examining whether other definitions  could meet user needs. For example, BLS said it plans to contact  stakeholders to ask about whether expanding the CPI-W to include all  labor force participants (thereby increasing sample size) would meet user  needs.

Quality change vs. inflation. A further challenge for all price indexes is  determining what portion of the price change is due to changes in quality  as opposed to inflation, according to eight of the nine experts we  interviewed. BLS has several methods to adjust for quality changes. For  example, if an older television is replaced with a new model with an  increased price, BLS analysts collect information on the characteristics of  those televisions and conduct an analysis to determine how much of the  price change is due to a change in quality (e.g., the new television has  additional features). The remainder of the price change is attributed to  inflation. While accounting for quality change is a challenge for all price  indexes, four of the nine experts we interviewed said it may be particularly  difficult when measuring inflation for older populations. According to these  experts, this is because older populations tend to consume more medical  care goods and services, for which quality changes are particularly  difficult to measure.

BLS Has Taken Steps to Incorporate Alternative Data Sources into CPIs, but Has Made Limited Use of Other Data Currently Collected by the Federal Government

Alternative data. To improve its price indexes, BLS is exploring the use  of alternative data sources, such as “big data” obtained directly from  companies, from third parties, or from the internet (see text box below).  For example, BLS recently purchased a large private dataset to use in an  experimental index for new vehicles. According to BLS, big data may lead  to methodological improvements and cost savings in the CPIs. Notably,  some big data may provide “real-time” expenditure data that could  potentially be used to capture consumer behavior in response to relative  price changes, thereby addressing substitution bias. According to agency  officials and most experts we spoke with, big data may be promising but  incorporating them in the CPIs requires additional considerations and  adjustments to the processes BLS currently has in place. For example,  the data may not be consistently available with the information needed to  produce CPIs. Additionally, big data are not always free and some  companies may be reluctant to share these data.

What is “big data?” Big data encompass a number of very large data  sets that can be structured or unstructured and have the potential to be  mined for information. Web-scraped data and scanner data are two  prominent types of big data relevant for consumer price indexes. Web- scraped data are price data collected on goods sold online. Scanner  data include price and quantity data on sales of goods obtained by  scanning bar codes for goods, such as at electronic points of sale in  retail outlets. Advances in technology have allowed large amounts of  data to be collected and stored easily and could be used in consumer  price index construction.

In addition to big data, BLS currently uses some administrative data  collected by the federal government to improve inflation estimates for  certain goods and services. For example, BLS obtains information from  the Department of Energy on household consumption averages for  electricity and piped gas service. It also uses administrative data from the  Centers for Medicare & Medicaid Services about which facilities provide  adult home care. According to BLS officials, they are unable to use some  administrative data (e.g., certain federal tax data) because of current  law.

Other data collected by the federal government (National Accounts  data). While BLS is exploring numerous alternative data sources, BLS  has not fully explored the potential to update expenditure weights on a  more frequent basis using supplementary data from the National  Accounts in years when the most current biennial weights using  Consumer Expenditure Survey data are not available. As discussed  earlier, BLS typically requires 2 years of data from the Consumer  Expenditure Survey to produce expenditure weights, which have a lag. In  contrast, National Accounts data comprise administrative and statistical  data representing the whole economy, many of which have a large  sample size and are available on an annual basis. Standards of internal  control call for agencies to obtain relevant data from reliable internal and  external sources in a timely manner to meet information requirements for  meeting their objectives. For BLS, this could include obtaining relevant  data from reliable sources for producing CPIs. As part of its strategic plan,  BLS maintains goals to improve the accuracy and timeliness of BLS data  and to ensure relevance in an ever-changing economy. Without  adequately exploring the potential of using National Accounts data to  supplement Consumer Expenditure Survey data, BLS may be missing an  opportunity to move closer towards those goals. Over time, expenditure  survey data lose their accuracy and relevance to the present-day  expenditure patterns of consumers, which can introduce bias in measures  of inflation used to adjust federal retirement benefits. For example, the  longer the time period between expenditure weight updates, the longer  the delay to include new products in the expenditure patterns reflected in  the CPIs. This delay could become increasingly important because of the  rapid development in new technology, such as smart phones.

Of the 15 publications we reviewed, six discussed ways to improve the  CPI and four of these suggested more timely expenditure weight updates  could make the CPIs more accurate and relevant. For example, a 2009  working paper by BLS staff found that more frequent weighting may offer  better representation of current price change, as well as a closer  approximation to a cost-of-living index. In particular, the authors  simulated updating expenditure weights annually, which resulted in  slower inflation increases that the authors posited are a closer  approximation to a cost-of-living index. While these improvements may  not be currently possible given the lag in Consumer Expenditure Survey  data, the authors conclude that further examination of the weighting issue  is a potentially fruitful avenue of research. The three other studies  similarly indicated that more timely weight updates would result in more  relevant CPIs, for example by better reflecting changes in consumer  spending patterns.

BLS officials acknowledged that updating the weights more frequently  would make the index more relevant, though they did not believe using  the Consumer Expenditure Survey to do so was practical in part because  they said it would require additional costs to increase the sample size.  In 2002, BLS increased the frequency of its weight updates from every 10  years to every 2 years, which they said was an improvement but required  a sample size increase in Consumer Expenditure Survey. As previously  described, the Consumer Expenditure Survey faces increasing costs and  declining response rates and, according to agency officials, obtaining a  large enough sample to update weights annually would require a 50 to  100 percent increase in sample size, for example, to avoid an increase in  sampling error. Indeed, three studies we reviewed suggested that it can  be challenging to obtain enough responses for household surveys such  as the Consumer Expenditure Survey, indicating that alternate data  sources may become more important.

In contrast, BLS officials acknowledged that National Accounts data could  provide useful supplementary information if the expenditure survey is not  providing timely enough data. However, BLS officials said they have not  explored using National Accounts data, in part because they have not  examined the effects of altering the expenditure weights in about 10  years. BLS officials expressed concern that National Accounts data can  be subject to revision. According to the Bureau of Economic Analysis  (BEA), the revisions do not reflect errors but are driven by the  incorporation of more complete source data. BLS officials also noted  that some National Accounts data are adjusted by the CPI, so BLS would  have to remove the CPI’s effect in order to use National Accounts data in  the CPI.

Moreover, the supplementary use of National Accounts data could also  help address some of the concerns with measurement error in household  surveys, according to some literature we reviewed. Specifically, National  Accounts data could be used to address underreporting due to recall bias,  the difficulty some survey respondents have recalling infrequent  purchases, or underreporting of certain goods that may be seen as  socially undesirable, such as tobacco and alcohol. For example,  according to a recent Brookings Institution report, the National Accounts  data used for the BEA’s Personal Consumption Expenditure index  weights are mostly based on business surveys and administrative data  and thereby avoid the reporting biases inherent in the Consumer  Expenditure Survey. BLS’s Technical Advisory Committee  recommended using administrative data to address such underreporting  in fiscal year 2016, as did a National Academy of Sciences report in  2013. While BLS has taken steps toward increased use of  administrative data, BLS has not fully implemented the Technical  Advisory Committee recommendation as of March 2020.

Selected Countries Use Various Strategies, Such As Obtaining Data from Alternative Sources and Bolstering Collaboration with Stakeholders, to Update Their Indexes for Retirement Benefits

Use of Retiree-Specific and Chained Price Indexes for Adjusting National Pension Benefits Is Relatively Uncommon

Our review of Organisation for Economic Co-operation and Development  (OECD) countries’ national pension systems revealed that it is relatively  uncommon to use a retiree-specific index (i.e., a CPI for the older  subpopulation) for the purpose of adjusting national pension benefits. Of  the 36 OECD countries, 27 have national pension programs in which  indexation is based, at least in part, on prices after initial benefits have  been set, similar to Social Security in the United States (see app. I).  Most OECD countries use their primary measures of inflation to adjust  national pension benefits, according to reports and documents about the  retirement systems in these countries. Of the 27 countries using prices to  adjust national pension benefits, we found evidence in 10 that the  national statistical agency produces an index for the older subpopulation.  Each of these 10 countries generally uses the same price information for  the older subpopulation index as the main CPI but reweights the price  information based on the expenditures for that subpopulation, rather than  gathering new information that is unique to that group (see text box). A  similar approach is used for the CPI-E in the United States. However, of  these 10 countries, only four countries use the index for the older  subpopulation to adjust their national pension benefits (Australia, Czech  Republic, Hungary, and the Slovak Republic). The others produce the  subpopulation index for research or other purposes, but do not use it for  pension benefit adjustments.

Agency officials in all three of our case study countries (Australia, New  Zealand, and the United Kingdom) said they generally saw a value in  having a primary index for macroeconomic purposes, such as inflation  targeting, and a subpopulation index that could be used for other  purposes, such as indexation of benefits.

Methods for Validating Use of Existing CPI Data in Subpopulation  CPIs  In the three case study countries we selected for review, each national  statistical agency relied upon different approaches to validate the use of  existing data from the primary (main) CPI in the subpopulation CPI.  Agency officials indicated that some of the methods for validating the  use of existing CPI data for the subpopulation CPIs were cost efficient.

Australia agency officials said they validated the use of existing data  in the index for the older subpopulation in part by both researching  whether pensioners pay different prices or shop at different outlets  and cross-checking some data from industry sources. Officials said  they expected that pensioners and the general population generally  pay the same prices for most items and included different prices in  the index for the older subpopulation for those items known to be  discounted for pensioners.

To get a better sense of the older population’s expenditures,  they also increased the sample size of the expenditure survey  from about 7,000 households to about 10,000 households to  include more pensioners.

New Zealand agency officials said they validated the use of existing  data in part by using existing expenditure data to confirm that goods  and services most important to the older subpopulation were  adequately represented in the data. They also said they consider  the coverage of the subpopulation group when determining the  make-up of the CPI basket. Since older people may shop at different  stores than the general population, New Zealand’s statistical agency  also developed separate outlet weights for the older subpopulation,  which more accurately reflect the different mix of outlets, or stores,  frequented by this group, according to agency officials. Overall,  officials said they found that using subpopulation-specific outlet data  instead of general CPI outlet data had very little impact on the index  for the older subpopulation.

United Kingdom agency officials said they validated the use of  existing data by organizing expenditure data from the household  survey into categories that align with national expenditure data,  which allowed them to generate bigger samples than exist in the  household survey data. As a result of the larger sample, their  statistical agency said they were able to achieve more precise  estimates for the index for the older subpopulation.

It is also relatively uncommon for a country to produce a chained index for  the purpose of adjusting national pension benefits. Of those 27 OECD  countries that are using price indexation, five of them produce a chained  index (Australia, Canada, the United States, the United Kingdom, and  New Zealand). However, none of the OECD countries use the chained  index to adjust their national pension benefits. In our three case study  countries, the statistical agencies used the chained index as an analytical  tool to measure bias in the CPI or for comparative purposes. Officials we  spoke with said that the delay required to produce a chained index made  it impractical to use the index to adjust benefits. While some of the  stakeholders in selected case study countries indicated it could be  theoretically possible to create a chained CPI for the older subpopulation,  we did not identify any countries with such an index during this review.

Selected Countries Are Supplementing CPI Data with Other Government- Collected Data to Help Bolster Gaps in Information

While government-collected data are often collected for reasons other  than the production of the CPIs, the three selected case study countries  are using government-collected data to help fill the gaps in data they  collect expressly for the CPI (see table 2).

According to agency officials in the three selected countries, use of this  government-collected data improves accuracy of the CPIs and can be a  relatively affordable way to supplement data collected for the CPI.

National Accounts, key sources of government-collected data, are  typically used for national summary measures like the Gross Domestic  Product. However, all three of the selected countries are also using  relevant consumption data from National Accounts to supplement their  CPI data, which agency officials in Australia said is in-line with  recommendations from the International Labour Organization (see text  box). Australia, New Zealand, and the United Kingdom are all using their  National Accounts data to supplement expenditure survey data in their  CPIs, while New Zealand is also using another form of government- collected administrative data to improve its CPIs.

International Guidance for Calculating CPIs and Subpopulation  Indexes  The International Labour Organization produces a manual that provides  an overview of issues that national statistical offices can consider when  making decisions on how to deal with the various problems in the  compilation of Consumer Price Indexes. Researchers from many  countries’ national statistical agencies, universities, and international  organizations (such as the World Bank, International Monetary Fund,  and Organisation for Economic Co-operation and Development) are  involved in creating the manual. The manual also establishes  international conventions, such as a suggestion that countries regularly  evaluate the use of average wages as opposed to price indexes (and  vice versa). Last published in 2004, an update to the manual is  scheduled to be released in 2020. The upcoming revised manual is  expected to elaborate on the use of National Accounts data and  alternative data sources to develop expenditure weights.

Australia. Australia’s statistical agency uses consumption data from  their National Accounts to update the CPI expenditure weights more  frequently than officials said was previously possible. Using this data  has helped reduce substitution bias, meaning that the data better  reflect changes in consumer purchases in response to price  changes. Previously, Australia updated its expenditure weights  every 6 years, when its household expenditure survey was released.  In other words, the CPI was previously calculated assuming that  consumers’ expenditure patterns did not change for 6 years. As a  result, the CPI did not account for substitution patterns to different  goods and services over significant periods of time, leading to bias in  the CPI. In 2018, the Australian statistical agency incorporated  National Accounts data in the CPI in those years when the  expenditure survey was not conducted, allowing the expenditure  weights to be updated annually to reflect what statistical agency  officials described as more timely and relevant consumption patterns  and to improve the accuracy of the data. According to Australian  statistical agency officials, they did not have the budget to increase  the frequency of their household expenditure survey, which they said  is very costly. Instead, officials said they researched alternative ways  that would allow for more frequent reweighting and settled on using  the National Accounts data in between survey years to update the  weights annually. This approach does not require a budget increase  because the National Accounts data are already produced. Australian  officials said more frequent weighting helped reduce substitution bias  in their CPIs by about 0.2 percentage points per year, which can have  a large impact on benefits over time.

By incorporating consumption data from the National Accounts,  Australian statistical agency officials said they can generate more  timely and relevant CPI measures, including the subpopulation  indexes. Australia’s index for the older subpopulation, called the  Pensioner and Beneficiary Living Cost Index, also benefits from more  frequent updates of the expenditure weights and subsequent  reduction in substitution bias in the CPI, according to agency officials.  Agency officials said that despite not having demographic information  in the National Accounts, their methods have made use of this  consumption data fit for purpose for the subpopulation indexes, and  the subpopulation indexes are as methodologically sound as the  primary CPI.

New Zealand. New Zealand’s statistical agency also uses National  Accounts data to estimate expenditure weights for insurance services,  which are relatively difficult to measure in survey data, according to  agency officials. Specifically, the expenditure weights for health and  life insurance are based on data from the National Accounts.

United Kingdom. In the United Kingdom, annual spending data from  the National Accounts are the main source for CPI expenditure  weights, as stakeholders noted that the National Accounts spending  data are more precise and timely than their household expenditure  survey. According to statistical agency officials, household  expenditure data are ultimately obtained by organizing the United  Kingdom’s expenditure survey data into categories that align with the  National Accounts and scaling up these data to the National Accounts  data. Officials said this method allows the United Kingdom’s statistical  agency to achieve larger sample sizes, and thus smaller variances  and more precision in estimates for subgroup indexes. United  Kingdom officials said that their National Accounts estimates are more  accurate and comprehensive than their household expenditure  survey, which has a smaller sample size of nearly 6,000 households.  Having more accurate expenditure data and weights leads to a more  accurate and relevant primary index for pension benefits, as well as a  more accurate subpopulation index, according to agency officials. The  National Accounts data also help the United Kingdom adjust for any  potential underreporting of particular goods in the household  expenditure survey, such as alcohol, further increasing the accuracy  and relevance of the dataset, according to officials. Collecting prices  directly from the source is more accurate than relying on someone to  recall how much they spent on items, according to one stakeholder.

Government agencies from selected countries also produce other  administrative data that can be useful in measuring the CPI. For example,  New Zealand’s statistical agency partnered with the Ministry for Business,  Innovation, and Employment to use its tenancy bond database, which  covers approximately 85 percent of all rental housing units in the country.  These data facilitated a new way to measure rent in their CPI. Moreover,  this partnership enabled New Zealand’s statistical agency to create an  index of rent prices monthly, instead of quarterly, which resulted in a more  accurate and timely depiction of what people are spending on rent and a  more accurate indexation of benefits overall. According to agency  officials, the transition to these administrative data replaced the CPI survey of landlords, and in doing so it lowered respondent burden,  increased the timeliness of the rental component of New Zealand’s CPI,  and improved population coverage. In all of our case study countries,  various data are used to measure housing prices (see text box).

Housing and the Consumer Price Index  Measuring the change in housing prices for CPI is widely acknowledged  by experts to pose methodological and data challenges. In response,  national statistical agencies have developed a variety of approaches to  address the measurement of owner-occupied housing costs, both in the  primary CPI and subpopulation indexes. Officials in the national  statistical offices of the case study countries said that one of the factors  underlying the approach to housing is whether the measure should  reflect inflation in the economy overall or inflation as experienced by  households.

In Australia and New Zealand, the primary CPI includes price  changes stemming from the purchase of a new home but not via  mortgage interest payments (known as the acquisitions approach),  while the subpopulation index excludes the purchase of a new home  but includes mortgage interest (referred to as outlays or payment  approach).

In the United Kingdom, there are two versions of the primary CPI:  one that uses “rental equivalence” (a calculation of what the owner  would pay in rent for an equivalent house) and one that excludes  owner-occupied housing costs. In addition, the United Kingdom’s  subpopulation index uses a payments approach.

Selected Countries Are Using Alternative Big Data Sources to Get More Data in a More Timely Way

Officials in our selected case study countries said they are using  alternative big data sources, such as web-scraping data and transactional  (scanner) data to help them more accurately index their national pension  benefits (see table 3).

These officials said that these alternative data sources allow countries to  obtain a higher volume of data and more accurate data to incorporate into  their CPIs, subsequently making the indexation of benefits more accurate.  Electronic price data obtained from a retailer, whether through the  retailer’s website or through scanner data the retailer shares with the  national statistical agency, reflects accurate and timely data on the price  and quantity of goods and services sold. Electronic price data can be an  improvement over data collected in household expenditure surveys, for  example, as several experts and agency officials in one case study  country noted that household expenditure surveys suffer from recall bias,  resulting in less accurate spending data. The three selected countries are  at different stages of incorporating scanner data into their CPI. Officials at  the national statistical agencies in all three of our case-study countries  stated that they are primarily focused on incorporating scanner data from  grocery stores into their CPI. Using grocery store data is possible, in part,  because these countries contain a relatively small number of stores that  dominate grocery sales, according to agency officials, which is a  difference from the United States.

Australia. According to stakeholders, the Australian statistical agency  developed a formula that incorporates a chained formula into a portion  of the CPI using high- frequency scanner data from the country’s  dominant grocery stores, which provides timely price and expenditure  data on food items for their indexes. Integrating this type of high- frequency data is not easy, they said, since the traditional CPI  formulas are not built to handle the volume of data that scanner data  produce. However, in consultation with academics and statistical  agencies from around the world, Australia was able to develop a  chained formula that uses an innovative statistical method, known as  a multilateral approach, to incorporate the scanner data. As a result,  the portion of the CPI for which Australia has scanner data (about  one-sixth of the CPI, comprised mostly of food and other grocery  data) is based on a chained formula. Incorporating these data allows  the country to include all of the products available in the datasets,  rather than a small sample of products, leading to a more accurate  calculation of food prices and a more accurate index overall, for both  the general population and the older subpopulation, according to  agency officials. Stakeholders in Australia noted that the international  price statistics community has since reached a consensus that  multilateral methods are the most effective way to capitalize the full  amount of information provided in scanner data, and they said that the  forthcoming update of the International Labour Organization’s CPI  Manual is expected to recommend this method as well.

New Zealand. New Zealand’s statistical agency is working towards  incorporating more scanner data, primarily from its two large  supermarket chains, in the production of the country’s CPIs, which will  help achieve a more accurate index for both the general population  and the older subpopulation, according to agency officials. New  Zealand started using retail scanner data to supplement its  expenditure data in its CPI in 2006, and in 2014 New Zealand  incorporated direct measurement from scanner data for consumer  electronics products into its CPI. Officials from the national statistical  agency said they hope to expand their use of this type of big data in  the near future. They have already received the data from  supermarkets, whose goods account for roughly 20 percent of the  goods and services in the CPI, but they have not yet integrated the  data into their CPIs. Agency officials said they expect to integrate this  in the next year. New Zealand’s statistical agency officials said they  have a goal to obtain scanner data for other CPI components soon as  well, such as fuel.

United Kingdom. In the United Kingdom, agency officials said  improvements in technologies have resulted in new alternative  sources for price data that could be used in the compilation of their  price indexes in the near future. The United Kingdom’s statistical  agency is currently exploring both scanner data and online price data.  The agency currently has several streams of research looking into the  expanded use of alternative data, including research studying the  feasibility of moving away from collecting prices manually towards  using electronic means wherever feasible and efficient. The agency is  now receiving web-scraped data from an online source that captures  prices from online sales of goods like clothing. The United Kingdom’s  statistical agency is also continuing to engage with retailers on  receiving scanner data covering areas such as clothing and groceries,  targeting some of the largest retailers from which the agency currently  manually collects prices. These data sources may provide a more  efficient way to capture the increase in online expenditures that has  occurred over the last decade, and will likely continue to occur. These  new data are initially being used for research work, but over time the  web-scraped online prices and scanner data will be used when  calculating primary inflation indexes, according to agency officials.  The research done by the United Kingdom’s statistical agency into  grocery store items has also enabled officials there to explore different  methods of collecting web scraped prices in-house. The officials said  this has led to wider benefits for the agency in general, with an  increase in knowledge and experience that has contributed to the  success of other big data projects.

Selected Countries Collaborate and Consult with National Stakeholders and Experts When Implementing Changes to Their CPIs

Our selected case-study countries use committees with stakeholders and  advisory panels, including academic researchers with subject matter  expertise, to implement innovative changes to their CPIs (see table 4).

The statistical agencies in these three countries have shown a willingness  to act on recommendations that came out of these collaborative efforts.  These countries are also seeking input from the international statistical  community, which country officials said has led to positive developments  in their CPIs.

Australia. Australia’s statistical agency has taken a variety of  approaches to collaborate with external stakeholders, which agency  officials said has led to positive changes to their CPIs, and thus  indexation of benefits over the years. According to agency officials,  Australia’s collaborative efforts include:  conducting regular reviews and seeking stakeholder input every 6  years with the release of the expenditure survey;  convening workshops with stakeholders including both academics  and users (e.g., the agencies that distribute benefits); participating in international conferences to receive feedback on  changes to the country’s CPI and subpopulation indexes; partnering with methodology experts in other agencies such as the  Treasury and central bank, occasionally by obtaining staff on  detail; and  commissioning reports that research and review measures to  strengthen the financial security of seniors.

These reviews and associated collaborative efforts have helped the  agency learn more about the issues it faces and have helped trigger  changes that will improve the accuracy of the nation’s CPI, according to  agency officials. For example, as discussed above, agency officials said  that a 2011 CPI review revealed concern by the Reserve Bank of  Australia and others that the infrequent reweighting was resulting in bias  in the CPI that affected inflation targeting by the central bank, as well as  benefit expenditures. This review helped spur innovations, such as  including the incorporation of scanner data into the nation’s CPI, which  delivered positive results with respect to more timely and relevant data  being used to estimate inflation. Australia’s statistical agency officials said  they sought extensive input from key governmental stakeholders, a  number of academic experts, as well as international experts to research  how to best incorporate scanner data into their CPI, which agency  officials noted was necessary to facilitate the integration of high-frequency  scanner data into the CPI. They also conducted numerous bilateral and  multilateral consultations with key stakeholders in the government that  use CPI data, including the Reserve Bank of Australia, the Treasury,  Department of Finance, Department of Social Services, and State  Treasuries. Australian statistical agency officials suggested that  consulting with users of the data frequently was an important part of  implementing changes to the measurement of the CPI and subpopulation  indexes.

New Zealand. New Zealand’s statistical agency has also used CPI  advisory committees composed primarily of external stakeholders who  make use of the agency’s CPIs. For example, in 2013 New Zealand’s  statistical agency convened a committee to independently review the  methods and practices used to compile the CPI and make  recommendations, for example, about how additional indexes should  be measured. The committee also incorporated public submissions on  the scope and uses of the CPI, for example, from nongovernmental  organizations and interest groups such as retiree advocacy groups.  The committee then released a report recommending the creation of  additional CPIs that are designed for microeconomic purposes, such  as the indexation of retirement benefits, to better reflect changes in  the purchasing power of the incomes of particular subgroups of the  population, like the older subpopulation. The committee also  recommended that New Zealand’s statistical agency review the  sample size and collection methods of their expenditure survey to  improve the reliability of expenditure estimates of the required  population subgroups so that the estimates could eventually be of  high enough quality to be published, which they subsequently were.  According to officials, the committee’s report helped lead to the  creation of New Zealand’s subpopulation indexes. Moreover, the  committee recommended that the statistical agency try to use retail  scanner data to measure price change and stated that the method  aligns with international best practices. New Zealand’s statistical  agency recognized these best practices and the international  consensus that multilateral methods are the preferred way to  incorporate big data. Indeed, it has started to use these methods in  the rental prices data and it plans to continue to research  implementing these methods further.

United Kingdom. The United Kingdom has also developed advisory  panels on consumer prices to provide independent advice to the  National Statistician, which officials said has allowed the United  Kingdom’s statistical agency to learn more about challenges with the  nation’s CPIs and to find possible solutions. Similar to the United  States, the United Kingdom has advisory groups on technical issues,  as well as on the uses of price indexes. The reports published by  various advisory groups have raised technical issues with the Retail  Price Index (RPI), which is the United Kingdom’s longest running  measure of inflation. These technical issues resulted in the RPI being  higher than the CPI. Ultimately, agency officials said consultations  and advisory panel input helped lead to the RPI being decertified as a  national statistic (see text box). The United Kingdom’s statistical  agency also hosted numerous meetings and a collaborative workshop  about the conceptual foundations of its subpopulation indexes, which  are currently being developed. According to agency officials, obtaining  input from internal and external stakeholders has been critical to  developing solutions to indexation challenges.

The United Kingdom’s Experience Changing Price Index Used for  Pension Adjustments  Changing the index used for benefit adjustments can be difficult, as  switching price indexes can involve tradeoffs. For example, public and  private pension benefits in the United Kingdom have traditionally been  indexed by the Retail Price Index (RPI), the oldest index in the United  Kingdom. The United Kingdom recently switched indexation of certain  government benefits, including pension benefits, from the RPI to the  slower-growing CPI. This is expected to result in lower payouts from the  government. In contrast, the government continued using the faster- growing RPI for some provisions, such as student loan interest rates,  that resulted in higher payments to the government. Stakeholders  suggested that having multiple measures of inflation can create  incentives for the government to use different indexes for its own  budgetary advantage, with pensioners receiving lower benefit  adjustments and students facing relatively higher loan payment  adjustments. The United Kingdom’s experience highlights that changing  the index for benefits may result in advantages and disadvantages for  different groups and thus may be politically difficult, according to agency  officials.

Federal retirement programs like Social Security have relied upon a  subpopulation price index to adjust benefits since automatic cost-of-living  adjustments were first enacted almost 45 years ago. This index estimates  changes in purchasing power for wage earners as opposed to changes in  the standard of living or some other type of measurement. In recent  years, numerous legislative proposals have been suggested to change  this index from one that measures the purchasing power of wage earners  to one that targets some different population, for example one solely  focused on the elderly. Much of the debate over using a different index  has centered on the ability (i.e., the accuracy) of the indexes to capture  changes in the cost of living for a particular group in society.

BLS is unsure whether the data sources it currently uses are adequate to  produce accurate CPI-E and CPI-W subpopulation indexes on a timely  basis, according to BLS officials and documentation. While the CPI-E is  experimental and not used by federal programs, the CPI-W is used to  adjust billions of dollars of Social Security and other federal retirement  program benefits. It is therefore critical that the measurement be as  accurate as possible. However, ensuring the measurement’s accuracy  may require a reexamination of the underlying data used to produce the  subpopulation indexes. BLS has not evaluated the adequacy of existing  data because it is costly to undertake a full evaluation, according to  agency officials. But experts we interviewed, including some on BLS  advisory groups, indicate there may be cost-efficient ways to conduct  such a review. Although the experiences of other countries may not be  directly applicable, other countries have found ways to evaluate the use  of existing data for their subpopulation indexes, and officials in all three of  our case study countries expressed the view that some of these methods  were cost efficient. Absent BLS evaluating the adequacy of the existing  data it uses to produce its subpopulation indexes, BLS will continue to be  uncertain if its subpopulation indexes are accurate and it may not learn of  potential areas for improvement.

In addition, BLS currently relies on the Consumer Expenditure Survey to  produce expenditure weights that measure the mix of goods and services  consumers purchase and, because of survey shortcomings and  processing lags, the weights reflect spending patterns that can be up to 4  years out of date. Although BLS has taken other steps to improve the  accuracy, timeliness, and relevance of data used in the CPIs, BLS has  not fully explored the potential to update expenditure weights on a more  frequent basis using annual data from the National Accounts, which are  currently collected in part to measure Gross Domestic Product. While not  specifically designed for use in CPIs, the National Accounts data may  provide BLS an opportunity to supplement Consumer Expenditure Survey  data in the intervening years. Moreover, some literature we reviewed  indicated that the use of National Accounts data has the potential to  mitigate measurement error in the Consumer Expenditure Survey,  thereby increasing accuracy. Without adequately exploring the potential  of such an option, BLS may be missing an opportunity to improve its  CPIs.

We are making the following two recommendations to the Department of  Labor:

The Secretary of Labor should ensure that BLS explores cost-efficient  ways to evaluate the data sources currently used to produce  subpopulation indexes, such as by engaging more directly with other  stakeholders or seeking input from its advisory groups and other  knowledgeable entities about approaches to expand data collection in  a cost-efficient manner. (Recommendation 1)

The Secretary of Labor should ensure that BLS explores the use of  already collected National Accounts data to produce more accurate,  timely, and relevant CPIs. (Recommendation 2)

We provided a draft of the report to the Department of Labor, the Social  Security Administration, and the Department of State for their review and  comment. We also sent an informational copy to the Bureau of Economic  Analysis. The Department of Labor and the Social Security Administration  provided technical comments, which we have incorporated where  appropriate. In an email, the Department of State said it had no  comments on the report. The Department of Labor also provided written  comments, which are reproduced in appendix III and discussed below.

In its written comments, the Department of Labor stated that BLS  continually improves its measures according to a guiding principle to  provide accurate, objective, relevant, timely, and accessible information.  The Department of Labor agreed with the first recommendation to explore  cost-efficient ways to evaluate the data sources currently used to produce  subpopulation indexes and stated that it would continue to investigate  improvements to subpopulation indexes.

The Department of Labor disagreed with the second recommendation to  explore the use of National Accounts data in the construction of its  indexes, stating that the National Accounts data are not a replacement for  Consumer Expenditure Survey data. While we agree that the National  Accounts data are not a wholesale replacement for the Consumer  Expenditure Survey data, we believe that it would be useful to examine  National Accounts data as an augmenting, alternative source of data that  could supplement or enrich the Consumer Expenditure Survey. Such an  effort could potentially lead to more accurate, timely, and relevant CPIs.  Although the Department of Labor stated that the Consumer Expenditure  Survey is a continuous survey and that data are received quarterly, most  CPIs still rely on expenditure weights based on Consumer Expenditure  Survey data that are up to 4 years out-of-date. In addition, the Consumer  Expenditure Survey faces increasing costs and declining response rates.

The Department of Labor stated in its comments that it is exploring ways  to accelerate the data collection and processing time and that it  periodically investigates the frequency of updating expenditure weights.  We commend the Department of Labor for considering these efforts, and  we maintain that they could take further action to explore additional  opportunities for improvement. For example, the Department of Labor  could research the extent to which there are instances or categories for  which the National Accounts data could be used to produce more up-to- date expenditure weights than the Consumer Expenditure Survey. As we  noted in our report, Department of Labor officials told us they periodically  examine National Accounts expenditure data to explore differences with  the Consumer Expenditure Survey data, not to explore supplementary  use of alternative data. While it cannot be ensured that every expenditure  data point in the National Accounts will be of use for producing CPIs, we  maintain that further exploring the National Accounts expenditure data as  a complement to the Consumer Expenditure Survey data may provide  opportunities for BLS to improve the accuracy, timeliness, and relevance  of its CPIs.

We are sending copies of this report to the Secretary of Labor, the  Commissioner of Social Security, and the Secretary of State. In addition,  the report will be available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-7215 or jeszeckc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix IV.

Appendix I: National Pension Indexation Formulas in the 36 OECD Countries

Appendix II: Additional Information about Selected Case Study Countries

Appendix II: Additional Information about  Selected Case Study Countries technical corrections as necessary. We note also that the fact that a legal feature was successful in  one or more of the countries we visited, which may have significantly different cultures, histories, and  legal systems than the United States, does not necessarily indicate that it would be successful in the  United States.

Appendix III: Comments from the Department of Labor

Appendix IV: GAO Contact and Staff Acknowledgments

Charles A. Jeszeck, (202) 512-7215 or jeszeckc@gao.gov  In addition to the contact named above, Michael Collins (Assistant  Director), Laura Hoffrey (Analyst in Charge), Emilio Fonseca, Kathleen  McQueeney, Tom Moscovitch, and Julie Miller made key contributions to  this report. Also contributing to this report were Deborah Bland, Alicia  Cackley, Charles Ford, Sarah Gilliland, Susan Irving, Kelsey Kreider,  Sheila McCoy, Jessica Orr, Oliver Richard, Joseph Silvestri, Almeta  Spencer, Curtia Taylor, Frank Todisco, Walter Vance, Adam Wendel, and  Sirin Yaemsiri.

Federal retirement programs often have Cost-of-Living-Adjustments (COLAs) to ensure benefits keep pace with inflation. This includes Social Security for more than 60 million older Americans, workers with disabilities, and their families. These COLAs are typically based on consumer price indexes for certain groups of Americans.

The Bureau of Labor Statistics (BLS) produces these indexes, but it hasn’t evaluated whether its data accurately reflect what these groups pay, where they shop, and what they purchase. We recommended that BLS evaluate the data they use, as well as explore using other data, to improve its indexes.