Leveraging Smart Data to Complement Official Statistics in Responsive Policy Design

The government has announced an independent investigation into the Office for National Statistics (ONS), after a series of errors in official economic indicators alongside delays to data publication.

Government departments, especially the Treasury, rely on figures produced by the ONS to set growth forecasts and fiscal policies, meaning that issues with the data produced by the ONS can end up impacting the financial wellbeing of UK households. 

As reported in The Financial Times, concerns about the accuracy and efficacy of the ONS’ labour force survey have been raised by the Bank of England, Office for Budget Responsibility (OBR)and government policy makers.

The ONS plays a vital, and tricky, role in producing the official statistics on which government relies. But it is challenged by declining participation rates in its surveys and as well as by providing timely data to support informed policy making. Low participation rates mean that the data is increasingly subject to sampling errors and biases, and lags between collection and publication mean that the data no-longer reflects the current reality by the time it is in use.

Richard Hughes, OBR chair, recently told the cross-party Treasury committee of MPs: “Ideally, we want a snapshot of where the economy just was, before we’re trying to figure out where it’s going over the next five years. Instead, at the moment we get an incomplete picture from different points in time, from what we get from the ONS at the moment.”

The problem with official statistics 

Above the specific issues which have launched the ONS investigation, official statistics may not always meet policy makers requirements because:

• there can be a time lag of months or even years associated with the collection and preparation of official statistics.
• collection methods can rely on individuals’ participation in surveys or panels
• figures are high-level and national, and may not provide the granularity required for targeted, localised interventions. 

In a recent webinar, Simon Pittaway of the Resolution Foundation contextualised the impact these issues are having on their ability to conduct impactful research:

“ONS data has never been particularly timely. What the ONS do is really important and it takes time to administer surveys, clean the data and make that available. 

However, in the latest iteration of the living cost of food survey, the headlines were published in August 2024 and that was for data going back to April 2022 to March 2023. That’s a two or three-year lag between the survey being out in the field and us getting our hands on the data. And that's particularly tricky at the moment when things are moving really quickly in the real world.”

These challenges are not new. Accurate and timely data has always been key to making policies which can deliver economic growth, the core mission of this Labour government. An increased focus on devolving power to the regions also ups the requirement for timely and accurate data at the national, regional and local level to support newly devolved authorities to devise targeted policy interventions which can meet local needs.

Alternative data sources for policymakers

Whilst the challenges of using official statistics for policy are not new, what is new is the availability of alternative data sources which can complement and enhance these statistics, supporting government and policy makers to move at the speed required of governing in turbulent times.

Smart data is one such source. Smart data is created whenever we interact with the digital world (which is pretty much all the time). Examples of smart data include energy usage from smart meters, online job postings, GPS data from public transport or banking data. When responsibly shared with policy makers it can be a powerful source of insight into the current state of people’s finances, public transport adoption, energy consumption or employment (and much more). And policy makers can use these insights to create more responsive, impactful policies.

If we take the example of consumer banking data, the benefits for policy makers are: 

Real-world, not reported

Banking data reflects how people actually earn, save and spend not what they report to have earned, saved or spent on a survey. This provides policy makers with a more accurate picture of economic wellbeing.

Real-time, not months or years out of date

Banking data can be available in near real-time, supporting policy makers with the tools to respond swiftly to emerging issues. The data is also more likely to be more relevant, as it reflects financial behaviours in the current climate not the economic climate of months or years past.

Smart data in practice

An example of how this data can be used in practice is the Economic Nowcast. Drawn from millions of de-identified consumer bank accounts and thousands of SMBs, the Nowcast equips policymakers, analysts, and public service providers with the timely insights necessary for proactive decision-making, enhancing their ability to address economic challenges swiftly and effectively. Indicators tracked by the Nowcast include monthly updates on average weekly income and the proportion of UK citizens whose income does not cover their expenditure, as well as quarterly updates on small business
productivity.

For more granular, localised insights, the Economic Wellbeing Explorer enables policy makers to compare national, regional and local economic wellbeing trends and gain data-driven insights into emerging challenges as well as track the impact of policy interventions. 

Enhancing policy effectiveness with real-time financial data

Integrating real-time financial data into government can support more informed decision-making, optimised resource allocation, and enhanced effectiveness of public services. It sets a foundation for innovation in public policy, ensuring that governments remain agile and responsive in a constantly evolving world.

Real-time financial data offers substantial benefits for policymakers by providing timely, accurate, and actionable information. As technology continues to advance, the integration of real-time data into public policy processes will likely become increasingly essential for effective government.