Turning Smart Data into Economic Insight

Earlier this month I spoke alongside Will Taggart from Sage UK at the TechNExt conference about how operational SMB data can be used to deliver economic impact, strengthen economic insights and enable better decision making for businesses, policymakers and institutions.  

Smart Data Foundry and Sage have been working together for several years, turning the data created by Sage’s customers in the course of their everyday operations into a unique, near real-time view of the economy which is grounded in actual business activity.  

That distinction matters more than it might appear. Too often, policymakers are making decisions with data that is months or years out of date, drawn from surveys with declining sample sizes (such as well publicised problems with the ONS’ Labour Force Survey), and which capture what people and businesses say they do rather that what they actually do.  

The result is that policy is often made with backward-looking, incomplete information. Policymakers cannot respond to changing economic conditions as they emerge, lack the granular, geographic detail that place-based decision-making requires and struggle to test whether their interventions are actually working. 

Smart data offers a different approach. 

What smart data is, and what it can do 

Smart data is created through everyday interactions with digital systems: bank transactions, payroll records, invoices, VAT returns. Samples can run into the millions – a scale which survey-based approaches cannot match. It is geographically granular, regularly updated, and it’s created in real-time by people and businesses.  

As a complement to official statistics, smart data can help fill the gaps by providing more timely insight into changing economic conditions, supporting place-based policy with finer geographic detail, and offering a window into trends that traditional surveys are increasingly struggling to track. 

At Smart Data Foundry, we work to make this kind of data responsibly accessible to researchers, data scientists, and policymakers. Financial data donated by partners - including anonymised consumer bank account data covering five million customers and SMB data spanning payroll, invoices, VAT, and cash flow - is ingested into a secure Trusted Research Environment. From there, it feeds research datasets, dynamic indicators, and analytical tools that are being used by local and national governments and academic researchers across the UK. 

From data to policy: what this looks like in practice 

The projects emerging from this work illustrate what becomes possible when smart data is harnessed for good.  

• Understanding economic inactivity: Financial transaction data can reveal real income flows in a way that official statistics cannot, allowing researchers to observe economic activity - and inactivity - as it happens. For young people in particular, where economic inactivity has become a growing concern, this kind of timely, granular evidence can help policymakers understand what is driving the problem and where interventions might have most impact in different locations across the UK. 

• Enabling productivity and growth: Invoice and payments data from SMBs can reveal how businesses are paid, how long they wait for payment and the costs in terms of lost investment, growth and productivity to SMBs. Sage data has identified that late payments costs the UK economy around £11bn a year – and is helping to inform legislation currently going through parliament to change payment culture. 

• Exploring regional living standards: Used alongside survey data, financial expenditure data can provide a clearer picture of living standards across localities. Expenditure patterns reveal inequalities that income figures alone can obscure - something that matters enormously for targeted, place-based policy. Read this working paper from the Institute for Fiscal Studies for an example.  

• Assessing the impact of recent policy changes. As the National Living Wage rises and employer NICs increase, understanding the real-world impact on small businesses is critical. Smart data on payroll, cash flow, and invoices can show, in near real-time, how small firms are responding: whether they are absorbing costs, raising prices, or reducing headcount. Our Fellowship programme funds independent academic researchers to work with this data, includes projects exploring exactly these questions. 

Building the UK’s smart data for good ecosystem 

The value of smart data grows with the breadth and diversity of the data, technology and skills ecosystem around it. That means building a network of data partners willing to share data for public benefit. It means providing researchers with access to novel data via secure, trusted infrastructure that protects privacy through rigorous access protocols. It means enabling users with the right tools and to work with large, complex datasets. 

Smart Data Foundry is part of that broader effort. Through Smart Data Research UK and partnerships with organisations like Sage, NatWest Group and the Royal Foundation, we are working to build the foundations for a smart data for good ecosystem. one where the insights locked in financial data can be put to work on the challenges that matter most to people's lives.