AI is reshaping the economy, but good luck measuring how

NEW YORK – Pretty much everyone agrees that artificial intelligence (AI) has the potential to reshape the economy in the coming decades. But no one is sure what effect the technology is having right now.

According to some measures, AI is contributing to high unemployment rates among new graduates and might already have destroyed tens of thousands of jobs. Other sources suggest companies might actually be adding workers as a result of the technology.

AI might be contributing to the US inflation problem, or be part of the solution to it. It might be responsible for a recent pickup in productivity growth, or might be playing virtually no role – or the productivity boom itself might be a mirage.

Researchers are unable to even agree on basic questions like how many companies are using AI or which workers are most vulnerable to the disruptions it could cause.

The conflicting signals partly reflect the challenge of detecting economic shifts in real time. Government statistics are inherently backwards-looking, and they are better at measuring broad trends than developments in specific sectors or regions. New technologies that might lead to the emergence of new products, jobs or entire industries can be particularly difficult to measure.

What makes AI different is the speed of its spread through the economy. It has taken less than four years for generative AI to go from a novelty useful mostly for writing limericks to a powerful tool adopted by the world’s largest corporations.

Economists have become convinced that the technology will have profound implications for workers and the economy, even as they disagree about what those implications will be. By the time the data is clear, they warn, it could be too late for policymakers to figure out how to respond.

“The stakes are super high,” said Nathan Goldschlag, director of research at the Economic Innovation Group, a think tank. “Getting the policy right is going to depend on getting the measurement right. You can’t get the policy right if you don’t know what’s happening.”

Goldschlag published a report on July 2 documenting the challenge of AI measurement and proposing steps to improve it. He and other experts argue that the government and the private sector should be devoting more resources to the problem.

They are getting at least a hearing in Washington. In June, a bipartisan group of senators introduced a Bill that would expand data collection and require the federal government to produce an annual report on AI’s effect on the labour force.

“The government’s got to make some big decisions about AI and about the economy, and if you’re doing that in a vacuum, you’re going to make mistakes,” said Senator Mark Kelly, one of the Bill’s sponsors. “This affects millions of Americans’ lives and millions of businesses. And you can’t do this smartly without reliable data.”

Mixed signals

Policymakers are not flying completely blind. Since 2023, the Census Bureau has asked companies about their AI use in a biweekly survey. It has also included questions about the technology in an annual business survey, although only intermittently.

Researchers have developed several measures of “AI exposure,” many of which use a government database of job descriptions to assess which occupations will be most affected.

Economists can use those measures to figure out whether the most exposed occupations are adding jobs more slowly, for example, or experiencing different rates of wage growth.

The trouble is that the sources often tell confusing or contradictory stories. Surveys reach wildly different estimates of companies’ AI use based on how questions are asked. AI exposure measures tell different stories about which jobs will be most affected.

In one study, economists at Northwestern University and American University found that when they used different exposure measures, those could influence not just the scale of AI’s effect on jobs but the direction. AI was hurting employment according to some measures, and helping according to others.

“It’s like going to the doctor and getting three different diagnoses for the same condition,” said Michelle Yin, a Northwestern University economist who was one of the study’s authors.

Part of the problem is that the best-known measures of the economy were developed for an era before personal computers and the internet, let alone AI.

The monthly jobs report from the Bureau of Labor Statistics, for example, provides breakdowns of job growth in manufacturing, retail and construction, but not in technology, which has adopted AI tools most aggressively. Instead, tech is spread across several categories, including information, a broad sector that also includes newspapers and film studios.

The jobs report provides even less information on occupations that might be vulnerable to displacement, such as software developers, accountants and customer service agents. The most recent breakdown of detailed occupations is from May 2025, an eternity ago in the fast-evolving world of AI.

Still, economists say that for all its shortcomings, government data will be crucial for understanding AI’s effect over time.

Researchers at the Yale Budget Lab, for example, have begun publishing a monthly analysis based on government data that tracks “occupational churn”, how quickly the types of jobs that make up a given industry are changing.

The measure is designed to be something of an early warning system for AI’s effects. As companies begin adopting the technology, the researchers theorise, they are likely to begin hiring for different roles even if their total number of employees does not change right away.

“It’s easy to pick up case studies in retrospect,” said Martha Gimbel, executive director of the lab. “What makes this time different is we are actually trying to measure this and figure this out in real time.”

But those efforts could be hampered by a federal statistical system that has been plagued by falling response rates to government surveys.

Shrinking budgets have made it hard for statistical agencies to fill the gaps. Erika McEntarfer, who led the Bureau of Labor Statistics until President Donald Trump fired her last year, said an additional US$10 million (S$13 million) a year in funding would allow the agency to expand the sample size of its monthly labour market survey so that it could do a better job of capturing economic shifts.

“The data we’re currently using to understand AI’s impact on the labour market is in jeopardy because of funding shortfalls,” she said. “It would take only some very modest investments to shore them up.”

Private data

Many economists are not waiting for the government to catch up. Several research teams have released AI measures based on private-sector data that is more detailed and more timely, albeit less comprehensive, than what is available from the government.

The Stanford University Digital Economy Lab in June released a dashboard of AI indicators based partly on data from ADP, the payroll processor.

That data shows that entry-level jobs have declined sharply in the most AI-exposed sectors since ChatGPT debuted in 2022. Erik Brynjolfsson, the lab’s director, called the trend a canary in the coal mine for AI-driven job losses.

“I think it’s comparable to the Industrial Revolution in terms of how it’s going to affect the labour market,” Brynjolfsson said. “I wish the federal government was investing more in it. But meanwhile, there’s some great private data sources that we’re pulling together, and that’s what I think is helping to fill that gap.”

But the private data is just as muddy as the government statistics. Research published last week by Ramp, an expense management company, and Revelio Labs, a labour market data firm, found that the companies using AI most intensely were adding jobs more quickly than those that had been slower to adopt the tools.

Ramp has access to data on which AI tools its customers are buying and how much they are spending on them. That allows it to distinguish heavy users from more cautious adopters – a crucial distinction, because it takes time and investment for companies to figure out how to use the tools effectively, said Ara Kharazian, lead economist at Ramp.

“It’s difficult to measure AI’s impact on a business, because it requires sustained adoption,” he said. “It’s clear in our work that a simple chat subscription does not drive productivity for a firm.”

Such data is not necessarily representative of the entire economy, however. ADP’s clients tend to be relatively large and well established. Ramp’s clients tend to be tech-savvy. But if AI is going to have the effects that its biggest boosters promise, it will need to be adopted by companies of all shapes and sizes.

Work in progress

Researchers generally agree on one thing: AI’s effect on the broad economy has been limited so far.

That is not necessarily surprising. Brynjolfsson and other economists have found that technological innovations often follow a J-shaped pattern, in which companies initially become less productive as they experiment with new tools, then experience rapid gains once they figure out how to take advantage of them.

The confusing economic evidence suggests that many companies are still on the downwards part of the J.

“The signals are mixed because, probably, the underlying economics are mixed, because we’re still in a period of experimentation,” said Goldschlag, the Economic Innovation Group economist. “The tools themselves are still becoming useful.”

If white-collar jobs really do begin disappearing en masse, as some in Silicon Valley predict, it will not take long for the losses to show up in the government’s data. But even then, it may not be obvious that AI is to blame.

The US economy has undergone a series of shocks in recent years that have nothing to do with AI: the Covid-19 pandemic and its ripple effects, including the return-to-office battles that persist to the present day; inflation and the high interest rates that the Federal Reserve has adopted to fight it; and drastic swings in government policy on immigration, trade and other areas.

If a company has cut jobs since 2022, it is not easy to tell whether that is the result of AI, high interest rates or both.

Over time, it should become easier for researchers to separate the effects of AI from other forces. But they still will not be able to resolve the question that policymakers and everyday citizens want most to answer: What comes next?

“What the data can almost never tell us is where we’re going to be in five to 10 years,” said McEntarfer, the former commissioner of labour statistics. “People are looking to data to answer that question, and it’s just too difficult.”

• This article originally appeared in The New York Times.