In the 1920’s, Joseph Stalin had a vision for turning Soviet Russia into an industrial powerhouse capable of going head-to-head with the West. He introduced a series of Five Year Plans designed and administered by Gosplan, the USSR’s central planning agency. Gosplan set quotas for factory output. And they really weren’t suggestions. If factory managers and their workforces missed those targets, losing their jobs was the least of their worries. Meeting them meant big bonuses.

Managers did what managers always do. They managed to the metrics.

That brings me to nails.

The USSR supposedly had a shortage, so Gosplan mandated higher output. When quotas were set by volume, factories churned out nails so small and flimsy they couldn’t hold up a picture frame. When quotas switched to weight, workers produced nails so massive that carpenters couldn’t lift them, much less use them.

On paper, production soared, and workers got big paydays. In reality, no one had a usable nail.

There is healthy debate about whether the Soviet Nail Factory story is pure urban folklore or an exaggerated story based on historical facts. That didn’t stop economists, years later, from using it to amplify what is known as Goodhart’s Law.

First articulated in 1975 by British economist Charles Goodhart, it holds that that “when a measure becomes a target, it ceases to be a good measure.” He was writing about monetary policy. But the lesson applies to every case where metrics and the inputs used to support them are separated from important business outcomes.

And that is exactly the problem with how a recently published MIT study chose to frame Gen AI’s ROI.

Businesses can hit every KPI they know how to measure today and still end up with nothing the market will want tomorrow. That was the Soviet Nail Factory’s fate.

Only the modern business penalty isn’t Siberia. It’s irrelevance.

Gen AI In Real Life

Recently, MIT’s much-publicized study, The Gen AI Divide, reported that 95% of Gen AI projects fail, and that enterprises see no ROI, as in zero.

That’s because the payback, according to the report, should be measured in “millions of direct dollar reductions in external spend,” and in a relatively short space of time. That expectation is the modern equivalent of Gosplan demanding nails by weight. It separates what’s being measured from the outcome.

That makes the MIT report narrative one that suggests Gen AI is floundering. The evidence over the course of the last 18 months says otherwise.

Each month since March of 2024, PYMNTS Intelligence has fielded a monthly study to enterprise CFOs, CPOs and COOs to benchmark the Gen AI sentiment, use cases and ROI impacts from the technology over time. We have more than 1,000 unique observations and 100 thousand data points from companies with $1 billion or more in annual revenue. I am not familiar with any other study that systematically measures the impact of Gen AI at the enterprise level over time.

Over those 18 months, we find that enterprise CFOs, CPOs and COOs view Gen AI as an integral part of their “digital operating systems.” Over that time, we have observed a steady shift in the use of Gen AI from simplifying routine tasks to becoming an embedded part of strategic functions within their firms.

To take a few examples. CFOs are using Gen AI to model complex financial scenarios, analyze working capital positions and detect anomalies in millions of transactions. Chief Product Officers are using it to automate RFPs, negotiate contracts with better visibility into supplier performance and model risk. Chief Operating Officers are embedding it into logistics, quality assurance and workforce planning. And getting feedback about it all in real time.

Nearly all (96%) executives at the enterprise level in our studies report favorable positive results, up significantly from even this time a year ago. That’s even though the technology is still in its very earliest innings of potential. And by everyone’s admission still has a long way to go.

Around the Enterprise Gen AI World in 18 Months

A year and a half ago, we saw enterprise execs using Gen AI with lower-value activities, producing better emails, crisper report summaries and more accurate meeting notes without the hassle of taking them. It was the most logical, low-risk entry point to get familiar with the technology.

In a relatively short space of time, today we find a 3x increase in enterprise executives saying Gen AI is highly effective for product development and improvement and a 40% increase in the share who find it highly effective for improving workflow management and internal processes. We also see a 28% increase in its effectiveness to manage and monitor cybersecurity activities.

We also find executives at some of the largest companies in the U.S. reporting a strong, or what I might even call compelling, positive impact of Gen AI across the enterprise, with 90% of enterprise chiefs citing a positive impact on their customer experience, and more than three quarters citing a positive impact on their competitive position.

When it comes to measuring the impact in the more classic dollars-and-cents terms, as I mentioned, 96% report a slight-to-strong ROI on their investments so far, with 28% reporting a very strong business case to support their Gen AI deployments. In just the last six months, the share who will increase their investments in the technology has more than tripled.

We also find that most enterprise executives have realistic expectations for when they expect positive payback from their investments. Not the millions and short-term payback chosen as success metrics by the MIT report. But something that is more aligned with the impact of Gen AI on the business, in ways that are harder to measure precisely right now.

More than eight in 10 of the more than 1000 enterprise execs we studied believe it could take between three and ten years, depending upon the use case and regulatory and compliance requirements, to generate a meaningful payback.

These enterprise executives also understand that big-“T” transformation doesn’t usually happen on a predictable timetable, nor with the expectation of an immediate or direct payback “in the millions.”

It didn’t happen with the commercial internet in the 1990s. It didn’t happen with eCommerce in the 2000s. And it didn’t happen with cloud technology in the 2010s.

It’s unrealistic to think that will happen with Gen AI today.

But that hasn’t stopped executives who see its enormous potential from getting their hands dirty and spending money on the tech. Rather than being stressed about the direct financial returns they may gain today, they’re more focused on what they might lose if they sit on the sidelines.

How Not to Fall into the Gen AI ROI Trap

Gen AI’s value is often the clearest when you look beyond the spreadsheet to outcomes that are sometimes too early to fully measure, but whose potential to reshape how entire industries operate is undeniably powerful.

Take Unlearn, founded by two physicists who saw a path to using Gen AI to reinvent clinical trials. For nearly a decade, they’ve used it to build digital twins that replace traditional placebo groups. Their first focus was on Alzheimer’s and ALS, diseases for which patients want the real deal, not sugar pills. With AI-generated placebos, more participants get the actual drug, trials finish faster and drug development costs drop by billions. The result is measured in life-saving therapies that reach patients years earlier and the scientific evidence and Big Pharma backing to support their claims.

Unlearn’s founders say that it’s the most significant change to trial protocols in 75 years. Yet under MIT’s framework, none of this would count toward ROI because the payoff is still a work in process, even as the impact today is extraordinary. Few would regard this as a  Gen AI failure.

Or consider Gen AI transcription in doctors’ offices.

Gen AI listens so physicians don’t have to type. Doctors can focus on their patients instead of their keyboards. Better conversations happen and subtler patient signals are picked up. Doctors reclaim time and reduce burnout while patients feel heard. The ROI isn’t only in hours saved or more patients that can be seen in a day. It’s in healthier outcomes and better patient/doctor relationships. It’s pretty hard to fit that into a direct-dollar-reduction-in-external-savings template. Yet anyone who’s ever spent time talking to the back of their doctor’s head during an examination clearly understands the value.

In the business setting, we see improvements in how Gen AI is strengthening cash forecasting across the important middle-market business segment, defined as those with between $50 million and $1 billion in annual sales.

PYMNTS Intelligence’s 2025/2026 Working Capital Index report, which will be published at the end of September and is based on a collaboration with Visa Commercial Solutions, finds that cash positions are more visible, forecasts are more accurate and working capital use cases are more strategic with the help of Gen AI. And that’s across the entirety of the nearly 1500 CFOs and Treasurers studied. For companies that live or die by how they manage liquidity, Gen AI is becoming the rising tide lifting all boats.

Then there is the next level of AI. Agents built to simplify complex transactions. Today, business often requires navigating endless workflows, systems and approvals. Agents collapse that into a conversation. Time saved. Friction reduced. Outcomes improved. By MIT’s definition, those gains wouldn’t count as ROI because they don’t immediately show up in million dollar savings on a balance sheet. But that’s the misstep. ROI here is speed, clarity and satisfaction. The advantages that business leaders understand create momentum and market share long before they hit the bottom line.

These aren’t isolated anecdotes.

Every major enterprise earnings call in Q2 included specifics about Gen AI, from sharper forecasting to more efficient procurement to productivity gains across operations to better risk management. Companies are using it to do more with fewer people, calling out headcount shifts and efficiency gains as tangible proof points. They are investing in Gen AI not as an experiment, but as the foundation for applying the most transformative technology in modern business history to improve how they do business.

Measuring the Return on Learning

The MIT study suggests that employees don’t understand Gen AI and that this contributes to the lack of a stronger ROI. But no one can master a transformative technology without using it, and a lot. The only way to understand Gen AI is to experiment, test and learn.

The irony is that the more transformative the technology, the harder it is to measure ROI in the moment. The internet didn’t prove its worth in quarterly sales. Cloud computing didn’t justify its price tag in its first quarter. The return came as companies built the infrastructure, skills and confidence to use them at scale.

Gen AI is no different. An important component of ROI is what companies learn by using it. Each experiment shortens the distance between analysis and action. Each iteration makes teams more fluent in how to work differently using it.

That’s why traditional ROI models collapse when applied to Gen AI right now. They assume stable inputs, when the technology is redefining what can be measured and how. They ignore the externalities from learning, including from mistakes.

Companies reporting the most success don’t see learning as they go as a waste of time or a drawback. They count it as an important part of the transformation process.

Managing What You Measure

Waiting for Gen AI to show a neat ROI before investing is like waiting for the internet to prove it could drive retail sales before building an eCommerce strategy. In the mid-1990s, online sales barely registered compared to brick-and-mortar stores. By traditional ROI math, eCommerce looked like a waste of time and money, and for years. But what mattered wasn’t the sales it generated in the short term. It was how it would rewrite the rules of retail.

Those who waited for online sales to be “big enough” to justify the investment found themselves chasing Amazon. Some still do. Many aren’t around anymore as foot traffic plummeted.

Cloud computing followed the same arc. In its early years, companies measured ROI by the cost savings of getting rid of servers. By that metric, the payoff looked modest. But the real return wasn’t a cheaper IT department. It was agility. With cloud, a startup could scale globally without buying hardware. Enterprises could launch new products without waiting months for infrastructure. Cloud made experimentation cheap, failure less costly and global scale possible in weeks, not years. Those who measured only cost savings missed the revolution happening under their noses.

Gen AI follows the same trajectory, although arguably much faster. Like eCommerce and cloud, it redefines the inputs that matter. The laggards will keep looking for a percentage savings in costs or headcount. The leaders will measure how fast their organizations can learn, adapt and build fluency. They see the return not in the pilot itself, but in the organizational muscle that comes from learning how to think and act differently.

When Everything Is a Nail

Managers like to repeat Peter Drucker’s line that what gets measured gets managed. But that only works if business leaders are measuring the right thing. The Soviet Nail Factory proved the danger of getting it wrong. Quotas by volume produced flimsy nails, quotas by weight produced nails no one could use. Metrics were met, but value was destroyed.

That’s why the leaders funding Gen AI aren’t hung up on short-term ROI. They see its edge strategically, not just tactically. They understand that the value right now isn’t visible on the spreadsheets that show a headcount reduction or a faster cycle time. It’s in building the capacity to think and act differently, something that’s much harder to measure quarter to quarter.

It’s the risk with dismissing Gen AI as hype. Measuring output is easy. The harder, more important task is knowing which inputs matter at the same time the rules of the game are being rewritten and the technology is evolving in real time. MIT’s insistence on millions in direct dollar savings is the modern equivalent of Gosplan’s quotas: an artificial target built on inputs that are still being defined.

But the real danger with Gen AI isn’t hype. It’s business executives misreading the inputs. It’s tempting to be skeptical, especially when Daron Acemoglu, a 2024 Nobel laureate in economics says that Gen AI, at best, will automate profitably only 5% of all tasks and have a relatively modest impact on GDP and productivity.  It’s easy to shrug off Gen AI as “LinkedIn hype,” just like one of MIT’s survey respondents was quoted as saying.

Fortunately, most executives have recognized the importance of investing and learning about Gen AI because they have seen enough evidence of its power to make them confident about its long-term importance for their businesses.

Those who don’t risk living in a future already designed by those who didn’t wait, using it to change how business gets done.

What’s your take?

Until NEXT time.

Join the more than 15,000 subscribers who’ve already said yes to what’s NEXT.

Image credit: Cartoon appeared in the Soviet magazine Krokodil in 1957.