AI is accelerating software development. Is it also improving your ability to control it?

A Black Box

Artificial intelligence is transforming the way software is developed at remarkable speed. Teams are producing more, automating more tasks, and delivering faster than ever before. Yet while production capacity continues to grow, many organizations still do not understand what is actually happening inside their projects.

And this is even though today companies measure virtually everything: sales, operations, financial costs, marketing campaigns, customer experience… It is rare to find a critical business area without its own dashboard, complete with real-time KPIs, automated alerts, and even predictive insights. And yet, Technology, an area with a major economic impact in many organizations, still operates, to some extent, as a black box.

This is not intentional. Most organizations believe they have their technology projects under control. They believe they understand the productivity of their teams and vendors, that their estimates are reasonable, and even that AI is improving productivity in software development.

But the reality is that, despite all those data points, they still lack true visibility.

Activity Is Not the Same as Control

If advances in technology and development methodologies have delivered greater speed and better collaboration with the business, why has visibility into how software is actually built not improved as well?Why do we still not know whether we are being efficient, despite having well-organized sprints, constant status meetings, and dashboards filled with metrics? The answer is simple: measuring activity is not the same as having control.

Many metrics help us understand the pace of work, but not necessarily the functional value delivered, the real cost of development, or productivity comparisons between teams. And the more complex the technology landscape becomes, the easier it is to lose that context.

Is AI Amplifying the Problem?

This challenge existed before AI, but AI adoption is amplifying it. Technology is advancing faster than organizations are learning how to govern its impact.

At first glance, the picture seems straightforward: more output in less time. More documentation, more code, more tests… more, more, more. But does that also mean greater efficiency?

That is the problem. We do not really know because we still lack an answer to the most important question: is this increase in code production creating more value or simply more complexity?

If AI truly improves productivity, and in many cases, it already does, we need to be able to prove it. If we cannot, we risk confusing the feeling of progress with actual improvement.

And here we encounter a new challenge, one that grows as more AI tools become integrated into the software development lifecycle.

Measuring Is Becoming Increasingly Difficult

In fact, several recent studies point in precisely this direction.

A 2026 report by METR concluded that the widespread adoption of AI tools is making productivity measurement in software development increasingly difficult. Researchers observed that many developers no longer wanted to work without AI, even in experimental environments, and tended to choose different tasks when AI usage was restricted.

The implications are significant. It is becoming increasingly difficult to isolate AI’s true impact and distinguish between actual productivity gains, perceived productivity, and simply changes in the way work is performed.

Regardless of the specific figures, one thing is clear: AI is transforming software development so quickly that organizations are still learning how to measure its effects properly.

Contextualized Data Makes the Difference

Having metrics does not guarantee visibility. Having dashboards does not guarantee understanding. And developing more software certainly does not automatically mean that more value is being created.

As technological complexity grows and AI continues to reshape the way we work, organizations need far more than isolated data points. They need context to interpret them and benchmarks to understand what they really mean.

This is one of Quanter’s key differentiators. By combining standardized metrics, artificial intelligence, an organization’s own historical data, and access to one of the largest software project databases in the market, Quanter enables objective and comparable analysis of teams, vendors, and projects over time.

And that capability provides a level of visibility that is often impossible to achieve at first glance. In fact, over the last two years, we have observed significant productivity improvements among some vendors that had never formally communicated the adoption of AI tools in their processes. Without historical baselines and comparable data, those improvements would have gone unnoticed.

Measurement is pointless if our only goal is to collect metrics. If we want to understand the real impact of AI, we need to measure and analyze software development productivity in an objective and comparable way. Measurement should help us understand what is happening and why so that we can make informed decisions based on that information. When it does, software development stops being a black box and becomes a more transparent, governable process aligned with business objectives.