If you’ve used ChatGPT or Claude, you know they can type back an answer. It’s like having a really smart assistant who can explain things, write emails, or even draft code snippets. But what if that assistant could do things, not just talk about them? That’s where a newer kind of AI comes in: AI agents that act.
Imagine you need a new piece of software built, like a customer feedback board. Instead of just asking an AI to describe how to build it, you could ask an AI agent to actually build it. This sounds powerful, right? And the thinking has been that to make these agents better, we just need to give them more capabilities – like giving them a calculator for math problems or a spellchecker for writing.
But what if that’s not the whole story?
A recent study looked closely at this idea. It asked AI agents to build the same application, a real-time retrospective board, 90 different times. The goal was to see what actually makes these AI builders reliable, especially when they need to get the code right on the first try.
Here’s the surprising part: simply giving the AI more tools, like a browser-based testing tool or fancy instructions, didn’t make it much better at building functional software. In fact, one of the testing tools actually increased the cost by up to 68 percent without improving the code’s reliability. That’s a hefty price for something that doesn’t help.
This might feel counterintuitive. We tend to think more tools equal better results, like a chef with a bigger knife collection. But for AI building software, the research suggests this isn’t always true.
Why this matters: For business leaders, this means we need to look beyond just the shiny new capabilities being added to AI. The real value might lie in something more fundamental: how well the AI can think.
The study found that the biggest predictor of success wasn’t the number of tools the AI had access to, but rather how much “reasoning effort” it put in. Think of reasoning effort as the AI’s equivalent of deep thinking or careful planning. It’s the AI’s internal process of figuring out the best way to solve a problem before it even starts writing the code.
When the AI agents were encouraged to spend more time reasoning, their ability to produce correct code on the first attempt shot up dramatically. Runs that were set to a higher reasoning effort level got it right 89 percent of the time, compared to 28 percent when the effort level was lower. That’s a massive jump in reliability.
This is like asking a junior developer to build something complex. If you just give them a manual and a toolbox, they might struggle. But if you first spend time with them, sketching out the plan, talking through potential issues, and making sure they understand the goal deeply, they’re much more likely to succeed without needing constant corrections.
The study highlighted that most of the first-try failures in code generation weren’t due to a lack of checking tools. Instead, they stemmed from weak reasoning by the AI. The AI simply didn’t “think” through the problem thoroughly enough.
This is a critical distinction. Most of the time, when AI agents fail to generate correct code, it’s not because they missed a bug that a testing tool would have caught. It’s because their initial plan or understanding of the task was flawed.
So, what’s the practical lesson here? It’s about matching the fix to the failure. If the problem is weak reasoning, then you need to improve the AI’s thinking process.
The catch: This doesn’t mean tools are useless. They have their place, but they are not the primary driver of first-try reliability in complex tasks like code generation.
The research focused on building a real-time retrospective board. This is a common type of application used in project management, where teams can share feedback and ideas asynchronously. Ninety independent AI runs were conducted, each attempting to build this application from a single, detailed specification.
The AI runs were scored on a fixed 14-criterion functional rubric, leading to a maximum score of 42 points. A visual quality review was also performed. The runs involved different AI model generations, two different agent harnesses (the systems that run the agents), varying reasoning effort levels, a testing tool, and two types of design-oriented prompts.
One of the most striking findings was about container deployment. This is a standard way to package software so it runs reliably anywhere. Container deployment was the biggest single reason for failure, with the AI failing to get it right on the first try in 44 percent of all runs. This failure rate shifted significantly between different AI model generations, even though the average scores across all criteria didn’t move much.
The testing tool, which was supposed to help catch these kinds of issues, surprisingly didn’t improve functional scores or reliability. It just made things more expensive.
The directive to increase reasoning effort, however, had a profound impact. When AI agents were prompted to put in more “xHigh” reasoning effort, their success rate for perfect first-time runs jumped from 28 percent to 89 percent. They also needed about five times fewer corrective prompts. This came with a cost increase of only 9 to 29 percent, which is a bargain for such a huge leap in reliability.
Interestingly, a design-oriented prompt that aimed to improve the visual quality of the application did just that. It raised the visual quality score from 3.0 to 4.5 on a 5-point scale. But it didn’t lift the functional score at all. Even more peculiar, a simple one-paragraph paraphrase of the prompt’s directive reproduced the entire visual quality improvement. This suggests that sometimes, the specific wording and clarity of instructions can be more impactful than complex, built-in features.
The practical lesson is clear: When building software with AI, focus on improving the AI’s ability to reason and understand the task deeply. This is far more effective than simply piling on more tools, especially when those tools don’t address the root cause of failure.
So, what does this mean for your business? If you’re exploring AI for tasks like code generation, customer service automation, or complex data analysis, it’s worth considering the AI’s “thinking” process.
Are you giving the AI clear, detailed instructions that encourage it to plan and reason? Are you using models known for their strong analytical capabilities, rather than just the ones with the most features?
The study, conducted by Achint Mehta, suggests that the frontier models (the newest and most advanced AI) clustered near the highest scores, while a lower-cost local model scored significantly lower, between 24 and 37 points. This indicates that the underlying model architecture and training play a huge role.
It’s a bit like hiring people. You can give a brilliant, experienced architect a basic set of tools and they’ll likely build something amazing. Give a less experienced person the most advanced tools in the world, and they might still struggle.
The implication for decision-makers is to invest in understanding the AI’s reasoning capabilities. This might mean choosing specific AI models, crafting better prompts, or even investing in training AI to be more methodical. The cost savings from avoiding rework and failed deployments could far outweigh the initial investment in better AI reasoning.
This is a shift from thinking about AI as a passive tool to be equipped, to thinking about it as an active collaborator that needs to be guided and optimized for its cognitive processes.
The study, which involved 90 independent agent runs, provides concrete numbers. For example, container deployment failed first try in 44 percent of runs. The testing tool raised cost by 42 to 68 percent. But raising reasoning effort from High to xHigh cut corrective prompts about five fold. These aren’t abstract concepts; they are measurable outcomes.
The key takeaway is that when AI agents are building software, the effort they spend thinking is what truly buys reliability, not just the number of extra features or tools they have access to. It’s about quality of thought, not just quantity of capability.
Sources: arxiv.org