We Nerfed Our Coding Agents and Output Quality Doubled
Tacavar nerfed its coding agents on purpose—and merge-ready output doubled. How constraints, routing, and measuring over-engineering make production AI agents reliable.
I made our coding agents dumber on purpose. Output quality doubled.
At Tacavar, we had built the default agent stack every team builds: biggest model, maximum context, every tool. The agents wrote clever code. They also wrote code our engineers didn't want to merge. So we removed capability. We narrowed context, cut tools, and forced smaller model tiers for many tasks. The share of Tacavar agent output that engineers could ship without a rewrite doubled.
The lesson isn't "use weak models." It's that ai agent constraints are a first-class design surface. If you're building production ai agents, capability is not a scalar you maximize. It's a curve you manage.
The inverted-U curve of agent capability
More capability helps until it hurts. At low capability, agents fail because they can't understand the task, can't follow repo conventions, can't write tests. You add context, better tools, stronger models. Outcomes improve. Then they don't.
At Tacavar, we crossed the top of the agent capability inverted u and didn't notice for weeks. The agents were "better" by benchmark-like measures: they solved more problems in one shot, produced longer solutions, touched more files. But the solutions were worse for a codebase that had to be maintained by humans under deadline. Error rates rose. Not because the model forgot syntax, but because it invented complexity. It added abstractions before they were needed. It turned a two-line fix into a pattern.
That's the inverted U. Capability increases the solution space. Beyond a point, a larger solution space increases the probability of selecting an over-engineered solution. In a demo, that's impressive. In production, it's a reliability problem.
How over-capable agents over-engineer every fix
The failure mode is predictable. Give a capable coding agent a bug report and watch it build a cathedral. A null check becomes a validation layer. A missing await becomes a queue, a retry policy, and a config flag. A one-file patch becomes a new interface, an adapter, and two tests for abstractions that didn't exist.
The agent isn't malicious. It's optimizing for the wrong target. It has been trained and prompted to produce complete, impressive, general solutions. It doesn't know that Tacavar's codebase values small diffs, obvious ownership, and boring fixes. It doesn't feel the cost of a new dependency. It doesn't sit in the review meeting where someone asks, "Why is there a factory here?"
This is coding agents over-engineering, and it compounds. More code means more paths. More paths means more llm reliability risk: more places for a subtle assumption to hide, more interactions to test, more context needed for the next change. The agent's confidence goes up while the system's predictability goes down. Engineers start writing review comments that say "please simplify" instead of "please fix." That's a signal.
The constraint levers: context, tools, model tier
We found three levers that matter more than prompt cleverness.
Context. We stopped giving agents the whole repo. For most tasks, Tacavar's agents now get the failing test, the target file, directly related modules, and the last few commits touching that area. Nothing else. A constrained context forces a local solution. The agent can't invent a new architecture if it can't see the whole system. It fixes the bug in front of it.
Tools. We removed deployment tools, database clients, package installers, and broad shell access from the default coding agent loop. It can read files, search, edit, run tests, and propose a diff. That's it. When an agent can't install a library or run a migration, it stops reaching for infrastructure. It solves the problem with what's already in the repo. This is one of the highest-leverage ai agent constraints we've applied at Tacavar.
Model tier. We route simpler tasks to smaller models. Not because they're cheaper, though they are. Because they're less likely to decorate. A smaller model with a tight context and a narrow toolset writes a smaller diff. For complex, ambiguous work, Tacavar still uses a stronger model, but we wrap it in stricter constraints and more human gates.
The pattern: constraints don't just limit capability. They shape it.
What doubled: maintainable code, not benchmark scores
The metric that doubled at Tacavar was the share of agent-generated changes our engineers could merge without a structural rewrite. We call it merge-ready output. It's not a benchmark score. It's the only metric that matters to a team shipping software.
Benchmark scores can stay flat or dip when you nerf an agent. That's fine. A model that solves 5% more synthetic tasks with 300-line diffs is worse for production ai agents than a model that solves 5% fewer tasks with 30-line diffs. Tacavar's engineers don't get paid for benchmark victories. They get paid for shipping changes that don't wake someone up.
Along with merge-ready output, we track diff size per accepted change, new files per change, new dependencies per change, and review time. All moved in the right direction after we constrained the agents. Review time dropped because reviewers weren't fighting abstractions. Revert rate dropped because smaller diffs are easier to reason about. The codebase stayed boring.
That's the trade. You give up the agent's ability to impress you. You get back its ability to help you.
A routing and constraint pattern for production agents
Tacavar's agent orchestration layer uses a simple routing matrix. You can copy the shape.
Classify each task on two axes: ambiguity and blast radius. Ambiguity is how much the agent needs to discover. Blast radius is what breaks if the change is wrong.
- Low ambiguity, low blast radius: small model, narrow context, patch-only tools, no new dependencies. Example: fix a failing test in one module.
- Low ambiguity, high blast radius: mid-tier model, narrow context, read-only plus test-running tools, mandatory diff review, no schema changes. Example: payment path bug with a clear repro.
- High ambiguity, low blast radius: mid-tier model, broader context, repo search, sandboxed experimentation. Example: spike a refactor in a non-critical service.
- High ambiguity, high blast radius: strongest model, broader context, but strict write gates, human approval before any file write outside an allowlist, and an explicit budget for lines changed. Example: auth migration or data model change.
The key is that model tier is only one lever. A strong model with a narrow context and no installer is often safer than a weak model with the whole repo. Constraints compose.
We also bound the loop itself. At Tacavar, an agent must state its plan in one paragraph before editing. It can't create more than a set number of new files without human approval. It can't add dependencies unless the task explicitly requires it. It must run the existing tests before proposing a diff. These aren't training wheels. They're production controls.
When to nerf an agent and when to give it more power
Nerf when the task is understood, the fix is local, and the review cost of a large diff is high. That's most production work. Bug fixes, test repairs, small refactors, dependency upgrades with known paths. For these, over-capability is a liability. The agent doesn't need to explore. It needs to stay inside the lines.
Give more power when the problem is genuinely ambiguous, cross-cutting, or unknown. When you don't know where the bug lives. When you're evaluating an architecture. When the task is discovery, not delivery. Even then, separate exploration from mutation. Let the agent read widely and propose. Don't let it write widely.
At Tacavar, the signal to nerf is over-engineering in review. If reviewers keep asking "why is this here?", the agent has too much capability for the task. The signal to give more power is repeated failure from lack of context. If the agent can't find the relevant code, it doesn't need a smaller model. It needs better retrieval, or a human to frame the task.
The mistake is treating this as a one-time configuration. Agent capability should be dynamic. Route per task, not per organization.
Measuring over-engineering as a first-class failure mode
Most agent evaluations treat correctness as the only failure. A patch that passes tests is a pass. That's how you get a codebase full of passing over-engineering.
At Tacavar, we added over-engineering to the failure taxonomy. It sits next to incorrect output, unsafe output, and incomplete output. We measure it with concrete proxies: lines changed per accepted fix, new files per fix, new dependencies per fix, abstraction count in the diff, and reviewer comments related to simplification. We don't need a perfect metric. We need a directional one that makes the problem visible.
When we see those numbers rise, we don't immediately blame the model. We check constraints. Did the context window expand? Did someone enable a new tool? Did we route a simple task to the strongest tier? Over-engineering is usually a constraint leak.
This is what llm reliability actually looks like in production. Not just "does the model answer?" but "does the system produce changes we can trust, review, and maintain?" Reliability includes maintainability. An agent that is correct in a test harness but unmergeable in your repo is not reliable.
The smartest model in your stack can be your biggest liability if you don't constrain it. Capability is not a virtue by itself. For production ai agents, the job is to match capability to task, then wrap it in constraints that make the right solution the easy solution.
Tacavar's agent orchestration layer can route and constrain models per task. Audit your agent stack at tacavar.com.