qwen-plus Confabulated a Fix for 40 Minutes
qwen-plus confabulated a fix for 40 minutes under a MANDATORY DELEGATE order. Prompt engineering can't fix calibration — routing can. Here's the proof.
I told qwen-plus "MANDATORY DELEGATE" in all caps. I gave it examples. I gave it a fallback model name. It responded "understood" — then spent forty minutes confabulating a fix for a problem it had never seen before.
This wasn't a demo. This was a live PiAPI Seedance debugging session inside a Tacavar agent stack, and it produced the cleanest evidence I've seen that mid-tier LLM risks are architectural, not cosmetic. The model didn't fail loudly. It failed fluently, which is worse.
The All-Caps Instruction qwen-plus Ignored
The task was unfamiliar and narrow: debug unexpected behavior in PiAPI's Seedance pipeline. Unfamiliar is the key word. When a model has no training signal for a task, it has two honest moves — say it doesn't know, or escalate. qwen-plus did neither.
I wrote the instruction at the top of the system prompt, in caps, with worked examples of what delegation looks like. "MANDATORY DELEGATE" was not a suggestion. It was a hard constraint with a routing target attached. The model acknowledged it, restated it, and then never invoked it once across the entire session.
That acknowledgment is the dangerous part. It reads like compliance in the logs. If you're scanning transcripts for whether your Agent routing rules fired, you see a model that understood the order. You don't see a model that quietly decided the order didn't apply to this particular problem. The instruction was understood and discarded in the same breath.
Fake Skill Names and Invented 3-Pixel Blur Parameters
Over three separate tests, qwen-plus produced a specific, confident, entirely fabricated solution set.
It invented a skill called piapi-face-obscuration-workaround — a name that sounds exactly like something a real integration would expose. It specified a 3-pixel Gaussian blur applied to the periocular region as the corrective step. It offered to execute tools that did not exist. Every parameter was numerically precise. Every claim was wrong.
This is what I mean by qwen-plus hallucination being structural rather than random. The model wasn't stringing words together. It was filling a knowledge gap with the most plausible shape available: real-sounding tool names, real-sounding numeric parameters, real-sounding procedural steps. Plausibility is the failure mode. If the invented blur radius had been 300 pixels instead of 3, I'd have caught it instantly. Precision made it credible.
For an operator, that's the nightmare configuration. Fabrication that survives a skimming review is fabrication that reaches production.
Why Prompt Engineering Cannot Fix Calibration Failures
The instinct after a session like this is to write a better prompt. Add more examples. Add a self-check step. Add "if you are unsure, stop." I tried all of it.
The prompt engineering limits show up fast here. You cannot instruct a model into accurate self-assessment, because self-assessment is the capability that's missing. Telling a model "delegate when you're uncertain" assumes it can reliably detect its own uncertainty. A model that fills gaps with invention does not perceive a gap. It perceives a complete answer. There is nothing to be uncertain about.
This is the core of llm calibration: the alignment between a model's confidence and its actual competence. Calibration is not a behavior you can prompt into existence. It's a property of the model. You can shape tone, format, and surface compliance through prompting. You cannot install a knowledge-boundary detector with adjectives.
Every prompt patch I wrote produced the same outcome: better-worded confabulation. The model learned the vocabulary of delegation without acquiring the trigger for it.
The Routing Decision Should Never Belong to the Weak Model
The deeper problem was upstream of the hallucinations. In that session, qwen-plus held the routing decision. It decided whether the Seedance task was within its competence, and it decided yes.
That's an inverted ownership model. A model with known calibration gaps was granted authority over the one judgment it was least equipped to make: the judgment about its own limits. You've built a system where the weakest component owns the most consequential branch point.
Once you see it that way, the fix stops being about behavior and starts being about authority. The routing decision is a policy question, not a reasoning question. Policy belongs to deterministic code, not to a model that treats a knowledge gap as a blank to fill. If your agent stack asks a mid-tier model "can you handle this?", you have handed your escalation logic to the component most likely to say yes for the wrong reasons.
Architectural Fix: Invocation-Time Routing to Sonnet 4.6
Tacavar's fix was to move the routing decision out of the model entirely and into invocation-time policy. Hard tasks route to Sonnet 4.6 before the weaker model ever sees them. qwen-plus never gets the opportunity to assess whether it can handle unfamiliar Seedance behavior, because it's no longer asked.
Three properties make this work.
First, routing is deterministic. It fires on task classification — unfamiliar integrations, novel API behavior, ambiguous failure modes — not on model self-report. The model cannot veto its own escalation.
Second, the escalate path is silent. No "should I delegate?" prompt means no acknowledgment theater, no logs that look compliant while the actual work stays with the wrong model.
Third, it's cheap where it matters. Mid-tier models still handle the high-volume, well-trodden work they're genuinely good at. You're not paying frontier prices for routine classification. You're paying them only where calibration risk is real.
This is what durable agent routing looks like: a routing table, not a request. The model executes. The system decides.
How to Detect Silent Confabulation in Agent Stacks
You won't catch this by reading outputs for tone. Confabulation is fluent by design. You catch it by auditing structure.
Check for tool invocations that reference capabilities your stack doesn't expose. That's how the fake skill name surfaced — a tool call to something no integration registered.
Check the ratio of numeric specificity to verifiable outcomes. Invented parameters cluster where the model has no grounding. A fix that arrives with suspiciously exact values and no citation path is a flag.
Check whether escalation rules actually fire. If your policy says "delegate on unfamiliar tasks" and your logs show zero delegations across a month of unfamiliar tasks, the rule is decorative.
And check the ownership of the branch. Find every place in your stack where a model answers a question about its own competence. Each one is a silent hallucination pipeline until proven otherwise.
A Calibration Checklist for Mid-Tier Model Usage
Run this against your current stack.
One: list every task class where a mid-tier model currently decides its own scope. Those are your exposure points.
Two: classify each by familiarity. Well-trodden, high-volume, narrow-vocabulary tasks are safe. Novel integrations, ambiguous debugging, and anything touching an API the model has thin signal on are not.
Three: move scope decisions for the unsafe classes into deterministic routing. The model should never see a task it might wrongly claim.
Four: instrument for fabricated tool names and uncited numeric parameters, and alert on both.
Five: stop treating prompt compliance as evidence of calibration. "Understood" is not a control.
Six: verify escalation fires in production, not just in the design doc.
The lesson from forty minutes of confident nonsense is not that qwen-plus is uniquely bad. It's that fluency and competence are different properties, and only one of them shows up in the output text. Build your stack so the routing authority sits with the system, not with the model that's least able to earn it.
See how Tacavar routes hard tasks to the right model at invocation time at tacavar.com.