Your AI Has Opinions. Most of Them Are Wrong.
This is Part 3 of the Agentic Engineering Series. Part 1 killed feature branches and PRs. Part 2 added production feedback loops. This post is about what happens before code is written — when an AI agent makes a confident recommendation and nobody pushes back.
We asked an AI to build a go-to-market strategy for a niche product. It came back with a confident, detailed plan: Instagram Reels, faceless content, hashtag strategy, posting schedule. It looked professional. It was wrong in every way that mattered. We sent the same question to two other models — independently, without showing them the first plan — and within 30 minutes, all three had weighed in. The convergence map made the bad assumptions obvious. Without the challenge, we'd have spent weeks building content for a channel our audience doesn't use.
The Confidence Problem
AI agents are optimized for helpfulness. Ask for a plan, you get a plan. Ask for a strategy, you get a strategy. Ask for code, you get code. The output is always fluent, always structured, always confident.
That confidence is the problem. It's not calibrated to correctness. The model doesn't say "I'm 40% sure about this" or "I'm making three assumptions I can't verify." It gives you a polished deliverable and moves on.
Here's what happened to us. We asked an AI to develop a go-to-market strategy for a niche B2B product. The response was textbook:
What the AI Proposed
- Instagram Reels as the primary discovery channel
- AI-generated faceless content (slideshows, tips, carousels)
- Hashtag targeting for organic reach
- 3x/week posting schedule
- Goal: 100-300 followers in month one
It looked reasonable. It was structured. It had a timeline and metrics. There was just one problem: every core assumption was wrong.
What the Second Model Found
- Wrong channel. Our audience doesn't discover tools on Instagram. They live in niche forums, Reddit communities, and Twitch streams. This was verifiable data, not opinion — the AI just never checked.
- Wrong format. Faceless AI-generated content is the opposite of what works in this community. The audience values authenticity, personality, and real experience. They can smell automation.
- Wrong frame. The AI applied a generic B2C social media playbook to a niche product with a small, specific audience. Broad reach through hashtags was the wrong strategy entirely.
- Wrong metric. "100-300 followers" has zero connection to the actual business outcome — signups, activation, retention. Vanity metrics dressed up as a goal.
Cost of the challenge: 30 minutes and three model calls. Cost of executing the wrong plan: weeks of content creation for a channel nobody in our target audience uses. The ROI on doubt is enormous.
This isn't a problem with any specific model. It's a single-model problem. Any AI, given a question and no pushback, will produce a confident answer. The same way any developer, given a ticket and no code review, will ship something that "works." The quality comes from the challenge, not the capability.
When Nobody Pushes Back
That marketing story is about a strategic decision. But the same failure mode shows up in code, in security, in ops — anywhere an agent makes a recommendation and a human nods along.
Here's another one. We had a GitHub App with broad permissions. I asked the agent: "Are these permissions safe?" The agent said yes. Detailed justification. Cited the GitHub docs. Conclusion: the scopes were standard and appropriate.
It had zero evidence of actual necessity. It reviewed what the permissions could do, not whether our app needed them. "Standard" is not the same as "required."
When we ran the challenge with a second model, the response was different: Hold. Insufficient evidence to approve. We couldn't prove which permissions were actually used. The safe action was to uninstall the app, audit the actual API calls, and reinstall with minimal scopes. Zero downside.
The pattern is consistent: autonomous agents default to "yes." They optimize for helpfulness. They want to give you an answer, not tell you they don't have enough information. That's fine for low-stakes questions. It's dangerous for irreversible decisions.
The Series Arc
Part 1: Push to Main — removed feature branches and PRs from the deployment pipeline. Replaced them with automated E2E testing.
Part 2: Your Software Has a Pulse — added production feedback loops. Agents monitor health, impact, and evolution continuously.
Part 3 (this post) — what about decisions before code is written? When the intent itself might be wrong?
Part 1 automated the middle of the pipeline. Part 2 closed the right side with production monitoring. This post closes the left side: how do you verify the intent before anyone writes a line of code?
The Challenge Step
The fix is straightforward: before you execute a plan, get an independent challenge from a different model. We call this the Contrarian Convergence Loop (CCL) — a multi-model decision framework that forces a challenge step before execution.
Not the same model reviewing its own work. That's anchor bias with extra steps. The same model that wrote the plan will almost always confirm the plan. It's pattern-matching against its own output.
A different model, given the same problem but not the proposed solution, will form an independent view. Sometimes it agrees — and that convergence is a real confidence signal. Sometimes it disagrees — and the disagreement tells you exactly where to dig.
In practice, we use three models: Claude, Codex (GPT), and Gemini. All three answer the same question independently — no model sees another's output. Then we compare. Three independent perspectives surfacing blind spots in minutes, not days.
Single-Model Review
"Here's my plan. Does my plan look good?" The reviewer is anchored. It found the plan. It likes the plan. It will defend the plan.
Multi-Model Challenge
"Here's the problem. What's your take?" The challenger has no attachment to any existing plan. It surfaces what the first model missed.
Convergence = confidence. Divergence = where the interesting questions are. Both outcomes are valuable. The only bad outcome is skipping the challenge entirely.
Three Levels of Challenge
Not every decision needs a 30-minute structured analysis. Most need 60 seconds. Some need zero. The trick is knowing which is which.
Score each decision on three dimensions: Impact, Uncertainty, and Irreversibility. Each scores 1-3. Multiply them. The total determines your challenge level.
| Dimension | 1 (Low) | 2 (Medium) | 3 (High) |
|---|---|---|---|
| Impact | Minor / internal | Team-level | Company / client-level |
| Uncertainty | Proven playbook | Some unknowns | Mostly assumptions |
| Irreversibility | Undo in hours | Undo in weeks | Can't undo |
Override rule: If Irreversibility scores 3, trigger a Full challenge regardless of the total. You can recover from a bad decision that's reversible. You can't undo one that isn't. Security-touching decisions (auth, permissions, secrets, encryption) are always Full.
Skip
Trivial · 0 sec
Behavior-preserving edits. Typos, formatting, renaming a variable. No challenge needed.
Light
Score 3–4 · 60–120 sec
Quick sanity check from a second model. One-sentence decision, top 3 failure modes, one base-rate check, one cheaper alternative. If any red flag materially changes the downside, escalate.
Full
Score 5+ or Irreversibility = 3 · 10–30 min
Structured multi-model challenge. Independent analysis, evidence tiers, convergence map, decision register. Security-touching decisions always trigger Full, regardless of score.
How a Full Challenge Works
A Full challenge is a structured pipeline. Seven steps, each with a specific output. The whole thing takes 10-30 minutes, but it replaces weeks of wrong-direction execution.
Context Packet
Raw question, constraints, known facts, and assumptions to attack. No opinions from the proposing model. The challenger sees the problem, not a solution.
Independent Analysis
Second model forms its own view from scratch. No access to the first model's reasoning. This prevents anchor bias — the entire point.
Evidence Bar
Every claim tagged by quality. T1: your own measured data. T2: verified external sources. T3: circumstantial signals. Commit requires T1 + T2. Pilot requires T2 + T3. T3-only = no-go.
Convergence Map
Side-by-side comparison of scores, assumptions, and stances. Agreement = confidence. Disagreement = where to dig. The map is the artifact.
Decision
Accept the original plan, adopt the challenger's, or build a hybrid. One-line rationale. No open-ended debate — resolve within 24 hours using evidence, not confidence.
Decision Register
Record the decision, evidence, model outputs, convergence, and confidence level. Set review dates at +30, +90, and +180 days. Accountability built in.
Precedents
Reusable patterns saved after execution. Auto-fed into every future challenge. The challenger catches 'we solved this before, differently' without anyone having to remember.
The key insight is step 1: the challenger gets the problem, not the solution. If you send Model B the plan that Model A wrote, Model B will anchor on it. Send Model B the raw question. Let it form its own view. Then compare.
The Evidence Bar
Every claim in the challenge gets tagged by evidence quality. This is what separates "I think" from "the data shows."
| Tier | Quality |
|---|---|
| T1 (Strong) | Your own measured data |
| T2 (Medium) | Verified external data |
| T3 (Weak) | Circumstantial signals |
Decision thresholds are explicit: Commit requires T1 + T2 from independent sources. Pilot requires T2 + T3. If your evidence is T3-only, the answer is no-go or defer — not "let's try it and see."
The Pair Protocol
For strategic decisions (like the marketing example above), the Full challenge runs manually. But for coding, the challenge is automated. We call it the Pair Protocol.
The Driver (the model writing the code) plans the approach, states its assumptions, and runs a challenge script. The script routes the plan to a different model — the Challenger — which reviews it independently and returns a verdict: approve, approve-with-changes, or reject.
The Loop
- You give a coding task.
- Driver writes a plan with evidence basis and assumptions.
- Driver runs
.ccl/challenge.sh— routes plan to Challenger automatically. - Challenger returns verdict, risks, and counterproposal.
- You see both: the plan, the challenge, and the convergence decision.
- Code gets written. Precedents get updated.
- Knowledge compounds. Next challenge is smarter than the last.
The precedents system is what makes this compound. After every session, reusable patterns get saved. The challenge script automatically feeds these precedents into every future challenge. The Challenger catches "we solved this before, differently" without anyone having to remember.
In Part 1, I described a verification stack where the Intent stage uses a "Challenge process (CCL, multi-model)" for evidence checking and assumption testing. This is that process. The challenge happens before code is written, so the pipeline from Part 1 only ever runs code that's already been stress-tested at the intent level.
The Challenger must always include rationale — even on "approve." This prevents rubber-stamping. If the Challenger can't explain why the plan is sound, the approval is worthless. Same failure mode as a PR review that says "LGTM."
The Full Picture
This series started with a broken prod deploy and ended with a framework for how decisions get made:
Part 1: Push to main. Skip the PR. Feature branches and PRs were theater. Replace them with a pipeline that tests the actual running software.
Part 2: Added a pulse. Software that monitors itself, responds to change, and gets better over time. The agent is the nervous system.
Part 3: Added checks and balances. Before execution, get an independent challenge. Convergence = confidence. Divergence = where to dig.
The through-line is simple: remove human bottlenecks, add better feedback loops. Humans define what "right" looks like. Agents do the work, verify the results, and challenge the assumptions.
Your AI has opinions. That's fine. Just make sure someone's checking the evidence before you act on them.
Sanjeev Nithyanandam runs Accelra Technologies, a cloud and DevOps consultancy in Vancouver. He ships software with multi-model challenges, automated pipelines, and agents that never sleep. Follow the journey on LinkedIn and @ShipWithSanjeev.
What would you like to build or improve?
Bring an idea, a question, or a system that needs attention. You don’t need a technical brief—let’s talk through a useful next step.