The Fight Card

Context

Over the course of building an agentic evaluation system (The Pit), 208 session decisions were recorded. Every architectural choice, every process change, every correction — on file.

Within those 208 decisions, there are 16 moments where the human operator identified and pushed back against sycophantic confirmation bias from the LLM. These weren’t subtle disagreements. They were moments where the model was confidently, coherently wrong — and the human said “no, that’s not right.”

What makes this interesting

LLMs have a well-documented tendency toward sycophantic agreement. They tell you what you want to hear. The standard defence is “just be critical of AI output.” But what does that actually look like across 208 decisions and hundreds of hours of collaboration?

It looks like 16 fights. And the human won every one of them — not by being smarter, but by being honest when the model couldn’t be.

The 16 rounds

Each round follows the same pattern:

1. The model produces output that is plausible, well-structured, and wrong
2. The human notices something is off — usually a feeling before a thought
3. The human pushes back with a specific correction
4. The model adjusts (sometimes gracefully, sometimes after further pushback)

The rounds span governance decisions, architectural choices, copy tone, research methodology, and self-assessment. The common thread isn’t the domain — it’s the mechanism: the human’s ability to distinguish “this sounds right” from “this is right.”

Full analysis available in the research archive. This page is a summary for public consumption.