Two agents agreeing is one observation

Chain a few agents together and the arithmetic gets unpleasant fast. Three steps at 70% each is 34% end to end. And 70% is generous: measured on SWE-bench Verified, GPT-5 submitted a patch on 100% of runs and resolved 44%. Put three of those in a row and you are under 10%.

The standard response is to add a checker. Have a second agent verify the first.

It helps far less than it appears to, and the reason is worth understanding before you spend money on more agents.

Agreement is not corroboration

Two agents reporting the same number is not two observations. If they used the same method, it is one observation reported twice, and the second report adds no information while roughly doubling your confidence. That is the worst possible trade.

I watched this happen. Two sessions independently reported the same progress figure for a large batch. Both were wrong, in the same direction, by the same mechanism: both had sampled the top of a table.

Those record ids were creation-ordered. The top of the table was the oldest rows, enriched during earlier runs. Both agents were asking “did today’s work succeed?” against a sample that contained none of today’s rows.

That is not bad luck, it is structural. Neither sample could have contained a counterexample. The check was incapable of returning bad news, and would have kept agreeing with itself indefinitely.

Self-referential checks cannot see absence

Here is the version of this that fooled me longest.

I had a QA suite on a data batch. Row counts matched one to one. Zero duplicates. Zero overlap between segments. Valid JSON throughout. Every field inside its size limit. All green.

The batch was missing about ten thousand rows.

Every one of those checks takes the artifact’s own rows as its universe. One to one compares my file to my file. Not a single check could observe a row that was not there. It did not feel like cutting corners, it felt like diligence, and it was diligence, pointed entirely inward.

Absence is invisible to any check that starts from what is present. So the question a review has to answer is not “is everything here correct” but “what would show me something that should exist and does not”. If no check answers that, you have no coverage for the largest failure class there is.

The fix is one line of principle: at least one check must be anchored to an independent upstream count. Never the artifact against itself. A reader’s output versus the raw source line count. A live record pull versus the exported file. Two numbers derived different ways that have to agree. When those disagree by exactly one row, you have found a real bug. When you only ever compare a file to itself, you never find anything.

Make the check able to fail

The other habit worth stealing: before believing a passing check, confirm it could have failed.

“Zero rows incorrectly routed” is not a pass when the destination table is empty. It is not exercised. I have been handed that reassurance three times, and the third time it was about to justify releasing sixteen thousand rows.

Proving a check can fail is usually cheap. Feed it a row you know is bad and confirm it complains. If it cannot fail, it is not a check, it is a sentence that sounds like one.

The mechanism underneath all of it

Every failure in this post has the same root, and it is not technical.

We measure to confirm what we already believe, instead of to falsify it.

“Is the job running?” sent everyone looking for evidence that it was. Rows had output, so yes, it must be early. Nobody asked what they would see if it were not running, which was a single empty field, visible in seconds. That question got asked 81 minutes in, by someone who came to it fresh.

So the instruction that actually changes behaviour is not “verify carefully”. It is narrower: before you measure, say what you would see if the thing were false, then go looking for that.

And a number that confirms the story already in play deserves more scrutiny than one that contradicts it, not less. That is backwards from how it feels, which is exactly why it has to be a rule rather than an instinct.

What to do instead of adding an agent

Add a method, not an opinion.

A second reader that starts from a different source than the first.
One check anchored upstream, outside the artifact.
A verifier told to refute rather than review, defaulting to “this is wrong” when it cannot tell.
Several verifiers with genuinely different lenses when a thing can fail in several ways, instead of the same review run three times.

Three agents running the same flawed check are not three votes. They are one flawed check, wearing more confidence.

--- title: Two agents agreeing is one observation date: 2026-08-04 tags: working-with-ai, ai-agents url: https://mihajlomaiga.com/blog/two-agents-agreeing-is-one-observation --- Chain a few agents together and the arithmetic gets unpleasant fast. Three steps at 70% each is 34% end to end. And 70% is generous: measured on SWE-bench Verified, [GPT-5 submitted a patch on 100% of runs and resolved 44%](https://arxiv.org/abs/2603.25764). Put three of those in a row and you are under 10%. The standard response is to add a checker. Have a second agent verify the first. It helps far less than it appears to, and the reason is worth understanding before you spend money on more agents. ## Agreement is not corroboration Two agents reporting the same number is not two observations. If they used the same method, it is **one observation reported twice**, and the second report adds no information while roughly doubling your confidence. That is the worst possible trade. I watched this happen. Two sessions independently reported the same progress figure for a large batch. Both were wrong, in the same direction, by the same mechanism: both had sampled the top of a table. Those record ids were creation-ordered. The top of the table was the **oldest** rows, enriched during earlier runs. Both agents were asking "did today's work succeed?" against a sample that contained none of today's rows. That is not bad luck, it is structural. Neither sample could have contained a counterexample. The check was incapable of returning bad news, and would have kept agreeing with itself indefinitely. ## Self-referential checks cannot see absence Here is the version of this that fooled me longest. I had a QA suite on a data batch. Row counts matched one to one. Zero duplicates. Zero overlap between segments. Valid JSON throughout. Every field inside its size limit. All green. The batch was missing about ten thousand rows. Every one of those checks takes the artifact's own rows as its universe. One to one compares my file to my file. Not a single check could observe a row that was not there. It did not feel like cutting corners, it felt like diligence, and it was diligence, pointed entirely inward. **Absence is invisible to any check that starts from what is present.** So the question a review has to answer is not "is everything here correct" but "what would show me something that should exist and does not". If no check answers that, you have no coverage for the largest failure class there is. The fix is one line of principle: **at least one check must be anchored to an independent upstream count.** Never the artifact against itself. A reader's output versus the raw source line count. A live record pull versus the exported file. Two numbers derived different ways that have to agree. When those disagree by exactly one row, you have found a real bug. When you only ever compare a file to itself, you never find anything. ## Make the check able to fail The other habit worth stealing: before believing a passing check, confirm it could have failed. "Zero rows incorrectly routed" is not a pass when the destination table is empty. It is not exercised. I have been handed that reassurance three times, and the third time it was about to justify releasing sixteen thousand rows. Proving a check can fail is usually cheap. Feed it a row you know is bad and confirm it complains. If it cannot fail, it is not a check, it is a sentence that sounds like one. ## The mechanism underneath all of it Every failure in this post has the same root, and it is not technical. **We measure to confirm what we already believe, instead of to falsify it.** "Is the job running?" sent everyone looking for evidence that it was. Rows had output, so yes, it must be early. Nobody asked what they would see if it were **not** running, which was a single empty field, visible in seconds. That question got asked 81 minutes in, by someone who came to it fresh. So the instruction that actually changes behaviour is not "verify carefully". It is narrower: **before you measure, say what you would see if the thing were false, then go looking for that.** And a number that confirms the story already in play deserves *more* scrutiny than one that contradicts it, not less. That is backwards from how it feels, which is exactly why it has to be a rule rather than an instinct. ## What to do instead of adding an agent Add a **method**, not an opinion. - A second reader that starts from a different source than the first. - One check anchored upstream, outside the artifact. - A verifier told to refute rather than review, defaulting to "this is wrong" when it cannot tell. - Several verifiers with genuinely different lenses when a thing can fail in several ways, instead of the same review run three times. Three agents running the same flawed check are not three votes. They are one flawed check, wearing more confidence.