The honesty gate · published 18 Aug 2026

Our own fine-tunes are losing our own arena.

Honest measurement, embarrassing to us, verifiable by anyone. A measurer publishing a result that embarrasses it is what pulls outsiders in.

The verdict, plainly

Our two council fine-tunes — council-inhouse-ft and council-safe — are losing to base models in our own arena, on our own GPU, with our own Elo ladder. We trained them. We measure them. They lose. We publish it.

The numbers (live from the arena league, 2026-08-18)

ModelEloGamesNote
qwen3:4bbase model1326.7672no council adapter
qwen2.5:1.5bbase model1311.5711smallest base, still ahead
mistral:7bbase model1252.4639base
council-safeour fine-tune1124.6533−202 vs leader
qwen2.5:0.5bbase model1113.0730tiny base
council-inhouse-ftour fine-tune1015.8496dead last, −311 vs leader

Headline: our two council fine-tunes occupy the bottom half. One is last. The base models we started from beat the adapters we built on them.

Why this is the most credible thing we can publish

  1. It contradicts our own product narrative. No one buys measurement from a body that hides its own losing results.
  2. It is fully reproducible. 3,700+ signed arena rounds on the pod (reborn_league.json + reborn_rounds.jsonl, Ed25519-signable). Any stranger can rerun.
  3. It matches the known literature pattern. Small-base fine-tunes on narrow governance batteries typically do not beat their base — our own earlier finding: a base Qwen2.5-0.5B beats every council fine-tune on 8 of 9 measured governance axes.
  4. It is the honest ceiling, stated before anyone else does: this instrument governs provenance, not correctness. An attested answer is attested, never verified. Our fine-tunes prove the point — they are signed, and they still lose.

What it means (and doesn't)

  • Does mean: adapter-souping weak bases does not beat the base. The measurement rail works — it caught us.
  • Does NOT mean: the instruments are broken. The instrument that shows us losing is the same one we sell. That is the point.
  • Next honest step: base model + statute retrieval — the only path that beat the fine-tunes in our own evals — not weight-merging weak specialists.

The Elo carve-out, disclosed

Our doctrine keeps arena Elo internal: we never publish Elo as a verdict on anyone's model, and the public board reports only deterministic per-axis measurements with n and intervals. This page is the one exception, and it exists to publish evidence against ourselves. The base-model rows appear because without them our loss would not be checkable — they are context for our failure, not a ranking we endorse. The board at /gspc-arena remains the only measurement surface we stand behind.

The record

Every number above is from the live pod state at 2026-08-18, recorded in the arena league (its internal battery convention, not the public board). Recompute path: reborn_league.json, ~3,700 rounds, Elo K=32. See also the board's own catches: jail (council-inhouse-ft detected zero escapes) and human-vs-ai (council-safe aligned 0.25) — published on the live board, not hidden.

The instrument measures everyone, including the person selling it. Verify the board at GET councilof.ai/api/gspc — no account, no key.

REPORTED — figures by others, cited, never mixed with ours

Three data states run this estate: MEASURED (signed runs on our frozen instruments), GATED/UNMEASURED (honestly withheld), and REPORTED — figures published by others, cited and timestamped for context. Reported by the source, not measured here; unsigned; never enters the board; implies no endorsement of the source's method. The machine-readable set — each entry with its source URL, capture date, and attribution basis — lives at GET councilof.ai/api/reported. Scores move: treat every figure as "as of its capture date" and follow the source for the live number.