Math becomes a test harness; the harness becomes evidence. This is Stage Zero: code, numerical experiments, and checks that make claims falsifiable.
Everything up to here has been an argument about a game. This page answers the reasonable next question — is any of it real? Three public repositories, MIT or CC0, roughly eleven thousand lines and sixty-five tests, written by one person on compute paid for out of pocket. They have produced results, two of which are inconvenient for the person who produced them.
Three, deliberately separated: the testbed does not train, the trainer does not score, and the site labels every claim as measured, resampled, arithmetic, or planned.
| repository | holds | does not hold | size |
|---|---|---|---|
| flockbench MIT · data CC0 |
The testbed. Four environments — Shared Resource, Commons Harvest, public goods with punishment, boardwalk — plus the campaign planner and runner, the frozen trace schema, and Firebreak, the judge-free promotion gate, which is a module here rather than a separate project. | No training code, no model weights, no judge anywhere in the scoring loop. | 3.7k lines 49 tests 29 commits |
| continuous_judge MIT |
The training side. Span-restricted preference distillation, the legal-action masking that
keeps the action schema intact, the cached teacher pass, and the style controls —
snake_case as a positive control and uNsTaBlE_-0-_CaSe as the
known-bad candidate. Runbooks for each experiment. |
No environments and no scoring. It produces adapters; what they are worth is decided elsewhere. | 6.0k lines 16 tests 29 commits |
| multiagent MIT · figures CC0 |
This site. Every interactive re-implements the testbed's arithmetic in TypeScript, and the test suite pins the two implementations to each other — so a figure here cannot drift from the thing it claims to depict. | No results of its own. If a page disagrees with a receipt, the page is wrong. | 28 modules 176 tests 43 commits |
Progressive Denoising Distillation (PDD) inpaints selected reasoning spans. Only replacement tokens carry gradient; structured action fields stay locked.
Built, not yet proven better: mask selection, cached teacher edits, and
sparse loss live in continuous_judge. flockbench tests population effects.
Method and caveats.
Also public and adjacent, but not part of this proposal: freetimebench, an earlier judge-based harness. It is the thing whose limitations led to the judge being removed from the loop entirely.
What "public" is doing here. Not licensing hygiene: every number should be recoverable
by a stranger. tools/replay_receipt.py re-derives a published figure from a committed
receipt, without the run directories. An audit applied that standard to this site in August 2026
and withdrew a result that could not meet it.
The two most useful are embarrassing.
Why the awkward ones matter most. The gate is the tool this proposal is built around, failing a test nobody had thought to run; the dead commons config is the same story smaller. That is the argument for the whole approach in one line — in an environment with an answer key, an instrument can be caught being wrong, and in one without, it cannot.
| Stage 0 · now math → harness |
The game gives a closed-form target; the public harness implements numerical experiments; unit and integration tests prevent the simulation, the figures, and the claim ledger from drifting apart. This stage is built, including results that exposed flaws in the instrument. |
| Stage 1 · funded research harness → evidence |
A small team runs the calibrated, larger-scale training campaign: matched LoRA arms, multiple model families and population sizes, and a public dose-response rather than one promising trace. The campaign design and evidence ledger define what counts. |
| Stage 2 · only if Stage 1 earns it evidence → user control |
Build resources for people to customize, share, and remix evaluated agent personas without being locked to one inference substrate. The portable object is not a universal LoRA: it is a persona specification, provenance, evaluation receipt, and model-specific realization. Stage 1 determines which behavioral ingredients are worth packaging. |
Two things situate it beyond this site. The call's own starting points are answered one at a time on where this sits, in their vocabulary rather than ours. And the direction is a ladder of long-rollout text games — solvable games now, the same games with a channel next, then Diplomacy, then populations at MMO scale — where each rung trades away the answer key, which is the reason to be strict about the bottom one.
Repository figures are counted at flockbench 1b08ef0 and
continuous_judge 6915009, August 2026. Every result above is expanded, with its
evidence class, on the status page.