One base, many strategies
Create cheap adapters whose behavior differs because of learned updates, not merely a persona prompt.
One pilot · one population method · one open scaling question
What training changed—and what it did not solve.
Experiment 1 shows one Common Harvest trace lasting longer as more post-trained agents enter the population, without finding a sustainable strategy. This page replays that result, explains the PDD method used to create the adapters, and makes the remaining scaling uncertainty explicit.
33rounds before resource collapse · base population
90rounds · four post-trained seats
170rounds · eight post-trained seats
1 seedper population · diagnostic, not an estimate
Concrete pilot · why replication matters
Three trajectories. Resource collapse in every arm. No effect estimate.
Replay the three Commons Game runs on one time axis. Seeding changed the path to collapse, but one seed per arm cannot tell us whether the ordering is real, reproducible, or a consequence of that particular rollout.
Replay a concrete Commons pilot trace
Commons stock
Pilot · one seed per arm No interval and no effect estimate. The replay reads the checked-in trace directly. Receipt not yet committed.
Progressive Denoising Distillation (PDD)
A low-cost way to create strategy pools, not one universal agent.
We wanted to train many distinct commons-game personas without repeating a full-model fine-tune. The prototype uses a diffusion language model as an LLM-as-a-Judge: it inpaints rationale around a fixed target action, filters the proposed words, and distills accepted replacements into small LoRA adapters.
Longevity improved once
In the Commons Game—logistic stock regrowth and a harvest bid from every agent each turn—the trained diagnostic trace reached round 170 instead of round 33 for the baseline.
Training found the wrong extreme
In Shared Resource, the sustainable population alternates give and take. Both trained flocks instead gave every turn and all agents died by round 6.
Step through span-restricted PDD
original modelQwen● ● ●
pre-training rationale
“The pool is low; take now to protect my balance.”
observed action · schema intact
takeNext, condition a separate training target on the desired action: restore.
Baseline. This is a diagnostic rollout before training—not the record being edited. It gives the later evaluation a concrete behavior to beat.
desired actionrestore is fixed↘ ↓ ↙
target rationale · Mad Libs mask
“The pool is ░░░; ░░░░░░░ to preserve ░░░░░░░░░░.”
target action · locked
restoreNo training yet. First construct the target.
Mask. The desired action comes from the task specification or a selected trajectory. PDD does not edit it. It uses that fixed action and surviving context to condition the missing rationale.
conditioned on context + actiondiffusion
judge· ⁘ · ⁙ ·
judge· ⁘ · ⁙ ·
noisy
The pool is l••; r•••••• to preserve p••• ••••••••.
clearer
The pool is low; restore to preserve shared •••••••.
accepted
The pool is low; restore to preserve shared future turns.
action · locked throughout
restoreThe accepted completion is now a training target.
Inpaint. The fuzzy diffusion judge repeatedly refines only the blanks, like constrained Mad Libs. The fixed restore action anchors what the completed rationale is supposed to explain.
accepted rationale
“The pool is low; restore to preserve shared future turns.”
action · locked
restore
cross-entropy target
zero low; restore zero shared future turns.
loss on replacements only ↑same modelQwen● ● ●
fresh rollout · hypothesis
low pool → shared future
restore?Distill, then test. Cross-entropy loss is applied to accepted replacement spans only; the other tokens and the entire action field contribute zero gradient. There is no scalar reward or policy-gradient update. A fresh rollout must show whether take actually became restore.
masked / inpainted spanaccepted replacementtarget action excluded from gradient
Method under test These are one-seed diagnostic traces, not an effect estimate. The source-game improvement shows the adapter can matter; the synchronized self-sacrifice shows that more “cooperative” language is not the same as a viable collective strategy.
Open scaling questionHow cheaply can one base model become a heterogeneous flock?
The measured anchor is one Stage 0 timing: the longest Qwen2.5–7B run tested—2,000 steps—took about 3–4 hours. At the budget's planning rate of $2 per 80 GB GPU-hour, that is roughly $6–$8 for one attempt. Several attempts were needed to reach convergence, so the calculator keeps training time separate from the unknown number of attempts needed for each usable persona. It does not infer runtime from parameter counts or nominal token throughput.
One observed Stage 0 timing≈ 3–4 hrone 2,000-step 7B attempt · ≈$6–$8 at $2 per 80 GB GPU-hour
30personas×
4attempts each×
3.5GPU-hours / attempt×
$2/ GPU-hour=
$840adapter-training compute
training attempts120
cost / attempt$7
80 GB GPU-hours420
planned compute share2.1% of the planned compute line
Planned compute$840 of $40,000
5,714 training attempts is the theoretical maximum if the entire $40,000 compute line were spent at this rate—equivalent to 1,428 personas at four attempts each. $39,160 remains for evaluation rollouts and other compute.
What this includes: adapter-training compute only. The 3–4-hour timing is observed; four attempts per persona is a visible planning assumption, not a measured convergence rate. The cost of a usable adapter is therefore attempts-to-convergence multiplied by cost per attempt. Multi-agent evaluation rollouts, commercial APIs, storage, and labor are separate—and long population rollouts are expected to dominate the compute campaign. Below an unknown training floor, the “personas” may also be indistinguishable copies with different accents. Measuring both frontiers is part of the research.
The calculator turns an observed runtime into a planning range; it does not claim a cost per converged adapter. The proposed work measures which quantities—PDD steps, sequence construction, model size, LoRA configuration, and failed attempts—actually move that cost, then tests whether cheaper adapters remain behaviorally distinct at population scale.