A compile is a fact. A claim is not.
Ask a model whether the code it just wrote is correct and it will tell you yes. That answer is unfalsifiable — there is nothing underneath it you can independently check. Hand the same code to a compiler and something different happens: you get a result anyone can reproduce, from the same source, on their own machine, without trusting whoever ran it first. That is the whole idea on this page.
Train against the checker, not the claim.
Rail compiles itself, so the compiler is small enough to read and reproducible enough to trust as a referee. We use it as the fitness function. Programs it accepts become training data. Programs it rejects are the gradient. Nothing in that path asks you to believe us — you can re-run the referee.
-- every arrow below is something a stranger can re-execute
seeded generator → corpus → rail_native verifies every program
→ hash-chained ledger
→ Ed25519 witness on a separate machine
→ anchored to a public entropy pulse
→ training → the same oracle scores the resultthe corpus is regenerable from its seed; the ledger is a chain; the witness key is not ours alone
A model trained this way, from scratch.
A 138M-parameter model, trained only on oracle-verified
comment → program pairs, writing Rail. Small
by design: the point was never to win a benchmark, it was to make every
input to the model something you could audit.
The corpus behind it — 52,243 pairs — is bit-reproducible: regenerate it from its seed and you get identical bytes, twice over. Its hash is bound into a ledger signed by a witness key held on a separate machine, anchored to entropy pulse 2641877. You do not have to take the corpus on faith either.
What that does not say
The same model generalizes 0/10 to function names it never saw. At this size it memorizes the shape of a header and retrieves a body; it does not compose one from the comment. The benchmark number is real. So is this. Publishing the first without the second would be marketing, and the gap needs scale or a differently-shaped corpus — not another round of tuning.
The month the method lied to us.
In July we ran a self-improvement loop for about a week. A local model proposed changes, a deterministic executor applied them, every generation was scored on a held-out reward and signed into a tamper-evident chain. It reported a clean climb across roughly eleven signed generations. Every step of it was reproducible. Every step of it was attested.
It was noise
Re-scoring the identical lineage on a held-out sample five times wider flattened the entire curve. The “improved” frontier scored exactly what the base it started from scored — 672/1920 against 672/1920.
The apparatus was sound and the conclusion was worthless. The held-out reward was itself a tiny sample, and a deterministic optimizer maximizing a small reward will fit that sample — reward hacking one layer beneath the defense we had built against reward hacking.
Attestation of a computation is not validation of its metric. A signed chain proves the numbers were produced the way we said they were. Whether the numbers mean anything is a different question, and the signature is silent on it. Every reported gain here is now re-measured on a disjoint, much wider sample before it is believed — especially the flattering ones.
this is on the website, and in the repo's README, for the same reason: a project that says “check me” has no business hiding the time it fooled itself
The referee is downloadable.
Everything above rests on one binary being reproducible. That claim is the easiest on this site to falsify: fetch the release, fetch its attestation, and check the signature yourself. If the bytes we publish ever stop matching the record a separate machine signed, this is where it shows.
curl -sf https://ledatic.org/attest/verify.sh -o /tmp/v.sh && chmod +x /tmp/v.sh
curl -sf https://ledatic.org/releases/v5.3.0/rail_native -o /tmp/rn
curl -sf https://ledatic.org/releases/v5.3.0/rail_native.attestation.json -o /tmp/rn.att.json
/tmp/v.sh /tmp/rn /tmp/rn.att.json
# → ok artifact=rail_native pulse_id=… pk_fp=cac5f21a70564aebThe build behind that binary is attested too — the test suite bracketed by two beacon pulses, and a self-host record showing the compiler rebuilt itself to a fixed point that matches the committed seed byte for byte.