# Experiment Receipt Discipline for Small-Model Autoresearch — Research Notebook

## Question (pre-registered, dated)

[2026-07-06] What does experiment receipt discipline add to small-model autoresearch when every governed experiment records hypothesis, baseline, noise floor, measured outcome, and KEEP/DISCARD decision — and negative results are preserved as first-class evidence?

## Method (pre-declared, before any data/derivation)

Case study plus methods analysis. Start from the public `autoresearch-pipeline` corpus, then extract the receipt schema, map it to failure modes it prevents, and quantify the corpus. Keep the scope bounded to the 26M / short-training regime unless a replication extends it.

## Predictions (timestamped BEFORE observations)

[2026-07-06] The strongest contribution will be the negative-result preservation claim: 32 of 34 experiments were DISCARD, and the governance value is in keeping them rather than silently dropping them.

[2026-07-06] The most likely reviewer objection is scale-dependence — that the 26M / short-training findings do not generalize. The first draft should not overclaim beyond the corpus's own regime.

[2026-07-06] The cleanest measurable outcome will be receipt completeness: what fraction of governed experiments have all five receipt fields populated, and what fraction of negative results are preserved in the corpus.

[2026-07-06] The Pack 2 nanochat frontier port (5 experiments, all DISCARD) will be the most instructive negative-result case: it demonstrates that frontier features do not transfer to 26M scale, a lesson that receipt discipline preserves rather than silently loses.

## Observations (append-only, dated)

[2026-07-06] Issue hummbl-io/hummbl-papers#14 asks for a candidate paper on experiment receipt discipline for small-model autoresearch. The coordination-bus seed (#13) has already landed, clearing the gate ("do not draft until the coordination-bus seed PR lands").

[2026-07-06] The `autoresearch-pipeline` repository has a public experiment receipt corpus suitable as the worked example: `program.md`, `results.tsv` (243 data rows), and `docs/experiment_receipts/*.md` (9 receipt docs through Pack 2).

[2026-07-06] Corpus quantification at main commit `f39a3ee`:
- 34 governed experiments (PRs #16, #18-#24)
- 2 kept patches: BATCH 2x (PR #20), BATCH 2x 10min (PR #22)
- 32 DISCARD negative results
- 2 crash results
- Noise floor measured twice: 0.180 (untuned config, PR #16) → 0.005 (tuned config, PR #18)
- Pack 2 nanochat frontier port: 5 experiments, all DISCARD (PR #24)

[2026-07-06] The candidate source packet (research-source-packets#9) describes the corpus as a first-party experiment receipt source. It is candidate-only and bounds claims to the 26M / short-training regime.

[2026-07-06] The ROADMAP now lists this paper as seeded artifact #2 (PROPOSED) and pre-registered backlog entry #21.

## Interpretation (separated from Observations)

[2026-07-06] This paper is a methods paper, not an architecture paper. The contribution is the receipt discipline, not the 2 kept patches. The 32 DISCARD results are the evidence that the discipline works — they were preserved rather than silently dropped.

[2026-07-06] The noise-floor measurements (PR #16, PR #18) are the strongest example of receipt discipline preventing a failure mode: without measuring the noise floor first, the QK-Norm experiment would have been authorized against an untuned config where the noise floor (0.180) was 36-90x larger than the expected improvement. The receipt discipline caught this before any architecture experiment ran.

[2026-07-06] The Pack 2 frontier port is the strongest scale-dependence case: 5 frontier features tested at 26M scale, all DISCARD. This is not a failure of the features; it is a success of the receipt discipline in preserving the negative result so future work does not re-test them at the same scale.

## Dead ends (failed attempts, also first-class)

- [2026-07-06] Considered drafting a full paper before the source packet landed. Rejected because the source packet (research-source-packets#8/#9) is the prerequisite — the paper needs a bounded source contract before it can cite the corpus.

- [2026-07-06] Considered claiming generality beyond the 26M / short-training regime. Rejected because the corpus has not been replicated at other scales. The paper must bound its claims to the regime it has evidence for.

## Open questions

- What is the smallest public replay fixture that demonstrates receipt-discipline value without exposing private runtime?
- Should the first version remain a technical report, or target a workshop paper format?
- What scale-dependence thresholds separate "frontier feature applies" from "frontier feature is cargo-culting"?
- Can the receipt schema be validated automatically, or does it require human review?
- Should the paper propose a formal receipt schema (JSON Schema or similar), or keep the five-field description informal?

## Cross-references

- Source packet: hummbl-io/research-source-packets#9 (candidate)
- Source corpus: hummbl-io/autoresearch-pipeline (main commit `f39a3ee`)
- Parent issue: hummbl-io/hummbl-papers#14
- Batch receipt: hummbl-io/hummbl-papers#15
- Sibling paper: hummbl-io/hummbl-papers#13 (coordination-bus seed, SCOPED)
- Evidence graph: hummbl-io/hummbl-io#104 (Cross-Repo Evidence Graph v0.1)

## Source audit — 2026-10-10 (append-only correction)

**Method/provenance:** Read source Git objects at `autoresearch-pipeline` commit `f39a3eec3a24a02c1e11d421d5752953e49d10db` in an author-accessible checkout. Used the new paper-local `audit_corpus.py` to count run rows and receipt files and recompute selected table statistics; the generated `source_audit.json` includes numerical transcriptions and SHA-256 source hashes. This was a retrospective documentary audit, not a training rerun, independent replication, or preregistered efficacy study. Earlier "Predictions" entries already contain outcomes and must not be used as proof of preregistration.

**Observed corrections to earlier entries and manuscript:**

- The results table has 243 data rows: 55 keep, 186 discard, and two crash. These are run-row labels, not 55 accepted patches or a denominator for all attempted experiments.
- The source's 34 scheduled candidate slots include two no-ops (QK-Norm and logit soft-capping were already in the baseline). Counting the remaining 32 executed settings as 32 DISCARD outcomes is incorrect: the wave summaries contain two KEEP decisions and other dispositions, including within-noise judgments. No complete row-to-setting audit has been claimed.
- Both initial control receipts contain **five** seeds (42, 137, 271, 500, 999), not three tuned-control seeds. The recomputed means/ranges are 0.9219638/0.180419 and 0.4803666/0.004988. Ten configuration values changed between the controls, so batch size alone is not a causal explanation.
- A later BATCH 2x control also contains five seeds. Its range is 0.000777 and twice its sample SD is about 0.000567. The source changes its "noise floor" estimator from range to twice SD; the old 8.8× comparison is not a ratio of like quantities.
- Pack 2 tests five features against BATCH 2x, with matching control values 0.475223, 0.475353, and 0.475251. GELU belongs to Pack 1. The earlier paper incorrectly used the earlier ~0.4804 baseline and included GELU as a sixth listed feature. Recomputed paired descriptive deltas are recorded in the audit JSON; source summary deltas instead match use of the five-seed control mean, to rounding.
- The original program records a denied/conditional authorization sequence for QK-Norm and later a no-op discovery, not a QK-Norm KEEP. The previous replication protocol's KEEP assertion, identical control configurations, sign convention, and runtime estimate were therefore corrected.

**Interpretation:** Preserved records enabled these corrections, which demonstrates inspectability of the available material. It does not establish that receipts caused better research decisions, that unrecorded runs never existed, that any particular feature has a model-size threshold, or that a range is a statistical significance threshold. Earlier claims that the discipline "works," prevents all failures, or preserves all negative experiments should be read with this correction.

**Availability and remaining work:** Selected numerical evidence and arithmetic are now bundled; full training sources, datasets, launch ledger, logs, and a publicly accessible exact replay package are not. The cited short training commits `d2ce712` and `a17a560` were not resolvable in the inspected local `autoresearch-win-rtx` checkout. No completeness coding matrix supports the earlier manuscript's 34/34 or 24/34 field scores, so they have been withdrawn pending a proper audit. The manuscript and protocol now distinguish arithmetic replay, source access, performance replication, and an independent receipt-efficacy evaluation.

**Bibliographic correction:** Verified the full Pineau et al. author list against JMLR and removed the unsupported standalone "Pitt & Wipf (2020)" bibliographic item from this paper. Observation/interpretation separation remains a useful notebook convention without treating that unverified publication as an external source. No stored PDF or deposit metadata was altered.