# Evaluating the Proposed Base120 Axes

**Companion:** [Governance-as-Code](README.md), §§2 and 6

**Status:** Revised protocol, 2026-10-10; a bounded definition-only diagnostic is complete, while semantic and task studies remain prospective

**Earlier protocol:** 2026-08-18, retained in Git history

**Registration:** No external preregistration is claimed. The [completed diagnostic](analysis/README.md) has its own locally frozen exploratory plan, exact source hash, code, and results. It executes a bounded lexical/geometric comparison, not every proposed analysis below.

## 1. Separate the claims

The proposal pairs Perspective/Inversion, Composition/Decomposition, and Recursion/Systems. These names do not define a vector space. Four claims need different evidence:

1. **Lexical organization:** descriptions differ systematically by family.
2. **Semantic opposition:** a proposed pair represents opposing transformations on a shared, defined dimension.
3. **Geometric separation:** prespecified representations yield distinct, approximately orthogonal contrast vectors.
4. **Practical utility:** the grouping improves decisions on held-out tasks relative to alternatives.

Evidence for one claim is not evidence for all four. Cosine distance measures geometry in a chosen representation, not antonymy. Opposites can share vocabulary and embedding neighborhoods. High centroid rank does not prove three bipolar dimensions. Failure to reject a uniform temporal-term distribution does not demonstrate a new independent axis.

The earlier protocol used these measures as decisive tests of the combined claim. That interpretation is withdrawn. Its proposed Mann–Whitney comparison also treated family pairs as independent even though each family appears in several pairs. No results from that protocol are adopted here.

## 2. Freeze inputs before analysis

Record the registry repository, exact commit, path, content hash, license, and extraction mapping. Begin discovery at [hummbl-io/base120](https://github.com/hummbl-io/base120); do not assume the historical filename or a default-branch URL pins the dataset. Record every included identifier, family label, name, and description. Report actual counts by family rather than forcing a 20-per-family assumption.

Document missing descriptions, duplicates, aliases, and exclusion rules before reading diagnostic outputs. Preserve the source snapshot or provide a retrievable immutable reference. Stop a proposed confirmatory analysis if declared inputs are incomplete; exploratory repairs get a separate revision and rationale.

For each representation, freeze tokenization, normalization, stopwords, vector weighting, and any model checkpoint and software versions. TF–IDF and learned embeddings are separate analyses, not interchangeable steps. A remote model without a fixed revision is an explicitly unstable instrument. Store vectors or hashes where licensing allows; record randomness and environment information.

## 3. Lexical diagnostic

Compute per-family token frequencies, description lengths, and the 15 pairwise vocabulary-overlap coefficients. If either vocabulary is empty, return “undefined” rather than dividing by zero. Report label words within descriptions; editorial conventions can make family separation trivial.

An optional label-permutation diagnostic preserves observed family sizes and recomputes the chosen statistic. Freeze the statistic, random generator, seed, and permutation count beforehand. With `B` random permutations use the declared one-sided tail count `(1 + b) / (1 + B)`; report ties. This describes association between text and assigned labels under that permutation null. It does not test semantic independence of mental operations. Registry entries are curated, not a random sample of cognition.

## 4. Geometric diagnostic

For a frozen representation let `c_f` be the family mean vector. Define:

```text
d1 = c_P - c_IN
d2 = c_CO - c_DE
d3 = c_RE - c_SY
```

Report each contrast norm and, when nonzero, the three normalized dot products `d_i · d_j / (||d_i|| ||d_j||)`. Zero contrasts are undefined for this normalization and indicate no centroid separation in that representation. Predeclare numerical tolerances and any equivalence margin; do not choose them after seeing results.

Even near-zero cross-products establish only representation-specific geometry. PCA components are orthogonal by construction, so observing orthogonal PCs does not show that the proposed semantic axes are orthogonal. Uncentered centroid rank and centered contrast rank answer different questions. Report centering explicitly; six centered centroid rows have rank at most five.

For an exploratory pairing comparison, enumerate all `5 × 3 × 1 = 15` perfect matchings of the six families. Compare the proposal against those alternatives using a declared score, such as mean paired-centroid distance. If interpreting the uniform matching distribution as an exact randomization reference, include the observed matching in the tail count. Its smallest possible one-sided tail probability is `1/15 ≈ 0.0667`; it cannot meet a 0.05 threshold. More bootstrap iterations do not create additional independent families or remove this resolution limit. Report ranks and ties as diagnostics, not a discovery of semantic opposites.

## 5. Semantic judgment study

Before recruiting raters, define opposition for each proposed dimension: a common object, the transformation applied to it, and what makes two transformations opposing rather than merely different or sequentially useful. Include examples and counterexamples. Combining and separating components may form a contrast without being inverses on arbitrary inputs.

Use a development set to clarify the rubric, then freeze it. Present separate held-out task and operator descriptions to raters without the proposed pairing labels. Ask separately about shared dimension, opposing direction, co-applicability, and ambiguity. Record instructions, recruitment, expertise, compensation if any, missing judgments, and adjudication rules. Choose sample size and agreement statistics before outcomes are available. A single model's labels are not independent human validation.

Report disagreement and alternative pairings. Co-applicability may refute a mutually exclusive reading while leaving a weaker descriptive contrast useful. No raters have been recruited and no such data are supplied with this revision.

## 6. Temporal candidate and practical utility

A temporal keyword list can locate exploratory examples. Normalize for description length and report zero counts. Distribution alone cannot distinguish a new axis from an attribute shared by many operations. A nonsignificant test is not proof of uniformity, and uniformity is not proof of independence.

To evaluate a temporal extension, specify the prediction it improves, such as selecting a reasoning procedure for a held-out task. Compare the six-family scheme with the same scheme plus temporal labels. Include a simpler alternative: adding a time-orientation attribute without expanding the taxonomy. Equalize task information, training exposure, and evaluation budget. Freeze the rubric, primary outcome, minimum worthwhile effect, sample-size plan, and multiple-comparison handling before evaluation. Report task quality alongside time and disagreement.

The same design can evaluate the three pairings against an unpaired six-family taxonomy. A better-performing alternative or failure to achieve a prespecified worthwhile effect weakens practical claims. An underpowered null result remains inconclusive.

## 7. Required study package

Before calling the study reproducible, provide:

- An immutable source manifest, extraction code, environment specification, and schema checks.
- An analysis plan dated before confirmatory outcomes, distinguishing pilots and exploratory changes.
- Lexical and geometric outputs, including undefined values and every alternative matching.
- Rater/task materials and permissible data, with exclusions, disagreements, and privacy constraints.
- Commands regenerating tables from deposited data; no private ledger dependency for a reported result.

The [definition-only package](analysis/README.md) supplies a reproducible exploratory comparison of all 15 matchings, with its own frozen instrument and limitations. It does not supply human ratings, task outcomes, or a confirmatory study of cognitive axes. Those proposed studies still require executable packages and review of what each measurement supports. The original `clp-67429a7ba0a7` reference remains a historical provenance pointer only.