First Sample Kit

First Sample Kit The minimum we send a frontier lab to earn a scoped deal · v2 · 2 Nov 2026

Internal

Build one data sample and one environment sample to this standard and nothing more. Everything else waits until a buyer scopes a paid delivery and we agree acceptance criteria with them.

Stage 0 · first callMenu + one filled card

08 menu, one Environment Card or Data Sheet with real numbers, 2–3 baseline figures.

Stage 1 · first sample · this pageOne family, one tier

Echo-S1 colour-mixing environment (20–50 target RGBs) and the PP-01 interpretation set (~100 items).

Stage 2 · paid deliveryFull pack

Everything in the grey "later" rows, against acceptance criteria agreed in writing.

Four things that end the deal if missing

  1. It runs on their cluster with one command. A broken sample ends the conversation.
  2. Baselines show headroom. Frontier models at 0% or 95% means nothing to learn.
  3. The reward cannot be trivially gamed. Their RL team tries this first.
  4. Rights are clean. Legal stops the deal even if the tech passes.

1. Data sample

Target: PP-01 interpretation set, about 100 double-reviewed items across colour mixing, titration and absorbance.

DATA SAMPLEStage 1 minimum · one task family CORE Items 50–150 items, one per JSONL line Raw evidence spectra, images, instrument files Answer / rubric reference answer or weighted criteria Schema typed fields, units, all fields valid PROOF Double review 2 reviewers + adjudication; report the agreement Frontier baselines 3 models, pass rate; shows headroom PAPERWORK Short Data Sheet what, how made, by whom, known issues: one page Rights statement own production; experts consented to AI use STAGE 1 STOPS HERE · build the row below only after a buyer scopes a deal LATER · Stage 2, paid delivery Training uplift before / after run Full leakage report search log Holdout spec size, kept by GD Licence + canary formal terms Public HF slice gated, eval only INPUTS Expert network writes items, rubrics; reviews in pairs Lab records raw evidence from S1, S2, S4 runs SENT TO BUYER Sample folder one task family, e.g. PP-01 interpretation set (~100 items) Then: scope call agree acceptance criteria before Stage 2
Eight items to build. The grey row is Stage 2 and should not hold up the sample.
ItemDone whenOwner
Items50–150 records in one JSONL file; each has id, prompt, input file references and a difficulty tag—
Raw evidenceevery item links to its spectra, images or instrument files, with units—
Answer / rubricevery item has a reference answer or weighted binary criteria—
Schemaone schema file; a script validates every record without errors—
Double reviewtwo reviewers per item, disagreements adjudicated, agreement figure reported—
Frontier baselines3 frontier models scored on the full set; no model above roughly 60%—
Short Data Sheetone page: contents, counts, how made, who made it, known issues—
Rights statementconfirms own production and signed contributor consent for AI-training use—

Folder we send

pp01-interpretation-sample/
├── items.jsonl
├── raw/                # evidence files
├── schema.json
├── baselines.json      # 3 models
├── review_agreement.md
├── DATASHEET.md        # one page
└── RIGHTS.md           # one paragraph

2. Environment sample

Target: Echo-S1 colour mixing. Each task gives a target RGB; the agent proposes volumes of dyed stock solutions and gets back the RGB predicted by a model fitted to our real camera and spectrometer runs. 20–50 target colours. Every competent agent can reach the colour, so the score comes from the path: the final ΔE is a gate, then safety and efficiency decide the score. Many different paths score full marks.

ENVIRONMENT SAMPLEStage 1 minimum · one family, one tier CORE · 20–50 task folders, one per target RGB instruction.md what the agent reads: target RGB, budget, rules task.json config for the runner: timeouts, resources, network off environment/ FROM shared base: colour model + tools verifier/ rubric + scorer; scores the path: ΔE, safety, steps witnesses/ paths proven to score full; not the answer PROOF Baselines + headroom 3 models, pass@k, reward distribution; best in 10–60% Anti-hacking note probing, interpolation, blind guessing, doing nothing: all score low OPS Runs in one command no outbound network; run.sh also exports Harbor layout PAPERWORK Environment Card one page: goal, interface, scoring, valid range Rights statement data behind the twin is ours to license STAGE 1 STOPS HERE · build the row below only after a buyer scopes a deal LATER · Stage 2, paid delivery OpenEnv adapter + harness map Noise + coverage full model Ops figures latency, concurrency Hosted endpoint Live tier Licence, SLA safety spec BUYER'S SIDE Agent their model, run by their RL harness action obs, reward RL / infra team runs it, tries to break the reward SENT TO BUYER Sample pack one family, one tier: Echo-S1 colour mixing to a target RGB Then: scope call agree acceptance criteria before Stage 2
Five parts per task, plus five pack-level items. The rubric in verifier/ is the only place the full-score conditions are written; witnesses/ only proves they can be met. The anti-hacking note matters most: it is what their RL team checks on the first run.
ItemDone whenOwner
Task folders20–50 folders, each with instruction.md, task.json, environment/, verifier/, witnesses/—
task.jsonone template shared by all tasks: id, family, tier, version, difficulty, agent and verifier timeouts, CPU and memory, allow_internet false—
Shared base imageone image holds the colour model, propose_recipe and query_history; each task Dockerfile is a one-line FROM—
Rubricone rubric.json: ΔE ≤ 3 as the gate, then weighted safety, attempts and material use; scorer reads every step, never compares recipes to a witness—
Witness check2–3 witnesses per task taking clearly different paths all score full under the current rubric; check.json regenerates on every rubric change—
Baselines + headroom3 models, several runs each; pass@k within the attempt budget and ΔE distribution; share of runs reaching full score roughly 10–60% (tighten the rubric if higher), and compared with random search and Bayesian optimisation at the same budget—
Anti-hacking notetried: probing the twin, interpolating from query history, out-of-range or negative volumes, blind guessing on the first attempt, doing nothing to avoid violations, format tricks; each scored low—
One-command runa colleague outside the team runs it on a clean machine with network off; run.sh also exports the Harbor layout; rough GPU-hours recorded—
Environment Cardone page: goal, tools (propose_recipe, query_history), ΔE scoring, colour range the twin covers, held-out error against real runs—
Rights statementconfirms the S1 runs behind the twin are ours to license—

Folder we send

echo-s1-sample/
├── tasks/
│   └── task-0007/
│       ├── instruction.md     # target RGB, budget, rules
│       ├── task.json          # timeouts, resources, network off
│       ├── environment/
│       │   └── Dockerfile     # FROM echo-s1-base
│       ├── verifier/
│       │   ├── rubric.json    # the only scoring standard
│       │   ├── score.py       # reads every step
│       │   └── test.sh        # writes the reward
│       └── witnesses/
│           ├── expert_01.jsonl
│           ├── expert_02.jsonl
│           ├── bo_01.jsonl
│           └── check.json     # auto: scores under rubric
├── run.sh                     # one command; Harbor export
├── baselines.json
├── ENVIRONMENT_CARD.md
├── ANTI_HACKING.md            # NDA copy
└── RIGHTS.md

Our names and the Harbor names

OursHarborWhy ours differs
instruction.mdinstruction.mdsame
task.jsontask.tomlJSON instead of TOML, same fields
environment/environment/same
verifier/tests/says what it does
witnesses/solution/full score is a space, not one answer; these prove it is reachable

run.sh converts to the Harbor layout for buyers who run Harbor directly, including task.json to task.toml. Live tier later reuses instruction.md, task.json and verifier/ unchanged; only the world moves from the colour model to real instruments.