# Replication and data scaling protocol

Frozen before phase-2 GPU work on 2026-09-19. The user's $60 cumulative ceiling includes the pilot. Keep the existing $50 conservative operating ceiling and $5 reserve. Every GPU stage goes through the existing cumulative budget launcher; failures retain their full reservation until termination is verified. No public uploads.

## Replication gate

Train seed 20260919 on the exact same 512 captured target conversations, initializing from the original draft. All pilot hyperparameters remain fixed: 100 optimizer updates, 8 examples per update, BF16, learning rate 1e-5 with the existing decay, validation selection every 25 updates. This isolates training randomness; it does not independently resample the training corpus or target responses.

Run two new H100 sessions, each evaluating all candidates sequentially. Orders: second seed, pilot Rust draft, original draft; then original draft, pilot Rust draft, second seed. Single request concurrency, identical pinned stack and decoding settings, one measured repeat per session after warmup. Record GPU UUID; a fresh allocation is not guaranteed a distinct physical GPU.

Frozen evaluation: the pilot's 32 Rust prompts, 32 previously unused Rust held-out prompts, the same 32 Python prompts, and 24 locally authored diagnostics (8 each prose, reasoning, JSON tool calls). The latter are small workload checks, not standardized capability benchmarks. All use max 1024 output tokens and natural EOS. Preserve and report capped responses and failures.

Primary endpoint: median per-prompt latency reduction on the 32 fresh Rust prompts, aggregating each prompt's latency across the two sessions. Bootstrap prompts 2,000 times for an exploratory 95% interval. Report both seed comparisons separately, both sessions separately, exact output equality, and the same endpoint on old Rust and each control domain. Repeats and seeds are not independent prompts.

Gate to scaling: both adapted seeds must have positive fresh-Rust median improvement in each session and a pooled prompt-bootstrap interval wholly above zero, without request failures or output differences from the original on fresh Rust. If this fails, stop scaling and investigate/report; do not relabel the old Rust cohort as primary. Diagnostic controls do not establish broad absence of regression.

## Conditional scaling

If the replication gate passes, extend to a nested 2,048 Rust prompts from the same pinned source, preserving the original 512 and all crate-disjoint splits. Reuse existing target responses/features for those 512. Generate and capture only the additional 1,536. Train two candidates from the original draft with seed 20260918: 100 updates (fixed-update comparison), and 400 updates (same expected number of passes as the 512/100 pilot). Count actual prompt/answer tokens; fixed updates do not imply identical FLOPs or unique-token exposure. Each run has its own learning-rate schedule with the same peak and normalized shape. Use validation-only checkpoint selection.

Evaluate against the original draft and pilot 512 draft with the same reversed-session design. The 100-update comparison isolates data-pool size at approximately fixed update budget; the 400-update comparison changes both data and compute. Two data sizes cannot establish a saturation curve.

If budget allows, add a 2,048-example mixed candidate with 1,536 Rust plus 512 distinct general instruction prompts, trained for 400 updates from the original draft. Pin, inspect, and document the source before generation. Keep evaluation prompts excluded. This comparison is exploratory: mixture/token lengths and compute differ. Report domain latency regressions explicitly; do not claim frozen-target capabilities improved. Mixed-data work is subordinate to completing and auditing the replication and Rust-scaling comparisons inside budget.

Do not use phase-2 test results to tune hyperparameters or select the checkpoint. Keep pilot artifacts immutable. Batch-concurrency numerical differences from the pilot remain unresolved; this phase's primary endpoint is single-request serving.

Operational addendum, before scaling jobs: 400-update runs use two 200-update cloud stages to stay inside each stage's time bound. The schedule horizon remains 400 throughout; the second stage restores the complete step-200 checkpoint, including optimizer, scheduler, RNG and data position through SpecForge's native resume. Validation selection retains the existing best-checkpoint metadata across stages. This is not two independently restarted training runs.

Conditional general-data source: Databricks Dolly 15k, revision `bdd27f4d94b9c1f951818a7da7fd7aeea5dbff1a`, CC-BY-SA-3.0. Select 64 prompts from each of its eight categories (512 total), with fixed random seed, exact normalized evaluation dedup, and prompt length <=1,024 target tokens. Use only instructions/context; regenerate responses with frozen Qwen. Source README and selection manifest are retained in `data/general/`. No claim of semantic decontamination or comprehensive tool-call coverage.

Token accounting detail: reconstruct example order using the pinned SpecForge offline sampler and recorded seed. Report unique examples and teacher-reported prompt/completion token exposures both at the validation-selected checkpoint and at the completed training budget. Teacher usage counts are not exact FLOPs or multi-step loss-mask token counts. In particular, the 2,048-prompt/100-update candidate sees only 800 examples, while the 400-update candidate can cover the complete pool.

Mechanism diagnostic addendum, before scaling/mixed training outcomes: after each timed scaling/mixed model pass, run 24 separate native-endpoint requests (16 of the frozen fresh-Rust prompts and all 8 JSON-call prompts). Save accepted/proposed draft-token and verification counts, excluding bonus tokens from acceptance. These requests do not enter latency estimates. The selected prompts and hash are in `data/acceptance-manifest.json`. This may help explain speed changes; it does not replace the timed endpoint or expand the primary test set.

Compute-control addendum, before inspecting any larger-pool serving results: add `small400`, a 512-example/400-update run from the original draft with seed 20260918 and the same 400-update learning-rate schedule. Set its epoch bound to 7 so 400 updates (3,200 example exposures; 6.25 passes) are permitted; all other hyperparameters stay fixed. Train it in two fully resumed 200-update segments. Compare it against `scale400` and the original draft in a separate counterbalanced pair of sessions on the same frozen evaluation prompts. This tests dataset size at the larger update budget and avoids attributing extra-training gains solely to more data. Prioritize this control before the optional mixed-data study if budget becomes tight. No new data generation is needed. The cumulative $60 user ceiling and stricter operating guard remain unchanged.

Packaging recovery, before mixed-data generation: the first mixed launch failed locally before the GPU function began because `comparisons.csv` changed while the broad project directory was being uploaded. The failed app was confirmed stopped with zero tasks; its original $5 reservation and termination evidence remain in the ledger alongside the reconciled elapsed-time estimate. Restrict cloud packaging to code, configuration, patches, and frozen data directories, excluding generated reports/results. Preserve the failed logs and progress file under `mixed-attempt1-*` names, then retry the same unstarted generation stage. This changes packaging only, not data selection, training, or evaluation settings.

Budget operating update, announced during the second half of mixed training: retain the user's $60 cumulative absolute ceiling, but raise the internal operating threshold from $50 to $55 including the existing $5 reserve. This permits the planned final benchmarks' conservative $5 per-stage reservations; projected completion remains around $43. No additional candidates or data sweeps are authorized by this change. Preserve all previous charges and the cumulative ledger. The launcher now explicitly takes the lower of the operating threshold and the recorded user authorization, including lower historical authorizations, and logs the threshold change under its existing ledger lock at the next stage launch. Budget-blocking tests cover $60, $50, and $40 recorded authorizations. This supersedes the earlier internal $50 operating rule, not the user's $60 cap.