# Issue 106: complete MTP final-primary state Qwen's round-head stop/token-limit exit now re-arms its emitted primary for the existing final commit, exactly as pinned MTPLX 2.12.2 `9882703f3105363ddc37eca9f97aa09a1d387112`, `generation.py:13001-13019` does. That commit updates committed MTP history and forwards the primary once. Sampling, acceptance, RNG, snapshot ownership and other exit branches are unchanged. Qwen checkpoint magic is now `QWMTKV02`. Old `QWMTKV01` files are rejected before tensor allocation; Generator's existing incompatible-cache path removes the file, resets the owner and rebuilds cold. This deliberately invalidates all old Qwen checkpoints, including AR files, because V1 does not certify complete MTP state. It does not alter the DS4/GLM format or their owners. This is a DS4Server persistence version change around MTPLX's existing inference behavior. There is no in-process executable hot-upgrade mechanism: resident sessions belong to the process-local Generator. Installing a new executable takes effect on restart, which drops all resident state. Runtime engine changes also drop the old Generator (`runtime.rs`, `run_command`). New processes cannot import old resident objects, and old processes' subsequently written V1 files are still rejected. ## Acceptance review | Requirement | Evidence | | --- | --- | | Real pre-fix reproduction | Parent `5ecf8ab`, unchanged model, 12 first-primary cases: key 4, all 12 trunk offsets 3, MTP offset 2, pending absent and no error. `before.jsonl`. | | Actually persisted bad state | A real `QWMTKV01` file was saved from the sampled depth-1 reproduction, with IDs `[9707,11,1075,314]`. File identity and hash in `checks.json`; the 116 MB cache is not checked into Git. | | Full state, not offsets alone | All 12 cases now have trunk 4 and history 3. Every encoded tensor's raw bytes, dtype, shape and metadata, including GDN/conv, PLE, QSA/KV, MTP history, logits and hidden state, match independent teacher forcing. `native-state.jsonl`. | | Exactly once and next turn | Teacher forcing restores the prompt, explicitly appends one MTP history row and forwards one primary. All corrected snapshots match it, persist/reopen exactly, restore four tokens, then match the full state after suffix `[1075]`. | | All upstream exits | Eight cases compare token sequences, cache lengths and stop origin with the pinned upstream scripted-model test: bonus, deferred correction, accepted draft, sampled correction, first/fresh primary, and both max-token depths. Native tests script only logits; real model computation and caches still execute. Context copy is disabled in both oracle and native matrix so the intended draft/verify routes are reached. `oracle.jsonl`, `native-matrix.jsonl`. | | No false final capture on failure | Cancellation at finalization returns an error; failed final forward leaves pending/error and no safe final arrays; disabling final capture leaves pending. `native-state.jsonl`. | | Old cache recovery in product | The real old file is rejected and removed by Generator, and recovery reports zero reused tokens. Its entire newly persisted state and response equal an independent cold request. `native-product.jsonl`. | | Existing persistence contracts | Model-free checkpoint roundtrip, corruption, failed restore, cancellation and atomic-write regression run under the Metal supervisor. | | Other families | Production diff is confined to Qwen's MTP exit and checkpoint signature. DS4/GLM retain `DS4RKV01`/version 2 and their existing pending/state behavior; shared RNG and inference helpers are unchanged. Ordinary cargo tests retain DS4 token/RNG controls. | The former bug-asserting `issue103_mtp_final_receipts` diagnostic is replaced by these positive-contract regressions. The only new Execution hooks are `cfg(test)` scripted logits/failure injection, absent from release builds. No Python is called by the product. These receipts establish correctness, not speed. ## Reproduction Use the unchanged Bare-Speed revision `74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`, M5 Max 128 GiB, SSD PLE, MLX 0.32.2 for the isolated oracle. Artifact/environment hashes were refreshed in `../issue104-ar/environment.json`. Native raw prompt is `[9707,11,1075]`, chunk 2048, seed 0, depths 1/2/3, temperatures 0/.6. To regenerate an actual old file, use an isolated checkout of `5ecf8ab`, apply `baseline-save.patch`, build its tests and run `issue103_mtp_final_receipts` with `DS4_ISSUE106_BAD_CHECKPOINT` pointing to the output file. The patch only adds persistence to the old diagnostic; it does not manufacture a header or cache. Build current tests with `cargo test --all-features --bin ds4-server --no-run`. Set `DS4_QWEN38_ARTIFACTS` to the installed model, `DS4_ISSUE106_BAD_CHECKPOINT` to that real old file, and `DS4_ISSUE106_DIR` to a temporary output directory. Run each ignored test separately and serially through `target/debug/test-supervisor 114688 60 120 --command --ignored --nocapture --test-threads=1`: - `issue106_complete_state_and_checkpoint_recovery` - `issue106_product_discards_old_checkpoint_and_recovers_cold` - `issue106_upstream_exit_matrix` (also `MTPLX_CONTEXT_COPY=0`, `DS4_ISSUE106_ORACLE` pointing to fresh oracle JSONL). Generate the oracle with `MTPLX_CONTEXT_COPY=0 tools/issue106-reference.py `, with pytest and the pinned reference dependencies installed in that isolated environment. Run `qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state` separately with a 4096 MiB supervisor cap and `DS4_QWEN38_CHECKPOINT_DIR` set. All four mandatory commit gates and individual supervisor outcomes are recorded in `checks.json`.