Retire the Qwen AR pipeline

docs/issue104-ar/README.md

@@ -0,0 +1,120 @@
+# Issue 104: retire the Qwen AR pipeline
+
+The Qwen AR driver now samples every token through `sample_from_logits` and
+the request's PCG64 stream, following MTPLX `9882703f3105363ddc37eca9f97aa09a1d387112`.
+The obsolete pipeline option, admission, keyed decode loop, cached speculative
+tail, model mode and compile-policy lane are removed. Classic tail capture
+forwards the final sampled token before banking its state.
+
+The server no longer supplies or reads `MTPLX_AR_PIPELINE`. The installed-model
+reference harness cannot enable it either. Generic PLE staging/resident-table
+fields remain for their independently tested callers. Their classic defaults
+are preserved. The historical lazy sampling primitive and its scalar keyed
+categorical wrapper are test-only, retained to reproduce the original sampling
+law receipt and exercise the existing RNG fixtures.
+
+## Baseline and oracle
+
+Pre-fix production revision: `5773942c8ffe41352104d673ff0eef949df10c71`, with only
+the installed-model diagnostic added. Model: unchanged Bare-Speed revision
+`74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`; MLX 0.32.2; Apple M5 Max / 128 GiB.
+Each model process ran serially under `test-supervisor 114688 60 120`.
+
+| Configuration | Actual old lane | Temperature .6, p .95, k 20, seed 0, eight tokens |
+| --- | --- | --- |
+| SSD, pipeline disabled | Classic | 314, 279, 198, 2, 20732, 198, 2, 198 |
+| SSD, pipeline enabled | Classic | Same tokens and every final-state hash |
+| Full resident table, pipeline disabled | Classic | Same tokens and every final-state hash |
+| Full resident table, pipeline enabled | Keyed pipeline | 314, 279, 35067, 82, 13, 220, 19, 15 |
+
+The temperature-1 control also diverged at token 3 only with the resident
+pipeline. Its classic tokens were `314,279,198,2,20732,20086,198,2053`; pipeline
+tokens were `314,279,695,47,4920,82,5212,2502`. Full resident attachment was asserted,
+and the old driver's actual `Decoded::pipeline` was recorded, rather than
+inferring engagement from an environment flag.
+
+`tools/issue104-reference.py` runs the pinned original `generate_ar` and
+SessionBank against the installed artifacts. The checked-in
+`tests/fixtures/issue104-ar-model.json` contains cold and warm token IDs, final
+logits and every trunk-state array's shape/hash. Cold receipts equal both
+pre-fix native classic baselines in full. The warm turn restores all 11 committed
+tokens and adds suffix `[1075]` before decoding eight more tokens.
+
+Native regression `issue104_installed_ar_stream_and_state` checks those complete
+receipts with synchronous and asynchronous evaluation. It also cancels a warm
+turn after three callbacks, checks that the saved state is intact, and retries
+from that state. Every completed turn asserts that PLE staging remains enabled
+and resident preference remains in classic mode. Run with `MTPLX_AR_PIPELINE=1`
+to exercise an explicit obsolete flag without recreating the removed lane.
+
+## Scope review
+
+The checked route is request/session ownership → `Execution::decode_ar` →
+`ar::decode` → classic sampling → final-token capture. No production AR caller
+can reach the lazy primitive. Every former admission and mode call in policy,
+model, decoder, binding and installed reference tests was removed. GDN compilation
+still obeys its two existing environment gates, including explicit-off precedence.
+Normal prefill and MTP retain their compilation/staging paths. DS4/GLM sampling,
+RNG, inference and Metal kernels have no changes.
+
+The 1,120 fresh MTPLX orchestration fixtures cover both obsolete flag values,
+sync/async evaluation, official .6/.95/k20, 1/.95/k20, .5/k2, p=0/1, greedy,
+presence penalty, zero/one/two/three/twelve-token limits, stops and final capture.
+Existing sampler fixtures additionally exercise tied logits, frequency penalties
+and exact PCG64 continuation. The historical primitive-law receipt remains in
+`docs/issue103-analysis/native-sampling-final.log`.
+
+The server-policy oracle's `--server-only` mode compares all existing native
+defaults and verifies pipeline removal across 268 environment/MTP cases.
+Current upstream additionally supplies defaults for other, unported features and
+has replaced its old boundary-count diagnostic API; this scoped check does not
+claim those unrelated policy extensions are implemented.
+
+## Timing interpretation
+
+`receipts.jsonl` records measured decode seconds separately from state equality
+and process memory peaks. These are bounded eight-token correctness probes,
+including first-use and memory-pressure effects, not steady-state throughput
+claims. Old resident classic/pipeline times at .6 were 7.576/17.518 seconds;
+at 1.0 they were 1.332/5.452 seconds. SSD classic repeated at .6 took
+0.649/1.120 seconds and at 1.0 took 1.280/1.273 seconds. No overlap is retained
+to preserve these old timings.
+
+After removal, synchronous SSD decode took 0.642/1.309 seconds at temperatures
+.6/1.0; synchronous resident decode took 0.813/1.427 seconds. Async controls took
+1.182/1.298 seconds (SSD) and 1.435/1.426 seconds (resident). All produce the
+classic stream and state. Peak process physical footprint was 72.5 GB for the
+SSD regression and 104.5 GB for the resident regression.
+
+## Completed validation
+
+- Both installed-model regressions passed with the obsolete flag explicitly on,
+  including warm state, cancellation/retry, sync/async and pre-fix equality.
+- All 1,120 AR orchestration fixtures, 1,080 sampler-entry fixtures, the penalty
+  and distribution/RNG fixtures, the original sampling-law diagnostic, and the
+  connected decoder/cache control passed under the supervisor.
+- The scoped 268-case current server-default comparison passed. The ordinary
+  suite includes DS4's 64 exact token/RNG fixtures and compile-policy controls.
+- `cargo fmt --all -- --check`, warnings-denied Clippy, `make bundle` including
+  signing, and `cargo test --all-features` passed (308 application tests plus
+  two tool suites of four tests each; 211 opt-in application tests ignored).
+- Every artifact SHA256 was re-read and matched the unchanged revision;
+  see `environment.json`. Compact run results are in `checks.json` and
+  `receipts.jsonl`. Source review found no DS4/GLM changes and no remaining
+  production AR-pipeline admission or mode path.
+
+## Reproduction
+
+Build the test executable with `cargo test --all-features --bin ds4-server
+--no-run`; pass that executable, not Cargo, to the supervisor. Set
+`DS4_QWEN38_ARTIFACTS` to the installed artifact directory, and set
+`MTPLX_COMPILED_GDN=1 MTPLX_QWEN4EXP_COMPILE=1`. Run
+`issue104_installed_ar_stream_and_state --ignored --nocapture --test-threads=1`
+once with default SSD policy and once with `MTPLX_NGRAM_RESIDENT=1` and adequate
+RAM. Set `MTPLX_AR_PIPELINE=1` on both runs. Optional `DS4_ISSUE104_BASELINE`
+compares the cold records to a separately saved pre-fix classic receipt.
+
+For the isolated Python oracle, point `PYTHONPATH` at pinned MTPLX, use the
+MLX-0.32.2 environment, the same compilation flags, and
+`MTPLX_SUSTAINED_PREFILL=1 MTPLX_PREFILL_CHUNK_SIZE=2048`. Run
+`tools/issue104-reference.py MODEL_DIRECTORY` under the same supervisor.

docs/issue104-ar/checks.json

@@ -0,0 +1,36 @@
+{
+  "clippy": [
+    "    Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.49s"
+  ],
+  "bundle": [
+    "    Finished `release` profile [optimized] target(s) in 49.59s",
+    "target/release/DS4Server.app: replacing existing signature"
+  ],
+  "tests": [
+    "    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.25s",
+    "test result: ok. 308 passed; 0 failed; 211 ignored; 0 measured; 0 filtered out; finished in 6.01s",
+    "test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s",
+    "test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.30s"
+  ],
+  "mtplx_ar_driver_matches_reference": [
+    "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 2.10s"
+  ],
+  "mtplx_sampler_entry_matches_reference": [
+    "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 56.60s"
+  ],
+  "mtplx_sampler_penalties_match_reference": [
+    "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 7.94s"
+  ],
+  "mtplx_sampler_distribution_and_rng_match_reference": [
+    "test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 519 filtered out; finished in 0.00s"
+  ],
+  "issue103_sampling_receipts": [
+    "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 1.48s"
+  ],
+  "mtplx_decoder_connected_attention_and_moe_advance_caches": [
+    "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 0.79s"
+  ],
+  "fmt": "passed",
+  "diff_check": "passed",
+  "scoped_current_server_policy": "268 cases passed"
+}

docs/issue104-ar/environment.json

@@ -0,0 +1,45 @@
+{
+  "base_revision": "5773942c8ffe41352104d673ff0eef949df10c71",
+  "oracle_revision": "9882703f3105363ddc37eca9f97aa09a1d387112",
+  "model_revision": "74559cdf34fbfc0b593de72d17e93f37fd4f9ea7",
+  "model_root": "/Users/gb/Library/Application Support/de.rfc1437.ds4server/models/qwen3.8-flash-next",
+  "artifact_verification": "Every SHA256 re-read and matched on 2026-10-04",
+  "artifact_sha256": {
+    "LICENSE": "a0dc422560841fd68e06d974907f8b4c709bca44a67daad2b528437bdf676c08",
+    "chat_template.jinja": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
+    "config.json": "826302af9ff3293cfc9975c9a5a96546fa0aa9f0808505adfc30a62fef806965",
+    "generation_config.json": "e70c136c1b78ddc1fb0905bac8e733a4dc448d4f852a5dd75143fffc70be550e",
+    "merges.txt": "a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d",
+    "model-00001-of-00017.safetensors": "a27232c9434b9d8961f198cf36f44e228b3f23a5f8d874ba4ca8af32a1b23ffc",
+    "model-00002-of-00017.safetensors": "6268ae80950f6b935fca672fae5aa455eb3ea4f139730e54e6e5f8581faa94dc",
+    "model-00003-of-00017.safetensors": "71a9abbd420dce567f2c06593bfee2d6cea681989d51eaef1b1c5dffd65a844b",
+    "model-00004-of-00017.safetensors": "c8112da735bb1c8dbc4757926ab8ec1eba7a707a8767c7e43c6150586cf63a4c",
+    "model-00005-of-00017.safetensors": "48258b931e7596c1b43c8b38c044060787ef6d85cca2c1935852bc962801f792",
+    "model-00006-of-00017.safetensors": "f357d2d66f7b768ebb19ba464556bbe46c572cf3cd1f8226faddf9216b150d0c",
+    "model-00007-of-00017.safetensors": "d3152e808d013e3afa4a9c391f6d8088e74a10303c89d45fb61dad7e457160f6",
+    "model-00008-of-00017.safetensors": "1d93bd4872f338fa19416bac64ebeb05aedf2721a07ac48e37c53921252c95c1",
+    "model-00009-of-00017.safetensors": "d22d4be580296e3d4fa5f4fe5902c8fc448d8204b6dcde4641f87b9c675930ca",
+    "model-00010-of-00017.safetensors": "111d0166c888cabb7d6fc19c4bb7eb42e2b51946f71b4983dd750b031755637e",
+    "model-00011-of-00017.safetensors": "73dad882cf94a5486cfe0dcda876b0f740556bc469608d2ced7a07540e699803",
+    "model-00012-of-00017.safetensors": "a6057d5a06276461760a4f4e88991550a43abdc5f8afacef06e860f5e7843393",
+    "model-00013-of-00017.safetensors": "ff25dc6e1c250c9b9ece83d59111a4c4055dad1b1f160546c614dcc17395b152",
+    "model-00014-of-00017.safetensors": "d8c9de3ae28abccc8e3da09dc92fbbed4039d6d65dabb2657edebc11b2b09652",
+    "model-00015-of-00017.safetensors": "a5438d468074c709febdeadb9f6a2db54fa9fb30a37c33de78584f28d6fa1191",
+    "model-00016-of-00017.safetensors": "4a07409bd568cd1435641495d90c805dcb0d0e5d8dc64409273d604a7d06b342",
+    "model-00017-of-00017.safetensors": "9c064e0b8dc0054e2b453fe2b8450c287e1ede6e74bb74ecf5aaed1ef9271913",
+    "model-vision.safetensors": "5aff6983f36f52acbecc1f61f703ae7b927e74525bf3b1133ba276bb7a5b52b1",
+    "model.safetensors.index.json": "81e77f5b25bb29e25e239ae2f9f324c9cada379ee600441d6c229f1d95a13443",
+    "mtp.safetensors": "1bd4b2fc20a93b53f22b4cab0b14d68bc0fe666bb64e4e2f898872c30d7d26ae",
+    "mtplx_runtime.json": "381c7666872f878542494a48a874bb0784c72a955d18cc077324d3c9e74b8cdb",
+    "ngram-table.safetensors": "af7f0880c1798e60353b07cc6b0faa0d41f78734818d858ff013ab7e3bce9de5",
+    "preprocessor_config.json": "27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516",
+    "processor_config.json": "dcc7ce0121d6d1c514804e68e5673e04c239ad2547339ea6330289d272a748f8",
+    "tokenizer.json": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3",
+    "tokenizer_config.json": "b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27",
+    "video_preprocessor_config.json": "7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13",
+    "vocab.json": "ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003"
+  },
+  "physical_memory_bytes": 137438953472,
+  "os": "macOS 26.6.2 (25G83)",
+  "hardware": "Apple M5 Max"
+}

docs/issue104-ar/receipts.jsonl

@@ -0,0 +1,37 @@
+{"run":"before-ssd","event":"issue104_environment","resident_tables":[false],"mtp":false}
+{"run":"before-ssd","event":"issue104_ar","temperature":0.6,"legacy":false,"pipeline":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":0.64901575,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
+{"run":"before-ssd","event":"issue104_ar","temperature":0.6,"legacy":true,"pipeline":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":1.119514625,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
+{"run":"before-ssd","event":"issue104_ar","temperature":1.0,"legacy":false,"pipeline":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.279887375,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
+{"run":"before-ssd","event":"issue104_ar","temperature":1.0,"legacy":true,"pipeline":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.273476917,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
+{"run":"before-ssd","event":"test_resource_summary","elapsed_ms":34988,"peak":{"cpu_time_ns":32497220166,"resident_bytes":59135393792,"physical_bytes":72378445512,"peak_physical_bytes":72378478280,"disk_read_bytes":71780679680,"disk_write_bytes":9781248},"error":null}
+{"run":"before-resident","event":"issue104_environment","resident_tables":[true],"mtp":false}
+{"run":"before-resident","event":"issue104_ar","temperature":0.6,"legacy":false,"pipeline":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":7.5761748749999995,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
+{"run":"before-resident","event":"issue104_ar","temperature":0.6,"legacy":true,"pipeline":true,"tokens":[314,279,35067,82,13,220,19,15],"seconds":17.51813525,"state_sha256":"780fc40e50d844db43f003860b865aa123e28d84be3ad7ac585c5197ac3e502f"}
+{"run":"before-resident","event":"issue104_ar","temperature":1.0,"legacy":false,"pipeline":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.332490958,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
+{"run":"before-resident","event":"issue104_ar","temperature":1.0,"legacy":true,"pipeline":true,"tokens":[314,279,695,47,4920,82,5212,2502],"seconds":5.452344583,"state_sha256":"6d85808248625e1691696f38ebd1cccc4e46d5e96439d7e0fa156841014b33ff"}
+{"run":"before-resident","event":"test_resource_summary","elapsed_ms":115309,"peak":{"cpu_time_ns":72117394541,"resident_bytes":55280517120,"physical_bytes":104389150752,"peak_physical_bytes":104440955808,"disk_read_bytes":119938990080,"disk_write_bytes":18446744073709518848},"error":null}
+{"run":"after-ssd","event":"issue104_environment","resident_tables":[false],"mtp":false}
+{"run":"after-ssd","event":"issue104_ar","temperature":0.6,"asynchronous":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":0.642156958,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
+{"run":"after-ssd","event":"issue104_warm","temperature":0.6,"asynchronous":false,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
+{"run":"after-ssd","event":"issue104_ar","temperature":0.6,"asynchronous":true,"tokens":[314,279,198,2,20732,198,2,198],"seconds":1.182015375,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
+{"run":"after-ssd","event":"issue104_warm","temperature":0.6,"asynchronous":true,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
+{"run":"after-ssd","event":"issue104_ar","temperature":1.0,"asynchronous":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.30897475,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
+{"run":"after-ssd","event":"issue104_warm","temperature":1.0,"asynchronous":false,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
+{"run":"after-ssd","event":"issue104_ar","temperature":1.0,"asynchronous":true,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.298059,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
+{"run":"after-ssd","event":"issue104_warm","temperature":1.0,"asynchronous":true,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
+{"run":"after-ssd","event":"test_resource_summary","elapsed_ms":62881,"peak":{"cpu_time_ns":65904669875,"resident_bytes":61079650304,"physical_bytes":72499870168,"peak_physical_bytes":72499870168,"disk_read_bytes":71786532864,"disk_write_bytes":344064},"error":null}
+{"run":"after-resident","event":"issue104_environment","resident_tables":[true],"mtp":false}
+{"run":"after-resident","event":"issue104_ar","temperature":0.6,"asynchronous":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":0.813106875,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
+{"run":"after-resident","event":"issue104_warm","temperature":0.6,"asynchronous":false,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
+{"run":"after-resident","event":"issue104_ar","temperature":0.6,"asynchronous":true,"tokens":[314,279,198,2,20732,198,2,198],"seconds":1.435416458,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
+{"run":"after-resident","event":"issue104_warm","temperature":0.6,"asynchronous":true,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
+{"run":"after-resident","event":"issue104_ar","temperature":1.0,"asynchronous":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.426774333,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
+{"run":"after-resident","event":"issue104_warm","temperature":1.0,"asynchronous":false,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
+{"run":"after-resident","event":"issue104_ar","temperature":1.0,"asynchronous":true,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.4262525,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
+{"run":"after-resident","event":"issue104_warm","temperature":1.0,"asynchronous":true,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
+{"run":"after-resident","event":"test_resource_summary","elapsed_ms":115119,"peak":{"cpu_time_ns":89721565333,"resident_bytes":62532075520,"physical_bytes":104507069232,"peak_physical_bytes":104507069232,"disk_read_bytes":104724783104,"disk_write_bytes":0},"error":null}
+{"run":"oracle-warm","event":"issue104_oracle","temperature":0.6,"warm":false,"tokens":[314,279,198,2,20732,198,2,198],"cached_tokens":0,"seconds":4.613554999988992,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
+{"run":"oracle-warm","event":"issue104_oracle","temperature":0.6,"warm":true,"tokens":[314,279,198,2,20732,198,2,198],"cached_tokens":11,"seconds":0.18006633399636485,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
+{"run":"oracle-warm","event":"issue104_oracle","temperature":1.0,"warm":false,"tokens":[314,279,198,2,20732,20086,198,2053],"cached_tokens":0,"seconds":0.20107879099668935,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
+{"run":"oracle-warm","event":"issue104_oracle","temperature":1.0,"warm":true,"tokens":[25,198,2,198,15666,25835,1608,678],"cached_tokens":11,"seconds":0.17781587500940077,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
+{"run":"oracle-warm","event":"test_resource_summary","elapsed_ms":15510,"peak":{"cpu_time_ns":18162759666,"resident_bytes":61776969728,"physical_bytes":72906603160,"peak_physical_bytes":72906619568,"disk_read_bytes":103772155904,"disk_write_bytes":0},"error":null}

src/engine/metal/qwen_mtplx.rs

@@ -29,6 +29,8 @@ mod hyper;
 mod indexing;
 #[cfg(test)]
 mod issue103_analysis;
+#[cfg(test)]
+mod issue104;
 mod linear;
 mod load;
 mod mlp;

src/engine/metal/qwen_mtplx/ar.rs

@@ -4,9 +4,9 @@
 use super::array::{Array, Dtype};
 use super::execution::Throttle;
 use super::model::Output;
-use super::sampling::{SamplerConfig, lazy_sample, sample_from_logits};
+use super::ops;
+use super::sampling::{SamplerConfig, sample_from_logits};
 use super::stream::{Stream, Streams};
-use super::{ops, random};
 use crate::engine::Rng;
 use std::time::Instant;
 
@@ -15,36 +15,21 @@ pub(super) struct Options<'a> {
     pub(super) max_tokens: usize,
     pub(super) seed: u64,
     pub(super) stop_tokens: &'a [i64],
-    pub(super) pipeline: bool,
     pub(super) sync_eval: bool,
     pub(super) return_hidden: bool,
     pub(super) capture_final_state: bool,
 }
 
-impl Options<'_> {
-    pub(super) fn pipeline_eligible(&self, sampler: &SamplerConfig) -> bool {
-        self.pipeline
-            && sampler.temperature > 0.0
-            && sampler.top_k > 1
-            && sampler.top_k < 4096
-            && sampler.presence_penalty == 0.0
-            && sampler.frequency_penalty == 0.0
-            && self.max_tokens > 2
-    }
-}
-
 #[derive(Default)]
 pub(super) struct Decoded {
     pub(super) tokens: Vec<i64>,
     pub(super) final_logits: Option<Array>,
     pub(super) final_state_error: Option<String>,
-    pub(super) pipeline: bool,
     pub(super) verify_calls: usize,
     pub(super) forward_seconds: f64,
     pub(super) eval_seconds: f64,
     pub(super) decode_seconds: f64,
     pub(super) elapsed_seconds: f64,
-    cached_final_row: Option<Array>,
 }
 
 pub(super) fn last_row(logits: &Array, stream: Stream) -> Result<Array, String> {
@@ -65,14 +50,12 @@ fn token_array(token: i64) -> Result<Array, String> {
     )
 }
 
-/// The model owner enables/disables pipeline mode around this call, including
+/// Every token uses the request PCG64 stream, matching classic generate_ar.
-/// errors from the callback. Forward consumes resident token arrays unchanged.
 #[allow(clippy::too_many_arguments)]
 pub(super) fn decode(
     initial_logits: &Array,
     sampler: &SamplerConfig,
     options: &Options<'_>,
-    pipeline_ready: bool,
     streams: &Streams,
     stream: Stream,
     throttle: &mut Throttle,
@@ -83,7 +66,6 @@ pub(super) fn decode(
     let mut result = Decoded::default();
     let mut rng = Rng::new_qwen(options.seed);
     let mut logits = initial_logits.clone();
-    let mut final_row = None;
     let stopped = |token: i64| options.stop_tokens.contains(&token);
     let mut commit = |tokens: &mut Vec<i64>, token| -> Result<(), String> {
         tokens.push(token);
@@ -92,54 +74,6 @@ pub(super) fn decode(
         }
         Ok(())
     };
-    if options.pipeline_eligible(sampler) && pipeline_ready {
-        result.pipeline = true;
-        let mut key = random::key(options.seed & 0x7fff_ffff)?;
-        let row = ops::reshape(&logits, &[-1], stream)?;
-        let token = sample_from_logits(&row, sampler, &mut rng, &[], &[], streams, stream)?.0;
-        commit(&mut result.tokens, token)?;
-        if !stopped(token) {
-            let mut lane_step = |token: &Array| -> Result<(Array, Array), String> {
-                let (next, sub) = random::split(&key, stream)?;
-                key = next;
-                let out = forward(&ops::reshape(token, &[1, 1], stream)?, false)?;
-                let row = last_row(
-                    out.logits.as_ref().ok_or("AR forward omitted logits")?,
-                    stream,
-                )?;
-                let next = lazy_sample(&ops::reshape(&row, &[-1], stream)?, sampler, &sub, stream)?;
-                Ok((row, next))
-            };
-            let started = Instant::now();
-            let first_power_started = throttle.begin();
-            let (mut _row_lazy, mut token_lazy) = lane_step(&token_array(token)?)?;
-            ops::evaluate(streams, std::slice::from_ref(&token_lazy), stream, true)?;
-            result.forward_seconds += started.elapsed().as_secs_f64();
-            throttle.pause(streams, stream, first_power_started)?;
-            loop {
-                let power_started = throttle.begin();
-                let built = Instant::now();
-                let (row_next, token_next) = lane_step(&token_lazy)?;
-                ops::evaluate(streams, std::slice::from_ref(&token_next), stream, true)?;
-                let build_elapsed = built.elapsed().as_secs_f64();
-                let waited = Instant::now();
-                let token = ops::read_i64(&token_lazy, streams, stream)?[0];
-                let wait_elapsed = waited.elapsed().as_secs_f64();
-                result.forward_seconds += build_elapsed;
-                result.eval_seconds += wait_elapsed;
-                result.decode_seconds += build_elapsed + wait_elapsed;
-                result.verify_calls += 1;
-                throttle.pause(streams, stream, power_started)?;
-                commit(&mut result.tokens, token)?;
-                if stopped(token) || result.tokens.len() >= options.max_tokens {
-                    final_row = Some(row_next);
-                    break;
-                }
-                _row_lazy = row_next;
-                token_lazy = token_next;
-            }
-        }
-    } else {
     let mut counts = Vec::<(i64, i64)>::new();
     for step in 0..options.max_tokens {
         let row = ops::reshape(&logits, &[-1], stream)?;
@@ -150,8 +84,7 @@ pub(super) fn decode(
             result.eval_seconds += elapsed;
             result.decode_seconds += elapsed;
         }
-            let token =
+        let token = sample_from_logits(&row, sampler, &mut rng, &counts, &[], streams, stream)?.0;
-                sample_from_logits(&row, sampler, &mut rng, &counts, &[], streams, stream)?.0;
         commit(&mut result.tokens, token)?;
         if sampler.presence_penalty != 0.0 || sampler.frequency_penalty != 0.0 {
             if let Some((_, count)) = counts.iter_mut().find(|(id, _)| *id == token) {
@@ -178,15 +111,13 @@ pub(super) fn decode(
         logits = last_row(&next, stream)?;
         throttle.pause(streams, stream, power_started)?;
     }
-    }
     // Match generate_ar: response timing ends before session-bank tail capture.
     result.elapsed_seconds = started_all.elapsed().as_secs_f64();
-    result.cached_final_row = final_row;
     Ok(result)
 }
 
-/// Called after the owner restores non-pipeline mode, as in generate_ar's
+/// Extend the cache through the last sampled token, as in generate_ar.
-/// finally block. Capture failures preserve a successfully finished response.
+/// Capture failures preserve a successfully finished response.
 pub(super) fn capture_final_state(
     result: &mut Decoded,
     options: &Options<'_>,
@@ -194,20 +125,16 @@ pub(super) fn capture_final_state(
     stream: Stream,
     mut forward: impl FnMut(&Array, bool) -> Result<Output, String>,
 ) {
-    let final_row = result.cached_final_row.take();
     if options.capture_final_state
         && let Some(&last) = result.tokens.last()
     {
         let capture = (|| {
-            let row = match final_row {
+            let row = last_row(
-                Some(row) => row,
-                None => last_row(
                 &forward(&token_array(last)?, false)?
                     .logits
                     .ok_or("AR tail omitted logits")?,
                 stream,
-                )?,
+            )?;
-            };
             ops::evaluate(streams, std::slice::from_ref(&row), stream, false)?;
             Ok::<_, String>(row)
         })();
@@ -235,11 +162,13 @@ fn mtplx_ar_driver_matches_reference() {
         .lines()
         .map(|line| serde_json::from_str::<serde_json::Value>(line).unwrap())
         .collect::<Vec<_>>();
-    assert_eq!(cases.len(), 480);
+    assert_eq!(cases.len(), 1120);
     for (index, case) in cases.iter().enumerate() {
         eprintln!("AR driver reference {index}/{}", cases.len());
         let sampler = SamplerConfig {
             temperature: case["temperature"].as_f64().unwrap(),
+            top_p: case["top_p"].as_f64().unwrap(),
+            top_k: case["top_k"].as_i64().unwrap() as i32,
             presence_penalty: case["penalty"].as_f64().unwrap(),
             ..Default::default()
         };
@@ -253,7 +182,6 @@ fn mtplx_ar_driver_matches_reference() {
             max_tokens: case["limit"].as_u64().unwrap() as usize,
             seed: 4294967297,
             stop_tokens: &stops,
-            pipeline: case["pipeline"].as_bool().unwrap(),
             sync_eval: !case["asynchronous"].as_bool().unwrap(),
             return_hidden: false,
             capture_final_state: case["capture"].as_bool().unwrap(),
@@ -278,7 +206,6 @@ fn mtplx_ar_driver_matches_reference() {
             &row,
             &sampler,
             &options,
-            true,
             &streams,
             stream,
             &mut Throttle::new(100).unwrap(),
@@ -327,11 +254,6 @@ fn mtplx_ar_driver_matches_reference() {
             case["final"],
             "final cache row case {index}"
         );
-        assert_eq!(
-            result.pipeline,
-            !case["modes"].as_array().unwrap().is_empty(),
-            "pipeline admission case {index}"
-        );
         assert_eq!(
             serde_json::json!(result.verify_calls),
             case["verify_calls"],

src/engine/metal/qwen_mtplx/binding.rs

@@ -674,10 +674,6 @@ fn mtplx_installed_module_binding_matches_reference() {
             );
         }
         assert_eq!(serde_json::json!(tables), sidecar_case["tables"]);
-        assert_eq!(
-            model.set_ar_pipeline_mode(true),
-            sidecar_case["ar_ready"].as_bool().unwrap()
-        );
         drop(model);
         assert!(
             bank.materialize(&["missing.weight".into()], &streams, gpu)

src/engine/metal/qwen_mtplx/decoder.rs

@@ -532,7 +532,6 @@ fn mtplx_decoder_connected_attention_and_moe_advance_caches() {
             CompilePolicy::from_values(Some("1"), None),
         )
         .unwrap();
-        assert_eq!(model.set_ar_pipeline_mode(true), !with_ple);
         let mut caches = model.make_cache(4);
         let mut offset = 0;
         for (rows, keep, emit, capture) in [

src/engine/metal/qwen_mtplx/execution.rs

@@ -827,15 +827,10 @@ impl Execution {
         finish_decode: impl FnOnce(f64) -> Result<(), String>,
     ) -> Result<ar::Decoded, String> {
         let graph_options = self.graph_options(false, false)?;
-        let lane = options.pipeline_eligible(sampler)
-            && self
-                .model
-                .with_dependent_mut(|_, model| model.set_ar_pipeline_mode(true));
         let mut result = ar::decode(
             logits,
             sampler,
             options,
-            lane,
             &self.streams,
             self.stream,
             &mut self.decode_throttle,
@@ -859,10 +854,6 @@ impl Execution {
             },
             emit,
         );
-        if lane {
-            self.model
-                .with_dependent_mut(|_, model| model.set_ar_pipeline_mode(false));
-        }
         if let Ok(decoded) = &mut result {
             finish_decode(decoded.elapsed_seconds)?;
             ar::capture_final_state(
@@ -1185,7 +1176,6 @@ fn mtplx_full_installed_ar_matches_reference() {
                     max_tokens: usize::MAX,
                     seed: end["seed"].as_u64().unwrap(),
                     stop_tokens: &stops,
-                    pipeline: flag("MTPLX_AR_PIPELINE"),
                     sync_eval: !flag("MTPLX_ASYNC_AR")
                         || !std::env::var("MTPLX_EVAL_AUDIT")
                             .unwrap_or_default()
@@ -1222,7 +1212,7 @@ fn mtplx_full_installed_ar_matches_reference() {
         println!(
             "{}",
             serde_json::json!({"event":"canonical_ar_end", "tokens":result.tokens, "verify_calls":result.verify_calls,
-            "decode_elapsed_seconds":result.elapsed_seconds, "prefill_seconds":prepared.eval_seconds, "finish_reason":"stop", "pipeline":result.pipeline})
+            "decode_elapsed_seconds":result.elapsed_seconds, "prefill_seconds":prepared.eval_seconds, "finish_reason":"stop"})
         );
         phase.store(EvaluationPhase::Finishing as u8, Ordering::Relaxed);
     }));
@@ -1461,7 +1451,6 @@ fn mtplx_full_installed_ping_forward_matches_reference() {
                         max_tokens: usize::MAX,
                         seed: 0,
                         stop_tokens: &stops,
-                        pipeline: false,
                         sync_eval: true,
                         return_hidden: false,
                         capture_final_state: false,
@@ -1486,7 +1475,6 @@ fn mtplx_full_installed_ping_forward_matches_reference() {
             emitted.push(token as i64);
             assert_eq!(ar_result.tokens, emitted);
             assert_eq!(ar_result.verify_calls, generated.len());
-            assert!(!ar_result.pipeline);
         } else {
             assert!(
                 !receipts.iter().any(|r| r["event"] == "decode_reference"),

src/engine/metal/qwen_mtplx/issue104.rs

@@ -0,0 +1,224 @@
+//! Full installed-model AR regression; run serially under test-supervisor.
+use super::ar;
+use super::array::Array;
+use super::execution::Execution;
+use super::sampling::SamplerConfig;
+use super::session_cache::{DecodeOptions, SessionSnapshot, State, TurnOptions};
+
+fn fingerprint(execution: &Execution, array: &Array) -> serde_json::Value {
+    let values = execution.values_f32(array).unwrap();
+    assert!(values.iter().all(|v| v.is_finite()));
+    serde_json::json!({"shape":array.layout().shape(), "sha256":
+        super::super::qwen_mtplx_tests::hex_digest(&values.iter().flat_map(|v|v.to_le_bytes()).collect::<Vec<_>>())})
+}
+
+fn state(execution: &Execution, saved: &SessionSnapshot) -> serde_json::Value {
+    let states = saved
+        .trunk
+        .iter()
+        .map(|s| {
+            s.as_ref().map(|s| {
+                let arrays = match s {
+                    State::Recurrent(arrays) => arrays.as_slice(),
+                    State::Qsa(arrays) => arrays.as_slice(),
+                };
+                arrays
+                    .iter()
+                    .map(|a| a.as_ref().map(|a| fingerprint(execution, a)))
+                    .collect::<Vec<_>>()
+            })
+        })
+        .collect::<Vec<_>>();
+    serde_json::json!({"ids":saved.token_ids,"logits":fingerprint(execution,&saved.logits),"states":states})
+}
+
+#[test]
+#[ignore = "full pinned Bare-Speed model; external memory guard required"]
+fn issue104_installed_ar_stream_and_state() {
+    let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
+    eprintln!(
+        "#104 loading unchanged Bare-Speed model; resident={}",
+        super::ngram::resident_policy()
+    );
+    let mut execution = Execution::load(&root, false, false, 100).unwrap();
+    let tables = execution
+        .model
+        .borrow_dependent()
+        .layers
+        .iter()
+        .filter_map(|layer| {
+            layer
+                .ple
+                .as_ref()
+                .map(|ple| ple.embedding.table.resident.is_some())
+        })
+        .collect::<Vec<_>>();
+    assert!(!tables.is_empty());
+    assert!(
+        tables.iter().all(|r| *r == super::ngram::resident_policy()),
+        "full residency must actually attach"
+    );
+    println!(
+        "{}",
+        serde_json::json!({"event":"issue104_environment","resident_tables":tables,"mtp":false})
+    );
+    let ids = [9707, 11, 1075];
+    let prepared = execution
+        .prefill_prompt(
+            &ids,
+            &super::prefill::Options {
+                chunk_size: Some(2048),
+                final_logits_only: true,
+                external_cache_only: false,
+                external_emit_logits: true,
+                return_hidden: false,
+            },
+            |_| Ok(()),
+        )
+        .unwrap();
+    let baseline = execution
+        .capture_session(&ids, &prepared.logits, None, false, false)
+        .unwrap();
+    let mut records = Vec::new();
+    let mut warm_records = Vec::<serde_json::Value>::new();
+    for temperature in [0.6, 1.0] {
+        let sampler = SamplerConfig {
+            temperature,
+            ..Default::default()
+        };
+        let mut expected = None;
+        for asynchronous in [false, true] {
+            let (logits, _, reused) = execution.restore_session(&baseline, &ids).unwrap();
+            assert_eq!(reused, ids.len());
+            let mut emitted = Vec::new();
+            let out = execution
+                .decode_ar(
+                    &logits,
+                    &sampler,
+                    &ar::Options {
+                        max_tokens: 8,
+                        seed: 0,
+                        stop_tokens: &[],
+                        sync_eval: !asynchronous,
+                        return_hidden: false,
+                        capture_final_state: true,
+                    },
+                    |id| {
+                        emitted.push(id);
+                        eprintln!("#104 t={temperature} async={asynchronous}: token {id}");
+                        Ok(())
+                    },
+                    |_| Ok(()),
+                )
+                .unwrap();
+            assert!(out.final_state_error.is_none());
+            for layer in &execution.model.borrow_dependent().layers {
+                if let Some(ple) = &layer.ple {
+                    assert!(!ple.embedding.stage_disabled);
+                    assert!(!ple.embedding.table.prefer_lazy);
+                }
+            }
+            assert_eq!(out.tokens, emitted);
+            let mut key = ids.to_vec();
+            key.extend(out.tokens.iter().map(|id| *id as i32));
+            let saved = execution
+                .capture_session(&key, out.final_logits.as_ref().unwrap(), None, false, false)
+                .unwrap();
+            let value = state(&execution, &saved);
+            if !asynchronous {
+                expected = Some(value.clone());
+            }
+            println!(
+                "{}",
+                serde_json::json!({"event":"issue104_ar","temperature":temperature,"asynchronous":asynchronous,"tokens":out.tokens,"state":value,"seconds":out.elapsed_seconds})
+            );
+            assert_eq!(value, expected.clone().unwrap());
+            let mut prompt = key.clone();
+            prompt.push(1075);
+            let options = TurnOptions {
+                prefill: super::prefill::Options {
+                    chunk_size: Some(2048),
+                    final_logits_only: true,
+                    external_cache_only: false,
+                    external_emit_logits: true,
+                    return_hidden: false,
+                },
+                decode: DecodeOptions::Ar(ar::Options {
+                    max_tokens: 8,
+                    seed: 0,
+                    stop_tokens: &[],
+                    sync_eval: !asynchronous,
+                    return_hidden: false,
+                    capture_final_state: true,
+                }),
+                sampler,
+                draft_sampler: sampler,
+                lazy_kv: false,
+                store_min_suffix: None,
+                fused_max: 512,
+            };
+            // Cancel after forwards have advanced the live cache, then retry
+            // from the committed bank entry. The saved state must survive.
+            let mut callbacks = 0;
+            let cancelled = execution.session_turn(
+                &prompt,
+                Some(&saved),
+                &options,
+                |_| Ok(()),
+                |_| {
+                    callbacks += 1;
+                    if callbacks == 3 {
+                        Err("cancelled".into())
+                    } else {
+                        Ok(())
+                    }
+                },
+                |_| Ok(()),
+            );
+            assert!(matches!(cancelled, Err(ref error) if error == "cancelled"));
+            assert_eq!(state(&execution, &saved), value);
+            let warm = execution
+                .session_turn(
+                    &prompt,
+                    Some(&saved),
+                    &options,
+                    |_| Ok(()),
+                    |_| Ok(()),
+                    |_| Ok(()),
+                )
+                .unwrap();
+            assert_eq!(warm.cached_tokens, key.len());
+            assert!(warm.final_snapshot_error.is_none());
+            let warm_state = state(&execution, warm.snapshot.as_ref().unwrap());
+            println!(
+                "{}",
+                serde_json::json!({"event":"issue104_warm", "temperature":temperature,
+                "asynchronous":asynchronous,"state":warm_state})
+            );
+            if asynchronous {
+                assert_eq!(warm_state, warm_records.last().unwrap()["state"]);
+            } else {
+                warm_records
+                    .push(serde_json::json!({"temperature":temperature,"state":warm_state}));
+            }
+            if !asynchronous {
+                records.push(serde_json::json!({"temperature":temperature,"state":value}));
+            }
+        }
+    }
+    if let Ok(path) = std::env::var("DS4_ISSUE104_BASELINE") {
+        let expected: serde_json::Value =
+            serde_json::from_str(&std::fs::read_to_string(path).unwrap()).unwrap();
+        assert_eq!(serde_json::json!(records), expected);
+    }
+    let oracle: serde_json::Value = serde_json::from_str(include_str!(
+        "../../../../tests/fixtures/issue104-ar-model.json"
+    ))
+    .unwrap();
+    assert_eq!(serde_json::json!(records), oracle["cold"]);
+    assert_eq!(serde_json::json!(warm_records), oracle["warm"]);
+    println!(
+        "{}",
+        serde_json::json!({"event":"issue104_classic_baseline","records":records})
+    );
+}

src/engine/metal/qwen_mtplx/model.rs

@@ -14,7 +14,6 @@ use std::ops::Range;
 pub(super) struct CompilePolicy {
     explicit_off: bool,
     environment: bool,
-    lane: bool,
 }
 
 impl CompilePolicy {
@@ -29,7 +28,6 @@ impl CompilePolicy {
         Self {
             explicit_off,
             environment: !explicit_off && [gdn, family].into_iter().flatten().any(truthy),
-            lane: false,
         }
     }
 
@@ -42,7 +40,7 @@ impl CompilePolicy {
     }
 
     fn enabled(self) -> bool {
-        !self.explicit_off && (self.environment || self.lane)
+        !self.explicit_off && self.environment
     }
 }
 
@@ -305,23 +303,6 @@ impl<'a> TextModel<'a> {
         Ok(())
     }
 
-    pub(super) fn set_ar_pipeline_mode(&mut self, enabled: bool) -> bool {
-        let mut ready = true;
-        for layer in &mut self.layers {
-            let Some(ple) = &mut layer.ple else { continue };
-            if enabled && ple.embedding.table.resident.is_none() {
-                ready = false;
-                continue;
-            }
-            ple.embedding.stage_disabled = enabled;
-            ple.embedding.table.prefer_lazy = enabled;
-        }
-        if ready {
-            self.policy.lane = enabled && !self.policy.explicit_off;
-        }
-        ready
-    }
-
     #[cfg_attr(not(test), expect(dead_code, reason = "Reference diagnostic API"))]
     pub(super) fn clear_verify_capture(cache: &mut [Option<Cache>]) {
         for cache in cache.iter_mut().flatten() {
@@ -777,19 +758,16 @@ fn mtplx_model_compile_policy_and_run_boundaries() {
         assert!(CompilePolicy::from_values(Some(on), None).enabled());
         assert!(CompilePolicy::from_values(None, Some(on)).enabled());
         for off in ["0", "", "false", "off", "anything"] {
-            for mut policy in [
+            for policy in [
                 CompilePolicy::from_values(Some(on), Some(off)),
                 CompilePolicy::from_values(Some(off), Some(on)),
             ] {
-                policy.lane = true;
                 assert!(!policy.enabled());
             }
         }
     }
-    let mut automatic = CompilePolicy::from_values(None, None);
+    let automatic = CompilePolicy::from_values(None, None);
     assert!(!automatic.enabled());
-    automatic.lane = true;
-    assert!(automatic.enabled());
     let environment = CompilePolicy::from_env();
     assert_eq!(
         environment.enabled(),

src/engine/metal/qwen_mtplx/ngram.rs

@@ -127,7 +127,7 @@ impl Table {
                 self.resident = Some(resident);
                 self.prefer_lazy = false;
                 println!(
-                    "[qwen4_exp] ngram table resident: {:.1}G in {:.1}s (pipelined-AR lane armed)",
+                    "[qwen4_exp] ngram table resident: {:.1}G in {:.1}s",
                     nbytes as f64 / (1u64 << 30) as f64,
                     started.elapsed().as_secs_f64()
                 );

src/engine/metal/qwen_mtplx/policy.rs

@@ -53,7 +53,6 @@ pub(super) fn server_overrides(mtp: bool, get: impl Fn(&str) -> Option<String>)
         result.insert("MTPLX_SKIP_VERIFY_SNAPSHOT", "0");
     }
     for key in [
-        "MTPLX_AR_PIPELINE",
         "MTPLX_FAMILY_CAPTURE_COMMIT",
         "MTPLX_FUSED_HC_V3",
         "MTPLX_FUSED_GDN_INPROJ",
@@ -137,7 +136,6 @@ pub(super) fn turn_options(
             max_tokens: usize::MAX,
             seed: 0,
             stop_tokens: &[],
-            pipeline: flag("MTPLX_AR_PIPELINE"),
             sync_eval: !flag("MTPLX_ASYNC_AR")
                 || get("MTPLX_EVAL_AUDIT").is_some_and(|s| !s.is_empty()),
             return_hidden: false,
@@ -182,6 +180,42 @@ pub(super) fn turn_options(
     ))
 }
 
+#[test]
+fn retired_ar_pipeline_flag_cannot_change_server_or_turn_policy() {
+    let mut settings = crate::settings::effective_settings(
+        crate::model::ModelChoice::Qwen38FlashNext,
+        &crate::settings::GenerationPreferences::default(),
+        &crate::settings::RuntimePreferences::default(),
+        std::path::Path::new("/unused-policy-models"),
+    )
+    .unwrap()
+    .engine;
+    settings.speculative.glm_mtp = false;
+    for legacy in [None, Some("0"), Some("1")] {
+        let lookup = |key: &str| {
+            (key == "MTPLX_AR_PIPELINE")
+                .then_some(legacy)
+                .flatten()
+                .map(str::to_owned)
+        };
+        let overrides = server_overrides(false, lookup);
+        assert_eq!(overrides, server_overrides(false, |_| None));
+        let (options, _) = turn_options(&settings, |key| {
+            assert_ne!(key, "MTPLX_AR_PIPELINE", "retired option was read");
+            overrides
+                .get(key)
+                .map(|s| (*s).to_owned())
+                .or_else(|| lookup(key))
+        })
+        .unwrap();
+        let DecodeOptions::Ar(ar) = options.decode else {
+            panic!("AR request changed modes");
+        };
+        assert!(ar.sync_eval && ar.capture_final_state);
+        assert!(!ar.return_hidden);
+    }
+}
+
 #[test]
 #[ignore = "requires DS4_QWEN38_POLICY_REFERENCE from pinned MTPLX; CPU-only"]
 fn qwen_server_policy_matches_original_functions() {
@@ -198,6 +232,23 @@ fn qwen_server_policy_matches_original_functions() {
         let mtp = case["mtp"].as_bool().unwrap();
         let lookup = |key: &str| case["environment"][key].as_str().map(str::to_owned);
         let overrides = server_overrides(mtp, lookup);
+        if case["server_only"] == true {
+            // This scoped comparison owns AR retirement and the existing
+            // defaults. New upstream features are outside #104; retain the
+            // complete legacy policy comparison below for its own oracle.
+            assert!(
+                !case["overrides"]
+                    .as_object()
+                    .unwrap()
+                    .contains_key("MTPLX_AR_PIPELINE")
+            );
+            assert!(!overrides.contains_key("MTPLX_AR_PIPELINE"));
+            for (key, value) in &overrides {
+                assert_eq!(case["overrides"][key], *value, "default {key} case {count}");
+            }
+            count += 1;
+            continue;
+        }
         assert_eq!(
             serde_json::to_value(&overrides).unwrap(),
             case["overrides"],
@@ -253,7 +304,6 @@ fn qwen_server_policy_matches_original_functions() {
         match actual.decode {
             DecodeOptions::Ar(o) => {
                 assert!(!mtp);
-                assert_eq!(o.pipeline, expected["pipeline"]);
                 assert_eq!(o.sync_eval, expected["sync_eval"]);
             }
             DecodeOptions::Mtp(o) => {

src/engine/metal/qwen_mtplx/random.rs

@@ -143,6 +143,7 @@ pub(super) fn scalar(value: f32) -> Result<Array, String> {
 
 /// categorical(sv[None], key): the pinned runtime selects inverse CDF for
 /// one distribution with output shape [1], not the Gumbel-max route.
+#[cfg(test)]
 pub(super) fn categorical_row(
     logits: &Array,
     key: &Array,

src/engine/metal/qwen_mtplx/request.rs

@@ -284,7 +284,6 @@ fn qwen_request_settings_override_backend_policy() {
             max_tokens: 1,
             seed: 1,
             stop_tokens: &[],
-            pipeline: true,
             sync_eval: true,
             return_hidden: false,
             capture_final_state: true,

src/engine/metal/qwen_mtplx/sampling.rs

@@ -150,7 +150,9 @@ fn order_bounded_support(
 
 /// generation._mx_lazy_sample. Keep selection in the original model dtype,
 /// promote only the selected k logits, and return an unevaluated scalar token.
-/// The AR lane owns key splitting; neither this graph nor its RNG reads back.
+/// Historical diagnostic only: reproduces #104 and the keyed RNG fixtures.
+/// Production AR uses sample_from_logits for every token.
+#[cfg(test)]
 pub(super) fn lazy_sample(
     row: &Array,
     config: &SamplerConfig,
@@ -299,7 +301,6 @@ pub(super) fn distribution_from_logits(
 
 /// The ordinary generation._sample_from_logits path. Greedy evaluates model
 /// logits before constructing/evaluating ArgMax and never advances PCG64.
-/// The resident pipelined keyed-device sampler is a separate reference lane.
 pub(super) fn sample_from_logits(
     logits: &Array,
     config: &SamplerConfig,

tools/issue104-reference.py

@@ -0,0 +1,66 @@
+"""Pinned MTPLX full-model classic AR receipts. Reference only; no downloads.
+
+Run serially under test-supervisor with MLX 0.32.2 and pinned MTPLX on PYTHONPATH.
+"""
+import hashlib
+import json
+from pathlib import Path
+import sys
+
+print("#104 importing pinned MTPLX", file=sys.stderr, flush=True)
+import mlx.core as mx
+import numpy as np
+from mlx_lm.utils import load_model
+from mtplx.models import qwen4_exp
+from mtplx.runtime import MTPLXRuntime, _load_tokenizer_resilient
+from mtplx.sampling import SamplerConfig
+from mtplx.generation import generate_ar
+from mtplx.cache_state import snapshot_cache
+from mtplx.session_bank import SessionBank
+
+assert mx.__version__ == "0.32.2"
+root = Path(sys.argv[1])
+print("#104 loading unchanged Bare-Speed model", file=sys.stderr, flush=True)
+model, config = load_model(root, lazy=False, strict=True,
+    get_model_classes=lambda **_: (qwen4_exp.Model, qwen4_exp.ModelArgs))
+model.post_weight_load(root)
+tokenizer = _load_tokenizer_resilient(root, config)
+rt = MTPLXRuntime(model, tokenizer, root, False, None)
+
+def fingerprint(array):
+    values = np.asarray(array.astype(mx.float32)).astype('<f4')
+    assert np.isfinite(values).all()
+    return dict(shape=list(array.shape), sha256=hashlib.sha256(values.tobytes()).hexdigest())
+
+def final_state(out, prompt):
+    assert out.final_state is not None and out.final_state.safe_to_commit
+    final = out.final_state
+    snapshot = snapshot_cache(final.final_trunk_cache)
+    states = [[None if a is None else fingerprint(a) for a in arrays]
+              if arrays is not None else None for arrays in snapshot.states]
+    return dict(ids=prompt + out.tokens, logits=fingerprint(final.final_logits), states=states)
+
+records, warm_records = [], []
+for temperature in (.6, 1.):
+    prompt = [9707, 11, 1075]
+    sampler = SamplerConfig(temperature=temperature, top_p=.95, top_k=20)
+    bank = SessionBank()
+    for warm in (False, True):
+        out = generate_ar(rt, prompt, max_tokens=8,
+            sampler=sampler, seed=0, stop_token_ids=set(), capture_final_state=True,
+            session_bank=bank, session_id="issue104", store_prefix_snapshot=False,
+            token_callback=lambda ids: print(f"#104 t={temperature} warm={warm}: {ids}", file=sys.stderr, flush=True))
+        value = final_state(out, prompt)
+        (warm_records if warm else records).append(dict(temperature=temperature, state=value))
+        print(json.dumps(dict(event="issue104_oracle", temperature=temperature, warm=warm,
+            tokens=out.tokens, state=value, cached_tokens=out.stats.cached_tokens,
+            seconds=out.stats.elapsed_s)), flush=True)
+        if not warm:
+            final = out.final_state
+            assert bank.put(runtime=rt, token_ids=value["ids"], cache=final.final_trunk_cache,
+                logits=final.final_logits, hidden=None, hidden_variant="post_norm",
+                session_id="issue104", mtp_history_policy="cycle", snapshot_epoch=len(value["ids"])) is not None
+            prompt = value["ids"] + [1075]
+        else:
+            assert out.stats.cached_tokens == len(prompt) - 1
+print(json.dumps(dict(event="issue104_classic_baseline", records=records, warm_records=warm_records)), flush=True)

tools/mtplx-binding-reference.py

@@ -48,7 +48,7 @@ def main(root, mode=None):
                     hot_mb=sidecar._hot_cap_rows * sidecar._hot_row_bytes // 2**20,
                     prefetch=sidecar._pool is not None, shape=list(out.shape),
                     sha256=hashlib.sha256(np.asarray(out.astype(mx.float32)).astype('<f4').tobytes()).hexdigest()))
-            print(json.dumps(dict(mask=mask, mtp=mtp, tables=tables, ar_ready=model.set_ar_pipeline_mode(True)),separators=(",", ":")),flush=True)
+            print(json.dumps(dict(mask=mask, mtp=mtp, tables=tables),separators=(",", ":")),flush=True)
             del model
             continue
         parameters = tree_flatten(model.parameters())

tools/mtplx-kernel-fixtures.py

@@ -1627,7 +1627,7 @@ def ar_driver_fixtures():
     """Unchanged generate_ar with a tiny GPU transition-table model, no weights.
 
     Isolates orchestration (not model or throughput parity); actual prefill,
-    samplers, pipeline and final-state capture are all MTPLX implementations.
+    samplers and final-state capture are all MTPLX implementations.
     """
     from mtplx.generation import generate_ar
     from mtplx.runtime import MTPLXRuntime
@@ -1641,11 +1641,8 @@ def ar_driver_fixtures():
     table = mx.array(values.reshape(vocab, vocab))
     class Model:
         def __init__(self):
-            self.calls, self.modes = [], []
+            self.calls = []
         def make_cache(self): return []
-        def set_ar_pipeline_mode(self, enabled):
-            self.modes.append(bool(enabled))
-            return True
         def __call__(self, ids, cache=None, return_hidden=False, emit_logits=True, logits_keep=None):
             self.calls.append((ids, return_hidden))
             out = table[ids]
@@ -1653,12 +1650,11 @@ def ar_driver_fixtures():
     tokenizer = SimpleNamespace(decode=lambda ids, **kwargs: " ".join(map(str, ids)))
     for pipeline in (False, True):
         for asynchronous in (False, True):
-            for temperature, penalty in ((0., 0.), (.6, 0.), (.6, .4)):
+            for temperature, top_p, top_k, penalty in ((0., .95, 20, 0.), (.6, .95, 20, 0.), (.6, .95, 20, .4), (1., .95, 20, 0.), (1., .5, 2, 0.), (1., 0., 2, 0.), (1., 1., 2, 0.)):
                 for limit in (0, 1, 2, 3, 12):
                     for capture in (False, True):
-                        # 13/35 are the first/second sampled pipeline tokens:
+                        # Retain the historical stop IDs and explicitly set the
-                        # exercise immediate EOS and EOS with an in-flight next
+                        # retired flag to prove current MTPLX ignores it.
-                        # graph, not just limits and a non-triggering stop set.
                         for stop_ids in (set(), {0, 5, 36}, {13}, {35}):
                             os.environ["MTPLX_AR_PIPELINE"] = "1" if pipeline else "0"
                             os.environ["MTPLX_ASYNC_AR"] = "1" if asynchronous else "0"
@@ -1666,15 +1662,15 @@ def ar_driver_fixtures():
                             rt = MTPLXRuntime(model, tokenizer, Path("."), False, None)
                             emitted = []
                             out = generate_ar(rt, [1, 2], max_tokens=limit,
-                                sampler=SamplerConfig(temperature=temperature, top_p=.95, top_k=20, presence_penalty=penalty),
+                                sampler=SamplerConfig(temperature=temperature, top_p=top_p, top_k=top_k, presence_penalty=penalty),
                                 seed=4294967297, stop_token_ids=stop_ids, token_callback=lambda ids: emitted.extend(ids), capture_final_state=capture)
                             # Decode calls only: real prefill consumed [1] then [2].
                             calls = [{"ids": np.array(ids).reshape(-1).tolist(), "hidden": hidden} for ids, hidden in model.calls[2:]]
                             final = None if out.final_state is None else dict(shape=out.final_state.final_logits.shape,
                                 sha256=hashlib.sha256(np.array(out.final_state.final_logits).tobytes()).hexdigest())
-                            print(json.dumps(dict(pipeline=pipeline, asynchronous=asynchronous, temperature=temperature,
+                            print(json.dumps(dict(legacy_pipeline=pipeline, asynchronous=asynchronous, temperature=temperature, top_p=top_p, top_k=top_k,
                                 penalty=penalty, limit=limit, capture=capture, stops=sorted(stop_ids), tokens=out.tokens,
-                                emitted=emitted, calls=calls, modes=model.modes, final=final,
+                                emitted=emitted, calls=calls, final=final,
                                 verify_calls=out.stats.verify_calls, finish=out.finish_reason)), flush=True)
 
 

tools/mtplx-policy-reference.py

@@ -6,6 +6,8 @@ import os
 
 parser = argparse.ArgumentParser(description=__doc__)
 parser.add_argument("model")
+parser.add_argument("--server-only", action="store_true",
+    help="Compare server defaults without the older boundary-cache option API")
 args = parser.parse_args()
 print("reference: importing policy functions without model loading", flush=True)
 from mtplx.server.openai import _server_runtime_env_overrides
@@ -47,6 +49,10 @@ for mtp, env in itertools.product((False, True), cases):
     overrides = _server_runtime_env_overrides(argparse.Namespace(
         model=args.model, generation_mode="mtp" if mtp else "ar"), None)
     os.environ.update(overrides)
+    if args.server_only:
+        print(json.dumps(dict(event="policy_reference", mtp=mtp, environment=env,
+            overrides=overrides, server_only=True)), flush=True)
+        continue
     # Native app selects bounded prefill; emulate that request-local policy.
     os.environ["MTPLX_SUSTAINED_PREFILL"] = "1"
     error = None
@@ -59,7 +65,7 @@ for mtp, env in itertools.product((False, True), cases):
                 lazy_kv=_lazy_snapshot_enabled(), store=g._store_on_prefill_env_enabled(),
                 store_min=g._store_on_prefill_min_suffix(), fused_max=g._small_suffix_fused_max(),
                 boundaries=g._gdn_boundary_capture_enabled(), boundary_max=g._gdn_boundary_max_count(),
-                boundary_tail=g._gdn_boundary_tail_interval(), pipeline=g._env_truthy("MTPLX_AR_PIPELINE"),
+                boundary_tail=g._gdn_boundary_tail_interval(),
                 sync_eval=not g._env_truthy("MTPLX_ASYNC_AR") or bool(os.environ.get("MTPLX_EVAL_AUDIT")),
                 capture=g._family_capture_commit_enabled() if mtp else None,
                 lazy_history=g._env_truthy("MTPLX_LAZY_MTP_HISTORY_APPEND"),