Port Qwen row-map SwiGLU expert gathers

docs/issue112-row-map/README.md

@@ -0,0 +1,157 @@
+# Qwen row-map expert gather (#112)
+
+The sorted Qwen gate/up path now retains the uint32 token row map, builds GPU
+per-expert row/tile prefix sums, and dispatches the pinned 64×64×64 NAX kernel.
+The kernel reads original token rows and emits SwiGLU directly. Down projection
+continues through the #110 guarded stock gather. The unsorted decode/verify
+path is unchanged. Packed and separate gate/up weights preserve their original
+activation rounding and fallback ordering.
+
+Oracle: MTPLX `9882703f3105363ddc37eca9f97aa09a1d387112`, MLX 0.32.2.
+Installed Bare-Speed artifact: `74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`;
+artifact hashes and machine details are in ../issue104-ar/environment.json.
+Native pre-port baseline: `8965e77ea173c932e00f70759d0a83ac1c5072f0`.
+
+Only GPU shader source is shipped from the oracle: the MTPLX kernel body and
+its MLX 0.32.2 GPU template/functor closure, with MIT licenses in `metal/`.
+`tools/issue112-extract-metal.py` reproduces those files from the pinned
+reference environment. All scheduling, policy, graph construction, checking,
+model loading and sessions remain Rust. There is no production Python/MLX/C
+runtime dependency and the pinned runtime metallib is unchanged.
+
+## Route and failure contract
+
+- All original tensor/layout predicates, affine BF16/F16, b4/b8, groups
+  32/64/128, N/K multiples of 64, and the 4,096-routed-row threshold apply.
+- Below-threshold, non-NAX and ineligible inputs retain the stock path.
+  `MTPLX_MOE_SORTED_GATHER_KERNEL=0` and the GPU-family rehearsal switch are
+  resolved by Qwen policy without mutating the process environment.
+- Whole-first-call canaries are keyed by compiled instantiation and the
+  under/from-64-rows-per-expert regime. They compare uint16 bits, including
+  signed zero, then reuse the checked result. Compile/dispatch failures or
+  mismatches disable that key for the process with a reason and counters.
+- A failed packed SwiGLU canary tries the separately checked plain row-map
+  gather before guarded stock fallback, as upstream does. Separate SwitchGLU
+  returns to its original up/gate/compiled-activation chain.
+- The only generalized helper is Qwen's GatherQMM: its existing BF16 behavior
+  is retained, with the F16/group-128 specializations needed for the oracle's
+  supported canary/fallback contract. All callers are in the Qwen module.
+
+## Isolated verification
+
+The native synthetic matrix covers 72 cases: BF16/F16 × b4/b8 × g32/64/128 ×
+packed/separate × 4,096/8,193/32,769 routed rows. It includes empty expert runs,
+partial tiles, both canary regimes, a strided uint32 row map and int64 expert
+IDs. Every result equals guarded stock and the pinned oracle bit for bit;
+48 first-use canaries pass. Actual installed layer-0 projections additionally
+cover 410/2,048/3,277/4,095/4,096/8,192 tokens (12 packed/separate cases).
+
+Failure injection uses actual invalid Metal source, a changed output sign bit,
+and explicitly differing positive/negative zero. Each disables its key and
+subsequent calls use exact stock output without another canary. Metadata and
+policy tests cover all exclusion predicates and switch spellings. The full
+model off-switch run checks the unchanged output/state and absence of kernel
+activation.
+
+The full-model large-prefix control reuses #110's 7,374-token cold prompt and
+3,278-token warm suffix at chunks 2,048 and 4,096. It checks every logit and
+cache array against the pre-port, pinned-oracle receipts, plus cancellation
+and retry. Explicit hidden-state tests use an MTP-capable runtime with cold
+4,096-token and warm 2,048-token-suffix prefills; an AR-only runtime correctly
+omits warm hidden arrays in the oracle. The AR timing tests compare complete
+state after one generated token. State reads occur after measured intervals.
+
+## Paired release measurements
+
+M5 Max, 128 GiB; serial before/after processes, three repetitions per chunk
+and phase, identical installed artifacts and token IDs. Both default boundary
+splitting (tail interval 256) and full chunks (tail interval 0) are measured.
+The latter exercises the wide row regime and guarded partial tails. The
+release baseline uses unchanged committed production source with only the
+same measurement test module added. No other GPU workload or build ran during
+these measurements.
+
+Cells show median ± sample standard deviation across all three repetitions.
+Repetition zero includes first-use initialization/canary work; this is not an
+isolated measurement of canary overhead. All individual times and means are
+in release-performance.json. TTFT starts at the request call and ends at its
+first token callback, after model loading.
+
+| Tail policy / chunk / phase | Before TTFT s | After TTFT s | Before prefill tok/s | After prefill tok/s |
+|---|---:|---:|---:|---:|
+| Default / 2048 / cold | 2.959 ± 0.473 | 2.953 ± 0.480 | 1385.831 ± 183.340 | 1388.955 ± 186.819 |
+| Default / 2048 / warm | 1.839 ± 0.075 | 1.900 ± 0.088 | 1115.426 ± 45.227 | 1079.363 ± 52.619 |
+| Default / 4096 / cold | 3.690 ± 0.063 | 3.684 ± 0.029 | 1111.739 ± 19.009 | 1113.593 ± 8.843 |
+| Default / 4096 / warm | 2.012 ± 0.013 | 1.989 ± 0.019 | 1019.529 ± 6.863 | 1031.159 ± 10.182 |
+| Full / 2048 / cold | 2.411 ± 0.475 | 2.504 ± 0.428 | 1699.510 ± 259.818 | 1636.600 ± 239.080 |
+| Full / 2048 / warm | 1.262 ± 0.063 | 1.334 ± 0.052 | 1623.785 ± 78.593 | 1536.347 ± 64.230 |
+| Full / 4096 / cold | 3.100 ± 0.116 | 2.999 ± 0.136 | 1321.520 ± 49.755 | 1366.360 ± 58.429 |
+| Full / 4096 / warm | 1.767 ± 0.057 | 1.634 ± 0.086 | 1159.717 ± 36.444 | 1254.103 ± 61.907 |
+
+Peak physical footprint: default 84.724 → 84.669 GB; full chunks
+88.579 → 87.952 GB. The results are mixed and do not establish an overall
+speedup. In particular, the default 256-token tail grid often keeps the small
+stock route; setting chunk size 4096 does not imply every body call has that
+width. The port fulfills the current-route obligation without assuming the
+upstream isolated-kernel speedup applies to this server benchmark.
+
+## Reproduction
+
+Set `DS4_QWEN38_ARTIFACTS` to the installed artifact directory. Build the release
+test executable with `cargo test --release --all-features --bin ds4-server
+issue112 --no-run`, then copy the executable reported by Cargo outside `target`
+before running it. Run each ignored test in its own serial process under
+`target/debug/test-supervisor MEMORY_MIB 60 120 --command TEST_EXECUTABLE FILTER
+--ignored --nocapture --test-threads=1`:
+
+- `issue112_all_kernel_variants` and `issue112_canary_failures`: 4096 MiB.
+- `issue112_installed_gate_up`: 16384 MiB.
+- `issue112_full_model_state`: 114688 MiB,
+  `DS4_ISSUE110_DIR=OUTPUT_DIRECTORY`, `MTPLX_GDN_BOUNDARY_TAIL_INTERVAL=0`.
+- `issue112_hidden_prefill`: 114688 MiB,
+  `DS4_ISSUE112_DIR=OUTPUT_DIRECTORY`, `MTPLX_GDN_BOUNDARY_TAIL_INTERVAL=0`.
+- `issue112_paired_prefill`: 114688 MiB,
+  `DS4_ISSUE112_DIR=OUTPUT_DIRECTORY`, optionally tail interval 0;
+  `DS4_ISSUE112_REPEATS` defaults to 3. Also run a one-repetition full-chunk
+  control with `MTPLX_MOE_SORTED_GATHER_KERNEL=0`.
+
+Keep child stdout/stderr separate from the supervisor's resource output to
+avoid interleaved JSON writes. Under the isolated MLX 0.32.2 reference Python,
+run `tools/issue112-reference.py PINNED_CHECKOUT NATIVE_VARIANT_OUTPUT`,
+`tools/issue112-installed-reference.py PINNED_CHECKOUT MODEL_DIR NATIVE_OUTPUT`,
+and `tools/issue112-model-reference.py PINNED_CHECKOUT MODEL_DIR NATIVE_DIR
+ORACLE_DIR` (add `--hidden` for the hidden-state run; use matching tail policy).
+The full-model oracle uses the server's lazy-snapshot policy for cache
+fingerprints, matching the native session entry; a deep clone adds zero and
+would change negative-zero key bits during measurement itself.
+
+For the before arm, build commit `8965e77` with only `issue112_bench.rs` and its
+`#[cfg(test)] mod issue112_bench;` registration added. Do not use an off-switch
+run as the only baseline: the preserved pre-port executable is the baseline.
+Both arms use the same test harness. Run timing arms without other builds or
+GPU work; correctness checks may run alongside CPU-only compilation.
+
+## Completion review
+
+All 72 synthetic and 12 installed projection receipts match guarded stock
+and pinned MTPLX; compile/sign-bit/signed-zero failures pass sticky-fallback
+checks. All four large-prefix snapshots match #110's pre-port/oracle fixtures.
+The 24 release before/after state comparisons and four explicit hidden-state
+comparisons are exact. The direct pinned oracle matches all four representative
+release states and all four hidden-prefill states, including every cache bit.
+The off-switch matches all four baseline states with zero kernel activation.
+
+DeepSeek's `resident_multi_session_switching_preserves_each_kv_frontier` and
+GLM's `glm53_two_row_verifier_restores_recurrent_state` both pass on the preserved
+before binary and the final after binary with installed models. Source review
+confirms no DS4/GLM model, sampling, cache, shader, or orchestration path changed.
+Their DS4 oracle behavior remains assigned to those families; this is the Qwen
+MTPLX route port, not a new DS4Server-only feature. Non-NAX predicates are tested;
+no physical M1–M4 run is claimed.
+
+All required commit gates pass: formatting, warning-free Clippy, signed app
+bundle, and all-features tests (312 application + 8 tool tests; 234 explicitly
+ignored hardware/reference tests). Relevant ignored tests above were run
+separately under the resource supervisor. Machine-readable evidence, exact
+state fingerprints, gate logs, resource summaries and binary/source provenance
+are stored beside this file. No acceptance item is deferred to a stub.

docs/issue112-row-map/before-deepseek.log

@@ -0,0 +1,10 @@
+
+running 1 test
+test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
+ds4: Metal 4 tensor API enabled for Tensor kernels
+ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
+ds4: Metal model views created in 2.169 ms, residency requested in 7304.599 ms, warmup 5.656 ms (mapped 82697.67 MiB from offset 5.09 MiB)
+ds4: Metal mapped mmaped model as 2 overlapping shared buffers
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 539 filtered out; finished in 7.89s

docs/issue112-row-map/before-glm.log

@@ -0,0 +1,11 @@
+ds4: GLM MTP draft 5733 at position 12 in 2.3 ms
+verify2 scalar boundary cap=2 cancelled=true passed
+verify2 prefill 0
+verify2 prefill 4
+verify2 prefill 8
+verify2 prefill 12
+ds4: GLM MTP draft 5733 at position 12 in 2.4 ms
+engine-lifetime counters across new and restored chats passed
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 539 filtered out; finished in 9.18s

docs/issue112-row-map/bundle.log

@@ -0,0 +1,8 @@
+cargo build --release --all-targets --all-features
+   Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
+    Finished `release` profile [optimized] target(s) in 29.05s
+warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
+note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
+cargo packager --release --formats app
+codesign --force --deep --sign - --timestamp=none target/release/DS4Server.app
+target/release/DS4Server.app: replacing existing signature

docs/issue112-row-map/clippy.log

@@ -0,0 +1,4 @@
+    Checking ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
+    Finished `dev` profile [unoptimized + debuginfo] target(s) in 5.14s
+warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
+note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`

docs/issue112-row-map/deepseek.log

@@ -0,0 +1,10 @@
+
+running 1 test
+test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
+ds4: Metal 4 tensor API enabled for Tensor kernels
+ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
+ds4: Metal model views created in 1.781 ms, residency requested in 8958.289 ms, warmup 2.850 ms (mapped 82697.67 MiB from offset 5.09 MiB)
+ds4: Metal mapped mmaped model as 2 overlapping shared buffers
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 545 filtered out; finished in 9.53s

docs/issue112-row-map/full-2048-0-cold.json

@@ -0,0 +1 @@
+{"metrics":{"chunk":2048,"rep":0,"phase":"cold","prompt":4096,"cached":0,"ttft_seconds":3.126132542,"prefill_seconds":3.1225067920000003,"prefill_tokens_per_second":1311.766562203846,"elapsed_seconds":3.147092209},"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,Line truncated

docs/issue112-row-map/full-2048-0-warm.json

@@ -0,0 +1 @@
+{"metrics":{"chunk":2048,"rep":0,"phase":"warm","prompt":6145,"cached":4097,"ttft_seconds":1.262224125,"prefill_seconds":1.258582084,"prefill_tokens_per_second":1627.2279941337542,"elapsed_seconds":1.282883709},"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374Line truncated

docs/issue112-row-map/full-4096-0-cold.json

@@ -0,0 +1 @@
+{"metrics":{"chunk":4096,"rep":0,"phase":"cold","prompt":4096,"cached":0,"ttft_seconds":2.955218083,"prefill_seconds":2.9541866239999997,"prefill_tokens_per_second":1386.50685326507,"elapsed_seconds":2.9757388750000002},"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11Line truncated

docs/issue112-row-map/full-4096-0-warm.json

@@ -0,0 +1 @@
+{"metrics":{"chunk":4096,"rep":0,"phase":"warm","prompt":6145,"cached":4097,"ttft_seconds":1.606924,"prefill_seconds":1.6059842079999997,"prefill_tokens_per_second":1275.2304722538095,"elapsed_seconds":1.62792175},"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,Line truncated

docs/issue112-row-map/glm.log

@@ -0,0 +1,11 @@
+ds4: GLM MTP draft 5733 at position 12 in 2.3 ms
+verify2 scalar boundary cap=2 cancelled=true passed
+verify2 prefill 0
+verify2 prefill 4
+verify2 prefill 8
+verify2 prefill 12
+ds4: GLM MTP draft 5733 at position 12 in 2.4 ms
+engine-lifetime counters across new and restored chats passed
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 545 filtered out; finished in 11.86s

docs/issue112-row-map/hidden-2048-cold.json

@@ -0,0 +1 @@
+{"case":"hidden_prefill","chunk":2048,"phase":"cold","cached":0,"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,2Line truncated

docs/issue112-row-map/hidden-2048-warm.json

@@ -0,0 +1 @@
+{"case":"hidden_prefill","chunk":2048,"phase":"warm","cached":4096,"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,37Line truncated

docs/issue112-row-map/hidden-4096-cold.json

@@ -0,0 +1 @@
+{"case":"hidden_prefill","chunk":4096,"phase":"cold","cached":0,"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,2Line truncated

docs/issue112-row-map/hidden-4096-warm.json

@@ -0,0 +1 @@
+{"case":"hidden_prefill","chunk":4096,"phase":"warm","cached":4096,"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,37Line truncated

docs/issue112-row-map/provenance.json

@@ -0,0 +1,19 @@
+{
+  "baseline": "8965e77ea173c932e00f70759d0a83ac1c5072f0",
+  "mtplx": "9882703f3105363ddc37eca9f97aa09a1d387112",
+  "mlx": "0.32.2",
+  "artifact": "74559cdf34fbfc0b593de72d17e93f37fd4f9ea7",
+  "sha256": {
+    "metal/mtplx-moe-sorted-body.metal": "b639a135d2b9532d45c9587d94a503a093553078d8f5b5a9470c34d1bd9fca2e",
+    "metal/mtplx-moe-sorted-header.metal": "d3251a6cfcdbac5372c3e7e1f97720df48a91f0e456a44883c42b98cc8520e4a",
+    "metal/mtplx-runtime-0.32.2.metallib": "dc59d1cceb1a5c7e578232e6e41e28e2c73c9463ac6dbc3886c3ee17ffc270ed",
+    "/private/tmp/ds4-issue112-before-release-test": "9718805d3827bf5e998e2f574b48b7e453a45ed3de578ea9df6a450019aea0d6",
+    "/private/tmp/ds4-issue112-after-release-test": "9bcc33ddebe79fa026d7cb4a7c2a4dfcdba2645f66861a1e74778cf55e434912",
+    "/private/tmp/ds4-issue112-final-test": "f313f1ae0a43a05d359ce234977f00dba87ef8d1320034acf856592f2e2eba72"
+  },
+  "binary_scope": {
+    "before-release-test": "Full-chunk timing, hidden-state and family controls. The earlier default-tail run used the same baseline production source before the hidden-only test configuration was corrected.",
+    "after-release-test": "Default-tail and full-chunk timings; production identical to final-test.",
+    "final-test": "Final kernel, model, hidden-state, off-switch and family checks; corrected hidden-only test configuration."
+  }
+}

docs/issue112-row-map/release-performance.json

@@ -0,0 +1,838 @@
+[
+  {
+    "configuration": "default-tail",
+    "arm": "before",
+    "records": [
+      {
+        "chunk": 2048,
+        "rep": 0,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.7007944999999998,
+        "prefill_seconds": 3.6969075010000005,
+        "prefill_tokens_per_second": 1107.9530658779117,
+        "elapsed_seconds": 3.721188
+      },
+      {
+        "chunk": 2048,
+        "rep": 0,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.7724733750000001,
+        "prefill_seconds": 1.769735542,
+        "prefill_tokens_per_second": 1157.23505088536,
+        "elapsed_seconds": 1.793335125
+      },
+      {
+        "chunk": 2048,
+        "rep": 1,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.820473209,
+        "prefill_seconds": 2.81689246,
+        "prefill_tokens_per_second": 1454.0846191906098,
+        "elapsed_seconds": 2.841268834
+      },
+      {
+        "chunk": 2048,
+        "rep": 1,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.838782375,
+        "prefill_seconds": 1.836070168,
+        "prefill_tokens_per_second": 1115.4257803942492,
+        "elapsed_seconds": 1.8597940419999999
+      },
+      {
+        "chunk": 2048,
+        "rep": 2,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.959423792,
+        "prefill_seconds": 2.9556270000000002,
+        "prefill_tokens_per_second": 1385.831162051233,
+        "elapsed_seconds": 2.98031925
+      },
+      {
+        "chunk": 2048,
+        "rep": 2,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.922527375,
+        "prefill_seconds": 1.919643709,
+        "prefill_tokens_per_second": 1066.8646428492007,
+        "elapsed_seconds": 1.9441567499999999
+      },
+      {
+        "chunk": 4096,
+        "rep": 0,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.626180458,
+        "prefill_seconds": 3.6201352510000007,
+        "prefill_tokens_per_second": 1131.4494393181994,
+        "elapsed_seconds": 3.64758525
+      },
+      {
+        "chunk": 4096,
+        "rep": 0,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.994229042,
+        "prefill_seconds": 1.9914031639999998,
+        "prefill_tokens_per_second": 1028.4205815392588,
+        "elapsed_seconds": 2.016047875
+      },
+      {
+        "chunk": 4096,
+        "rep": 1,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.6903119589999998,
+        "prefill_seconds": 3.6843170420000004,
+        "prefill_tokens_per_second": 1111.7392866322168,
+        "elapsed_seconds": 3.711886
+      },
+      {
+        "chunk": 4096,
+        "rep": 1,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 2.011648875,
+        "prefill_seconds": 2.0087712079999998,
+        "prefill_tokens_per_second": 1019.5287506331085,
+        "elapsed_seconds": 2.033881333
+      },
+      {
+        "chunk": 4096,
+        "rep": 2,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.752376958,
+        "prefill_seconds": 3.7459722089999996,
+        "prefill_tokens_per_second": 1093.4411072669013,
+        "elapsed_seconds": 3.774499291
+      },
+      {
+        "chunk": 4096,
+        "rep": 2,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 2.020803542,
+        "prefill_seconds": 2.017895623,
+        "prefill_tokens_per_second": 1014.918698795354,
+        "elapsed_seconds": 2.043202292
+      }
+    ],
+    "summary": [
+      {
+        "chunk": 2048,
+        "phase": "cold",
+        "ttft_seconds": {
+          "mean": 3.1602305003333333,
+          "median": 2.959423792,
+          "sample_stddev": 0.47326936772912714
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1315.9562823732515,
+          "median": 1385.831162051233,
+          "sample_stddev": 183.340222420435
+        }
+      },
+      {
+        "chunk": 2048,
+        "phase": "warm",
+        "ttft_seconds": {
+          "mean": 1.844594375,
+          "median": 1.838782375,
+          "sample_stddev": 0.07519564639658331
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1113.1751580429366,
+          "median": 1115.4257803942492,
+          "sample_stddev": 45.227222310038556
+        }
+      },
+      {
+        "chunk": 4096,
+        "phase": "cold",
+        "ttft_seconds": {
+          "mean": 3.689623125,
+          "median": 3.6903119589999998,
+          "sample_stddev": 0.06310106989799923
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1112.2099444057726,
+          "median": 1111.7392866322168,
+          "sample_stddev": 19.00853664502584
+        }
+      },
+      {
+        "chunk": 4096,
+        "phase": "warm",
+        "ttft_seconds": {
+          "mean": 2.0088938196666666,
+          "median": 2.011648875,
+          "sample_stddev": 0.013499768886162316
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1020.9560103225738,
+          "median": 1019.5287506331085,
+          "sample_stddev": 6.863163415905101
+        }
+      }
+    ],
+    "resources": {
+      "event": "test_resource_summary",
+      "elapsed_ms": 41097,
+      "peak": {
+        "cpu_time_ns": 19599062416,
+        "resident_bytes": 25489604608,
+        "physical_bytes": 84724073856,
+        "peak_physical_bytes": 84724139512,
+        "disk_read_bytes": 64097386496,
+        "disk_write_bytes": 442368
+      },
+      "error": null
+    }
+  },
+  {
+    "configuration": "default-tail",
+    "arm": "after",
+    "records": [
+      {
+        "chunk": 2048,
+        "rep": 0,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.699041667,
+        "prefill_seconds": 3.6953074989999997,
+        "prefill_tokens_per_second": 1108.4327897227588,
+        "elapsed_seconds": 3.719328292
+      },
+      {
+        "chunk": 2048,
+        "rep": 0,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.776398,
+        "prefill_seconds": 1.7737045,
+        "prefill_tokens_per_second": 1154.6455455235075,
+        "elapsed_seconds": 1.797202167
+      },
+      {
+        "chunk": 2048,
+        "rep": 1,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.80434375,
+        "prefill_seconds": 2.8008109150000005,
+        "prefill_tokens_per_second": 1462.4336038050606,
+        "elapsed_seconds": 2.825135334
+      },
+      {
+        "chunk": 2048,
+        "rep": 1,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.900156542,
+        "prefill_seconds": 1.897415875,
+        "prefill_tokens_per_second": 1079.3627411808177,
+        "elapsed_seconds": 1.9216478750000001
+      },
+      {
+        "chunk": 2048,
+        "rep": 2,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.952652333,
+        "prefill_seconds": 2.948979292,
+        "prefill_tokens_per_second": 1388.9551585226932,
+        "elapsed_seconds": 2.973752041
+      },
+      {
+        "chunk": 2048,
+        "rep": 2,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.947212708,
+        "prefill_seconds": 1.944330169,
+        "prefill_tokens_per_second": 1053.3190466582735,
+        "elapsed_seconds": 1.9688718330000001
+      },
+      {
+        "chunk": 4096,
+        "rep": 0,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.637618958,
+        "prefill_seconds": 3.631528458,
+        "prefill_tokens_per_second": 1127.8997390139687,
+        "elapsed_seconds": 3.659774833
+      },
+      {
+        "chunk": 4096,
+        "rep": 0,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.9611875410000001,
+        "prefill_seconds": 1.958409373,
+        "prefill_tokens_per_second": 1045.7466289914453,
+        "elapsed_seconds": 1.982932208
+      },
+      {
+        "chunk": 4096,
+        "rep": 1,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.6904271250000003,
+        "prefill_seconds": 3.684310291,
+        "prefill_tokens_per_second": 1111.7413237440048,
+        "elapsed_seconds": 3.71194925
+      },
+      {
+        "chunk": 4096,
+        "rep": 1,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.988992875,
+        "prefill_seconds": 1.98611425,
+        "prefill_tokens_per_second": 1031.159209496634,
+        "elapsed_seconds": 2.010877
+      },
+      {
+        "chunk": 4096,
+        "rep": 2,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.684088083,
+        "prefill_seconds": 3.6781829150000003,
+        "prefill_tokens_per_second": 1113.5933406944227,
+        "elapsed_seconds": 3.705752166
+      },
+      {
+        "chunk": 4096,
+        "rep": 2,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.998740333,
+        "prefill_seconds": 1.9958163330000003,
+        "prefill_tokens_per_second": 1026.1465276825147,
+        "elapsed_seconds": 2.021001667
+      }
+    ],
+    "summary": [
+      {
+        "chunk": 2048,
+        "phase": "cold",
+        "ttft_seconds": {
+          "mean": 3.1520125833333332,
+          "median": 2.952652333,
+          "sample_stddev": 0.4795096169325091
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1319.940517350171,
+          "median": 1388.9551585226932,
+          "sample_stddev": 186.81918963228082
+        }
+      },
+      {
+        "chunk": 2048,
+        "phase": "warm",
+        "ttft_seconds": {
+          "mean": 1.8745890833333332,
+          "median": 1.900156542,
+          "sample_stddev": 0.0882308751191163
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1095.775777787533,
+          "median": 1079.3627411808177,
+          "sample_stddev": 52.619441949355554
+        }
+      },
+      {
+        "chunk": 4096,
+        "phase": "cold",
+        "ttft_seconds": {
+          "mean": 3.6707113886666667,
+          "median": 3.684088083,
+          "sample_stddev": 0.02883361907462447
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1117.7448011507988,
+          "median": 1113.5933406944227,
+          "sample_stddev": 8.843051732845977
+        }
+      },
+      {
+        "chunk": 4096,
+        "phase": "warm",
+        "ttft_seconds": {
+          "mean": 1.9829735830000002,
+          "median": 1.988992875,
+          "sample_stddev": 0.019486583946006573
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1034.3507887235314,
+          "median": 1031.159209496634,
+          "sample_stddev": 10.18236840333609
+        }
+      }
+    ],
+    "resources": {
+      "event": "test_resource_summary",
+      "elapsed_ms": 40871,
+      "peak": {
+        "cpu_time_ns": 19458778041,
+        "resident_bytes": 25328418816,
+        "physical_bytes": 84668646808,
+        "peak_physical_bytes": 84668761376,
+        "disk_read_bytes": 69973549056,
+        "disk_write_bytes": 18446744073709289472
+      },
+      "error": null
+    }
+  },
+  {
+    "configuration": "full-chunks",
+    "arm": "before",
+    "records": [
+      {
+        "chunk": 2048,
+        "rep": 0,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.182152,
+        "prefill_seconds": 3.17999279,
+        "prefill_tokens_per_second": 1288.0532348628376,
+        "elapsed_seconds": 3.202144875
+      },
+      {
+        "chunk": 2048,
+        "rep": 0,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.219195625,
+        "prefill_seconds": 1.21829175,
+        "prefill_tokens_per_second": 1681.042328325707,
+        "elapsed_seconds": 1.240087208
+      },
+      {
+        "chunk": 2048,
+        "rep": 1,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.317122125,
+        "prefill_seconds": 2.3159108330000002,
+        "prefill_tokens_per_second": 1768.6345871503593,
+        "elapsed_seconds": 2.338011208
+      },
+      {
+        "chunk": 2048,
+        "rep": 1,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.26216725,
+        "prefill_seconds": 1.261251084,
+        "prefill_tokens_per_second": 1623.7845310743853,
+        "elapsed_seconds": 1.283571459
+      },
+      {
+        "chunk": 2048,
+        "rep": 2,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.411353459,
+        "prefill_seconds": 2.410105791,
+        "prefill_tokens_per_second": 1699.510459373026,
+        "elapsed_seconds": 2.431755125
+      },
+      {
+        "chunk": 2048,
+        "rep": 2,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.34331275,
+        "prefill_seconds": 1.34238875,
+        "prefill_tokens_per_second": 1525.6385305672443,
+        "elapsed_seconds": 1.364779
+      },
+      {
+        "chunk": 4096,
+        "rep": 0,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.2133175,
+        "prefill_seconds": 3.212275166,
+        "prefill_tokens_per_second": 1275.1086965879188,
+        "elapsed_seconds": 3.233724875
+      },
+      {
+        "chunk": 4096,
+        "rep": 0,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.7273878329999999,
+        "prefill_seconds": 1.7264041659999998,
+        "prefill_tokens_per_second": 1186.2807332915115,
+        "elapsed_seconds": 1.7481919160000001
+      },
+      {
+        "chunk": 4096,
+        "rep": 1,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.980801042,
+        "prefill_seconds": 2.979894292,
+        "prefill_tokens_per_second": 1374.5454028340412,
+        "elapsed_seconds": 3.001205167
+      },
+      {
+        "chunk": 4096,
+        "rep": 1,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.7668409999999999,
+        "prefill_seconds": 1.765948334,
+        "prefill_tokens_per_second": 1159.7168278197203,
+        "elapsed_seconds": 1.787861709
+      },
+      {
+        "chunk": 4096,
+        "rep": 2,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.100441792,
+        "prefill_seconds": 3.0994622499999998,
+        "prefill_tokens_per_second": 1321.5195635952657,
+        "elapsed_seconds": 3.121023875
+      },
+      {
+        "chunk": 4096,
+        "rep": 2,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.838995291,
+        "prefill_seconds": 1.838062583,
+        "prefill_tokens_per_second": 1114.216686059378,
+        "elapsed_seconds": 1.860065208
+      }
+    ],
+    "summary": [
+      {
+        "chunk": 2048,
+        "phase": "cold",
+        "ttft_seconds": {
+          "mean": 2.6368758613333334,
+          "median": 2.411353459,
+          "sample_stddev": 0.4745676312458291
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1585.3994271287409,
+          "median": 1699.510459373026,
+          "sample_stddev": 259.81840735698063
+        }
+      },
+      {
+        "chunk": 2048,
+        "phase": "warm",
+        "ttft_seconds": {
+          "mean": 1.274891875,
+          "median": 1.26216725,
+          "sample_stddev": 0.06302937601316841
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1610.1551299891123,
+          "median": 1623.7845310743853,
+          "sample_stddev": 78.59329182503896
+        }
+      },
+      {
+        "chunk": 4096,
+        "phase": "cold",
+        "ttft_seconds": {
+          "mean": 3.098186778,
+          "median": 3.100441792,
+          "sample_stddev": 0.11627463019215153
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1323.7245543390752,
+          "median": 1321.5195635952657,
+          "sample_stddev": 49.755011058230345
+        }
+      },
+      {
+        "chunk": 4096,
+        "phase": "warm",
+        "ttft_seconds": {
+          "mean": 1.7777413746666666,
+          "median": 1.7668409999999999,
+          "sample_stddev": 0.05659655286508827
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1153.4047490568698,
+          "median": 1159.7168278197203,
+          "sample_stddev": 36.444320265304924
+        }
+      }
+    ],
+    "resources": {
+      "event": "test_resource_summary",
+      "elapsed_ms": 35856,
+      "peak": {
+        "cpu_time_ns": 21540742125,
+        "resident_bytes": 25113673728,
+        "physical_bytes": 88578784936,
+        "peak_physical_bytes": 88578866928,
+        "disk_read_bytes": 69525413888,
+        "disk_write_bytes": 524288
+      },
+      "error": null
+    }
+  },
+  {
+    "configuration": "full-chunks",
+    "arm": "after",
+    "records": [
+      {
+        "chunk": 2048,
+        "rep": 0,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.126132542,
+        "prefill_seconds": 3.1225067920000003,
+        "prefill_tokens_per_second": 1311.766562203846,
+        "elapsed_seconds": 3.147092209
+      },
+      {
+        "chunk": 2048,
+        "rep": 0,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.262224125,
+        "prefill_seconds": 1.258582084,
+        "prefill_tokens_per_second": 1627.2279941337542,
+        "elapsed_seconds": 1.282883709
+      },
+      {
+        "chunk": 2048,
+        "rep": 1,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.304937333,
+        "prefill_seconds": 2.303636208,
+        "prefill_tokens_per_second": 1778.058525810426,
+        "elapsed_seconds": 2.325319833
+      },
+      {
+        "chunk": 2048,
+        "rep": 1,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.333947917,
+        "prefill_seconds": 1.3330325,
+        "prefill_tokens_per_second": 1536.346638210246,
+        "elapsed_seconds": 1.354474625
+      },
+      {
+        "chunk": 2048,
+        "rep": 2,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.504060958,
+        "prefill_seconds": 2.502749457,
+        "prefill_tokens_per_second": 1636.6000953646394,
+        "elapsed_seconds": 2.524487958
+      },
+      {
+        "chunk": 2048,
+        "rep": 2,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.363442375,
+        "prefill_seconds": 1.362460209,
+        "prefill_tokens_per_second": 1503.1631650388993,
+        "elapsed_seconds": 1.3840412500000001
+      },
+      {
+        "chunk": 4096,
+        "rep": 0,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.955218083,
+        "prefill_seconds": 2.9541866239999997,
+        "prefill_tokens_per_second": 1386.50685326507,
+        "elapsed_seconds": 2.9757388750000002
+      },
+      {
+        "chunk": 4096,
+        "rep": 0,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.606924,
+        "prefill_seconds": 1.6059842079999997,
+        "prefill_tokens_per_second": 1275.2304722538095,
+        "elapsed_seconds": 1.62792175
+      },
+      {
+        "chunk": 4096,
+        "rep": 1,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 3.209116167,
+        "prefill_seconds": 3.2081535000000003,
+        "prefill_tokens_per_second": 1276.7468888256126,
+        "elapsed_seconds": 3.230131833
+      },
+      {
+        "chunk": 4096,
+        "rep": 1,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.767990417,
+        "prefill_seconds": 1.767019791,
+        "prefill_tokens_per_second": 1159.0136174088839,
+        "elapsed_seconds": 1.789304875
+      },
+      {
+        "chunk": 4096,
+        "rep": 2,
+        "phase": "cold",
+        "prompt": 4096,
+        "cached": 0,
+        "ttft_seconds": 2.998665042,
+        "prefill_seconds": 2.997745792,
+        "prefill_tokens_per_second": 1366.3600198959098,
+        "elapsed_seconds": 3.019893375
+      },
+      {
+        "chunk": 4096,
+        "rep": 2,
+        "phase": "warm",
+        "prompt": 6145,
+        "cached": 4097,
+        "ttft_seconds": 1.633976833,
+        "prefill_seconds": 1.633039209,
+        "prefill_tokens_per_second": 1254.1033850951462,
+        "elapsed_seconds": 1.656772416
+      }
+    ],
+    "summary": [
+      {
+        "chunk": 2048,
+        "phase": "cold",
+        "ttft_seconds": {
+          "mean": 2.645043611,
+          "median": 2.504060958,
+          "sample_stddev": 0.4283660515917751
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1575.475061126304,
+          "median": 1636.6000953646394,
+          "sample_stddev": 239.08001001547228
+        }
+      },
+      {
+        "chunk": 2048,
+        "phase": "warm",
+        "ttft_seconds": {
+          "mean": 1.3198714723333334,
+          "median": 1.333947917,
+          "sample_stddev": 0.052056635063226324
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1555.5792657942998,
+          "median": 1536.346638210246,
+          "sample_stddev": 64.22959541694213
+        }
+      },
+      {
+        "chunk": 4096,
+        "phase": "cold",
+        "ttft_seconds": {
+          "mean": 3.054333097333333,
+          "median": 2.998665042,
+          "sample_stddev": 0.1357949135761868
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1343.2045873288641,
+          "median": 1366.3600198959098,
+          "sample_stddev": 58.42895678910406
+        }
+      },
+      {
+        "chunk": 4096,
+        "phase": "warm",
+        "ttft_seconds": {
+          "mean": 1.6696304166666667,
+          "median": 1.633976833,
+          "sample_stddev": 0.0862495286509525
+        },
+        "prefill_tokens_per_second": {
+          "mean": 1229.4491582526132,
+          "median": 1254.1033850951462,
+          "sample_stddev": 61.90688583198103
+        }
+      }
+    ],
+    "resources": {
+      "event": "test_resource_summary",
+      "elapsed_ms": 34614,
+      "peak": {
+        "cpu_time_ns": 20995139625,
+        "resident_bytes": 25576030208,
+        "physical_bytes": 87952375368,
+        "peak_physical_bytes": 87952457288,
+        "disk_read_bytes": 64869085184,
+        "disk_write_bytes": 442368
+      },
+      "error": null
+    }
+  }
+]

docs/issue112-row-map/resource-summaries.json

@@ -0,0 +1,184 @@
+{
+  "final-variants": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 440,
+    "peak": {
+      "cpu_time_ns": 245994500,
+      "resident_bytes": 281100288,
+      "physical_bytes": 399917944,
+      "peak_physical_bytes": 419464080,
+      "disk_read_bytes": 0,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "final-failures": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 219,
+    "peak": {
+      "cpu_time_ns": 53213250,
+      "resident_bytes": 70057984,
+      "physical_bytes": 177390336,
+      "peak_physical_bytes": 177636096,
+      "disk_read_bytes": 0,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "final-installed": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 760,
+    "peak": {
+      "cpu_time_ns": 504222750,
+      "resident_bytes": 2076639232,
+      "physical_bytes": 4867475808,
+      "peak_physical_bytes": 5012703704,
+      "disk_read_bytes": 0,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "final-variants-oracle": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 762,
+    "peak": {
+      "cpu_time_ns": 562100250,
+      "resident_bytes": 142884864,
+      "physical_bytes": 359269216,
+      "peak_physical_bytes": 387105656,
+      "disk_read_bytes": 0,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "final-installed-oracle": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 1200,
+    "peak": {
+      "cpu_time_ns": 959967166,
+      "resident_bytes": 2091220992,
+      "physical_bytes": 3261778056,
+      "peak_physical_bytes": 4137257216,
+      "disk_read_bytes": 421888,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "after-hidden": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 19796,
+    "peak": {
+      "cpu_time_ns": 16081049958,
+      "resident_bytes": 25542393856,
+      "physical_bytes": 87610048312,
+      "peak_physical_bytes": 87611588312,
+      "disk_read_bytes": 65141735424,
+      "disk_write_bytes": 147456
+    },
+    "error": null
+  },
+  "final-model": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 29057,
+    "peak": {
+      "cpu_time_ns": 23619215916,
+      "resident_bytes": 25820266496,
+      "physical_bytes": 101435424688,
+      "peak_physical_bytes": 101440126416,
+      "disk_read_bytes": 71619133440,
+      "disk_write_bytes": 208896
+    },
+    "error": null
+  },
+  "off": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 18110,
+    "peak": {
+      "cpu_time_ns": 14743490958,
+      "resident_bytes": 25949437952,
+      "physical_bytes": 88457051624,
+      "peak_physical_bytes": 88460278864,
+      "disk_read_bytes": 66400653312,
+      "disk_write_bytes": 147456
+    },
+    "error": null
+  },
+  "final-model-oracle": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 19987,
+    "peak": {
+      "cpu_time_ns": 17635902708,
+      "resident_bytes": 25024905216,
+      "physical_bytes": 84086635888,
+      "peak_physical_bytes": 84086652296,
+      "disk_read_bytes": 103826788352,
+      "disk_write_bytes": 311296
+    },
+    "error": null
+  },
+  "hidden-oracle": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 19800,
+    "peak": {
+      "cpu_time_ns": 17457184041,
+      "resident_bytes": 25029591040,
+      "physical_bytes": 84092452352,
+      "peak_physical_bytes": 84092468760,
+      "disk_read_bytes": 103815700480,
+      "disk_write_bytes": 18446744073709027328
+    },
+    "error": null
+  },
+  "deepseek": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 9561,
+    "peak": {
+      "cpu_time_ns": 2757894458,
+      "resident_bytes": 130564096,
+      "physical_bytes": 307839816,
+      "peak_physical_bytes": 319898440,
+      "disk_read_bytes": 86739410944,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "glm": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 11928,
+    "peak": {
+      "cpu_time_ns": 3587549375,
+      "resident_bytes": 905478144,
+      "physical_bytes": 1335444992,
+      "peak_physical_bytes": 1488717432,
+      "disk_read_bytes": 96510627840,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "before-deepseek": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 7984,
+    "peak": {
+      "cpu_time_ns": 2510786166,
+      "resident_bytes": 130416640,
+      "physical_bytes": 301171504,
+      "peak_physical_bytes": 319750984,
+      "disk_read_bytes": 69381001216,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "before-glm": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 9244,
+    "peak": {
+      "cpu_time_ns": 2799064958,
+      "resident_bytes": 905216000,
+      "physical_bytes": 1335182800,
+      "peak_physical_bytes": 1488471648,
+      "disk_read_bytes": 68293218304,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  }
+}

docs/issue112-row-map/state-comparisons.json

@@ -0,0 +1,218 @@
+[
+  {
+    "configuration": "default",
+    "case": "2048-0-cold",
+    "before": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
+    "after": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "2048-0-warm",
+    "before": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
+    "after": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "2048-1-cold",
+    "before": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
+    "after": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "2048-1-warm",
+    "before": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
+    "after": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "2048-2-cold",
+    "before": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
+    "after": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "2048-2-warm",
+    "before": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
+    "after": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "4096-0-cold",
+    "before": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
+    "after": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "4096-0-warm",
+    "before": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
+    "after": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "4096-1-cold",
+    "before": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
+    "after": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "4096-1-warm",
+    "before": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
+    "after": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "4096-2-cold",
+    "before": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
+    "after": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
+    "exact": true
+  },
+  {
+    "configuration": "default",
+    "case": "4096-2-warm",
+    "before": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
+    "after": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "2048-0-cold",
+    "before": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
+    "after": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "2048-0-warm",
+    "before": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
+    "after": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "2048-1-cold",
+    "before": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
+    "after": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "2048-1-warm",
+    "before": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
+    "after": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "2048-2-cold",
+    "before": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
+    "after": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "2048-2-warm",
+    "before": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
+    "after": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "4096-0-cold",
+    "before": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
+    "after": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "4096-0-warm",
+    "before": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
+    "after": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "4096-1-cold",
+    "before": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
+    "after": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "4096-1-warm",
+    "before": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
+    "after": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "4096-2-cold",
+    "before": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
+    "after": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
+    "exact": true
+  },
+  {
+    "configuration": "full",
+    "case": "4096-2-warm",
+    "before": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
+    "after": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
+    "exact": true
+  },
+  {
+    "configuration": "hidden",
+    "case": "2048-cold",
+    "before": "9616a339d8ec57e48fd4342d244b0fbeb86148722f1e6733f5ff4ea5c4862bf5",
+    "after": "9616a339d8ec57e48fd4342d244b0fbeb86148722f1e6733f5ff4ea5c4862bf5",
+    "exact": true
+  },
+  {
+    "configuration": "hidden",
+    "case": "2048-warm",
+    "before": "fe0116cf8aebc195d3fb01970cb8be133d17750bf0284db1825747e6e21f0e0b",
+    "after": "fe0116cf8aebc195d3fb01970cb8be133d17750bf0284db1825747e6e21f0e0b",
+    "exact": true
+  },
+  {
+    "configuration": "hidden",
+    "case": "4096-cold",
+    "before": "40d9efadffeeec8cc6e42d31241276a1fb727bac9fba3e0621343057e7588403",
+    "after": "40d9efadffeeec8cc6e42d31241276a1fb727bac9fba3e0621343057e7588403",
+    "exact": true
+  },
+  {
+    "configuration": "hidden",
+    "case": "4096-warm",
+    "before": "2a40fec32283b8a2434181655260daa14fe9b3731b37e6a82c842a75df14d586",
+    "after": "2a40fec32283b8a2434181655260daa14fe9b3731b37e6a82c842a75df14d586",
+    "exact": true
+  },
+  {
+    "configuration": "off",
+    "case": "2048-0-cold",
+    "exact": true
+  },
+  {
+    "configuration": "off",
+    "case": "4096-0-warm",
+    "exact": true
+  },
+  {
+    "configuration": "off",
+    "case": "4096-0-cold",
+    "exact": true
+  },
+  {
+    "configuration": "off",
+    "case": "2048-0-warm",
+    "exact": true
+  }
+]

docs/issue112-row-map/tests.log

@@ -0,0 +1,589 @@
+   Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
+    Finished `test` profile [unoptimized + debuginfo] target(s) in 6.12s
+warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
+note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
+     Running unittests src/main.rs (target/debug/deps/ds4_server-5679c54d8848b708)
+
+running 546 tests
+test agent::tests::anchored_edits_require_unique_head_and_tail ... ok
+test agent::tests::ai_risk_response_requires_structured_risk_and_reason ... ok
+test a2ui::tests::streaming_parser_only_accepts_complete_a2ui_jsonl_lines ... ok
+test agent::tests::ai_shell_risk_assessment_receives_the_dev_brain_root ... ok
+test a2ui::tests::embedded_catalog_and_validator_cover_the_same_components ... ok
+test agent::tests::dev_brain_paths_are_trusted_by_shell_risk_assessment ... ok
+test a2ui::tests::basic_formatters_follow_catalog_examples ... ok
+test a2ui::tests::catalog_validation_rejects_invalid_variants_and_function_arguments ... ok
+test a2ui::tests::catalog_accepts_pie_donut_and_heatmap_charts ... ok
+test a2ui::tests::dismissal_boundaries_replay_history_and_a_fresh_active_epoch ... ok
+test a2ui::tests::lifecycle_updates_incrementally_and_json_pointer_upserts ... ok
+test a2ui::tests::latest_created_surface_is_the_active_surface ... ok
+test agent::tests::present_svg_parsers_recover_exact_source_for_both_model_transports ... ok
+test a2ui::tests::catalog_validation_and_actions_use_current_local_data ... ok
+test agent::tests::ralph_loop_stops_on_cap_blocker_failure_and_cancellation ... ok
+test a2ui::tests::all_basics_compose_into_one_surface ... ok
+test a2ui::tests::v1_action_and_function_round_trips_follow_current_schema ... ok
+test agent::tests::ralph_report_semantics_and_handoff_bound_are_enforced ... ok
+test agent::tests::risky_shell_commands_require_one_time_approval ... ok
+test agent::tests::generated_schemas_match_the_executable_tool_contracts ... ok
+test agent::tests::model_adapters_preserve_primitives_and_report_recoverable_syntax_errors ... ok
+test agent::tests::scripted_ralph_backend_gets_fresh_prompts_and_only_validated_handoffs ... ok
+test agent::tests::repeated_call_advisories_use_canonical_arguments_and_reset_per_user_turn ... ok
+test agent::tests::project_boundary_rejects_parent_paths ... ok
+test a2ui::export::tests::static_svg_renderer_covers_the_complete_catalog ... ok
+test agent::tests::ralph_report_schemas_are_child_only_and_parse_for_both_models ... ok
+test agent::tests::stored_tool_cards_restore_ai_approval_reasons_by_call ... ok
+test agent::tests::tool_output_keeps_terminal_lines_but_parameter_summaries_do_not ... ok
+test agent::tests::present_svg_executes_without_side_effects_and_maps_only_completed_cards ... ok
+test agent::web::tests::browser_helpers_preserve_ds4_wire_values ... ok
+test agent::web::tests::headless_browser_extracts_a_local_page ... ignored, requires an installed Chrome browser
+test agent::tests::prompts_and_parsers_expose_the_reference_tool_set ... ok
+test agent::tests::validation_is_precise_and_prevents_side_effects_until_corrected ... ok
+test app::generation::tests::attached_image_marker_is_model_only ... ok
+test agent::tests::present_svg_validation_enforces_the_static_vector_boundary ... ok
+test agent::tests::glob_confinement_skips_symlinks_and_honors_cancellation ... ok
+test agent::tests::successful_structured_file_tools_report_project_paths ... ok
+test agent::tests::glob_finds_all_regular_file_names_deterministically_without_reading_them ... ok
+test agent::tests::standard_agent_skills_are_discovered_prompted_and_read_only ... ok
+test agent::tests::shell_commands_skip_user_startup_files ... ok
+test app::generation::tests::plain_chat_uses_the_same_unmodified_prompt_in_ui_and_headless ... ok
+test app::generation::tests::queued_guidance_is_one_reference_style_user_turn ... ok
+test app::generation::tests::title_context_starts_at_the_first_user_and_ignores_system_rows ... ok
+test app::generation::tests::turn_summary_accumulates_model_continuations ... ok
+test agent::tests::shell_environment_is_loaded_once_from_login_and_interactive_startup_files ... ok
+test agent::tests::standard_file_tools_can_maintain_only_managed_dev_brain_pages ... ok
+test app::git::tests::configured_git_diff_options_change_the_rendered_patch ... ok
+test app::git::tests::unified_diff_parser_aligns_replacements_for_split_view ... ok
+test agent::tests::ralph_round_contexts_are_unique_and_child_jobs_are_cleaned_up ... ok
+test app::git::tests::unstage_works_before_the_first_commit ... ok
+test app::preferences::tests::runtime_config_updates_after_lock_poisoning ... ok
+test app::git::tests::libgit2_status_merges_staged_and_worktree_changes ... ok
+test app::projects::tests::discarding_a_session_removes_every_checkpoint_stage ... ok
+test app::projects::tests::draft_titles_follow_the_stored_session_count ... ok
+test agent::tests::denial_and_stop_cancel_commands_awaiting_approval ... ok
+test app::projects::tests::session_states_are_exclusive_and_ordered ... ok
+test app::projects::tests::sweeping_removes_checkpoints_of_sessions_that_no_longer_exist ... ok
+test app::projects::tests::git_state_lists_and_switches_local_branches ... ok
+test app::tests::background_activity_drives_polling_indicators_and_quit_confirmation ... ok
+test agent::tests::shell_environment_probe_stops_blocked_startup_files ... ok
+test app::git::tests::origin_fetch_pull_and_push_use_the_current_branch ... ok
+test app::tests::chat_html_export_embeds_loaded_a2ui_images_as_data_urls ... ok
+test agent::tests::oversized_shell_output_is_bounded_and_retained_until_cleanup ... ok
+test app::tests::chat_only_follows_updates_while_near_the_tail ... ok
+test app::tests::ds4_gib_and_streaming_cache_inputs_are_typed ... ok
+test app::tests::one_shot_titles_are_reduced_to_a_single_sidebar_line ... ok
+test app::tests::owned_runtime_paths_stay_in_application_support ... ok
+test app::tests::preferences_shortcut_and_dspark_support_are_explicit ... ok
+test app::tests::source_does_not_depend_on_a_sibling_ds4_checkout ... ok
+test app::tests::stored_session_always_selects_its_model ... ok
+test app::tests::tab_scrolls_a_field_back_into_the_preferences_viewport ... ok
+test app::view::a2ui::tests::renderer_input_helpers_cover_dates_numbers_choices_and_weights ... ok
+test app::view::chat::tests::chat_divider_groups_token_counts_by_thousands ... ok
+test app::view::chat::tests::composer_uses_shift_enter_for_line_breaks ... ok
+test app::view::git::tests::split_diff_width_covers_the_longest_side_and_tabs ... ok
+test app::tests::chat_export_preserves_completed_svg_and_keeps_failures_generic ... ok
+test app::view::tests::action_buttons_share_geometry ... ok
+test app::view::tests::context_pie_wedge_follows_the_fill_grade ... ok
+test app::view::tests::download_measurements_are_readable ... ok
+test app::view::tests::raised_surfaces_stay_dark ... ok
+test compaction::tests::a_new_summary_replaces_the_previous_rebuild_summary ... ok
+test compaction::tests::hard_trigger_and_retry_reserve_match_the_reference ... ok
+test compaction::tests::soft_trigger_matches_reference_thresholds ... ok
+test compaction::tests::summary_budget_keeps_answer_room_and_rejects_exhausted_contexts ... ok
+test compaction::tests::summary_control_markup_is_private ... ok
+test compaction::tests::tail_prefers_the_first_user_boundary_after_target ... ok
+test config::tests::defaults_write_an_empty_file_and_reload_unchanged ... ok
+test config::tests::deprecated_flash_config_migrates_and_persists_only_0731 ... ok
+test config::tests::explicit_0731_dspark_opt_out_survives_the_default ... ok
+test config::tests::invalid_files_are_reported_instead_of_reset ... ok
+test config::tests::legacy_flat_model_settings_are_dropped_but_the_prompt_is_kept ... ok
+test config::tests::model_profiles_expose_only_their_upstream_thinking_modes ... ok
+test app::preferences::tests::selectors_rehydrate_their_generation_and_model_profiles ... ok
+test config::tests::only_changed_values_are_written_and_read_back ... ok
+test config::tests::qwen_selection_and_native_defaults_survive_restart ... ok
+test app::preferences::tests::multiline_system_prompt_round_trips_through_the_editor ... ok
+test app::generation::tests::legacy_generation_rows_collapse_into_their_user_turn ... ok
+test app::generation::tests::model_history_uses_persisted_content_without_changing_the_transcript ... ok
+test app::view::stats::tests::current_session_summary_uses_persisted_chat_facts ... ok
+test app::generation::tests::assistant_markdown_keeps_llm_tables ... ok
+test app::generation::tests::forced_compaction_requires_new_visible_chat ... ok
+test app::generation::tests::last_compaction_selects_its_tail_without_hiding_history ... ok
+test app::tests::assistant_stream_keeps_reasoning_separate_from_the_answer ... ok
+test app::generation::tests::extension_context_deduplicates_session_start_and_rearms_after_compaction ... ok
+test app::generation::tests::complete_tool_call_in_reasoning_gets_one_correction ... ok
+test app::tests::chat_export_formats_visible_turns_reasoning_and_bash_tools ... ok
+test app::tests::chat_html_export_is_safe_standalone_and_renders_embedded_surfaces ... ok
+test app::generation::tests::a2ui_auto_switch_waits_for_a_renderable_surface ... ok
+test dev_brain::tests::path_confinement_and_unrelated_notes_are_preserved ... ok
+test database::tests::context_work_reactivates_an_archived_session ... ok
+test dsml::tests::parses_reference_and_legacy_dsml_syntaxes ... ok
+test dsml::tests::recovers_complete_invokes_before_an_incomplete_one ... ok
+test dsml::tests::recovers_complete_invokes_without_the_outer_closing_tag ... ok
+test engine::ds4_sampling::tests::heap_fallback_preserves_rng_and_signed_zero_ties ... ok
+test engine::gguf::tests::accepts_exact_mxfp4_block_layout ... ok
+test database::tests::a2ui_dismissal_persists_a_fresh_surface_boundary ... ok
+test engine::gguf::tests::keeps_unknown_tensor_kinds_rejected ... ok
+test engine::gguf::tests::mxfp4_scalar_dot_matches_independent_values ... ok
+test engine::gguf::tests::maps_metadata_and_tensor_payload_without_copying_weights ... ok
+test engine::gguf::tests::rejects_mxfp4_dimension_overflow ... ok
+test database::tests::inline_svg_tool_call_survives_session_switch_compaction_and_reopen ... ok
+test database::tests::sessions_are_ordered_by_recent_use_within_sidebar_groups ... ok
+test engine::gguf::tests::rejects_mxfp4_rows_that_are_not_block_aligned ... ok
+test engine::gguf::tests::rejects_truncated_mxfp4_payload ... ok
+test engine::metal::glm::tests::dsa_indexer_schedule_matches_the_reference ... ok
+test engine::metal::glm::tests::glm53_m5_text_performance_gate ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm53_m5_vision_performance_and_oracle_gate ... ignored, requires GLM 5.3 Q2, its vision sidecar, a fixed image, and Apple Metal
+test engine::metal::glm::tests::glm53_reference_logits_replay_separates_sampling_from_execution ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm53_reference_prompt_tokens_match_shared_runtime ... ignored, installed GLM and DS4SERVER_GLM_REFERENCE_LOG from the standalone reference
+test engine::metal::glm::tests::glm53_two_row_verifier_restores_recurrent_state ... ignored, installed GLM 5.3 and Metal; run under test-supervisor
+test engine::metal::glm::tests::glm_byte_budget_reserves_prefill_before_dynamic_experts ... ok
+test engine::metal::glm::tests::glm_mtp_preserves_target_tokens_and_drafts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm_prefill_schedule_matches_ds4_boundaries ... ok
+test engine::metal::glm::tests::glm_resident_decode_flushes_periodically_without_a_final_or_ssd_flush ... ok
+test engine::metal::glm::tests::repeated_glm_prompt_rewinds_one_token_for_logits ... ok
+test engine::metal::glm::tests::resident_and_streamed_glm_match_ds4_decode_oracles ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
+test engine::metal::glm::tests::streamed_glm_uses_ds4_indexed_prefill_for_long_prompts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
+test engine::metal::gpu::qwen_mtplx_allocation_tests::backend_allocation_policy_is_explicit_inside_and_outside_qwen_scope ... ignored, requires Apple Metal; allocation only, no GPU dispatch or model
+test engine::metal::gpu::qwen_mtplx_allocation_tests::dispatch_retains_allocation_without_changing_data_donation ... ignored, requires Apple Metal, no model weights
+test engine::metal::gpu::qwen_mtplx_allocation_tests::empty_storage_preserves_data_identity_views_and_zero_allocator_usage ... ignored, requires Apple Metal context; empty descriptors, no GPU allocation
+test engine::metal::gpu::qwen_mtplx_allocation_tests::mtplx_tensor_cpu_access_releases_native_storage ... ignored, requires Apple Metal; CPU buffer access only, no GPU dispatch or model
+test engine::metal::markov::tests::installed_markov_worker_dispatch ... ignored, requires installed 0731 DSpark support; CPU dispatch diagnostic, not parity acceptance
+test engine::metal::markov::tests::markov_workers_reuse_threads_and_buffers_with_ordered_ties ... ok
+test engine::metal::power_throttle_keeps_ds4_ewma_and_full_power_bypass ... ok
+test engine::kvstore::tests::opening_reclaims_checkpoints_the_index_cannot_reach ... ok
+test engine::metal::qwen::tests::affine_split_k_matches_reference_shapes_and_scratch_bound ... ok
+test engine::metal::qwen::tests::dense_attention_crossover_is_phase_specific ... ok
+test engine::kvstore::tests::budget_evicts_the_least_valuable_unprotected_entry ... ok
+test engine::metal::qwen::tests::qwen_affine_qmv_and_qmm_match_mlx_reference ... ignored, requires the pinned Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_affine_qmv_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_core_boundary_and_checkpoint_are_stable ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_fused_gdn_input_rows_match_separate_projections ... ignored, requires the pinned Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_gdn_conv_norm_rows_matches_mtplx ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_gdn_delta_rows_matches_mtplx ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_lazy_target_probabilities_match_batched_sampling ... ignored, requires the pinned Qwen metadata and Apple Metal; no model inference
+test dev_brain::tests::skill_index_migration_does_not_overwrite_an_unrelated_root_note ... ok
+test engine::metal::qwen::tests::qwen_m5_performance_gate ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_metal_primitives_match_reference_vectors ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_mtp_matches_target_and_restores_its_cache ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_ple_gather_copies_only_the_requested_row ... ok
+test database::tests::session_model_migration_preserves_data_and_accepts_catalog ... ok
+test engine::metal::qwen::tests::qwen_qsa_dense_mask_accepts_unsorted_blocks ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_full_context_preallocation_succeeds ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qsa_selection_and_capacity_cover_the_full_native_window ... ok
+test engine::metal::qwen::tests::qwen_qsa_logical_depth_oracles_cover_prefill_decode_and_state ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_real_layer_executes_the_first_sparse_token ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_selection_and_attention_remain_bounded_at_native_depths ... ignored, requires Apple Metal and allocates the full QSA score window
+test engine::metal::qwen::tests::qwen_reference_layer_trace ... ignored, diagnostic trace against the pinned MLX-LM reference
+test engine::metal::qwen::tests::qwen_reference_sdpa_trace ... ignored, diagnostic comparison with captured MTPLX SDPA inputs
+test engine::metal::qwen::tests::qwen_reference_stateful_trace ... ignored, diagnostic trace against the pinned MLX-LM reference
+test engine::metal::qwen::tests::qwen_reference_verify_trace ... ignored, diagnostic trace against the pinned MTPLX verifier
+test engine::metal::qwen::tests::qwen_sorted_routed_prefill_matches_grouped_output ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_stage_counters_preserve_async_batches ... ignored, requires pinned Qwen embeddings and Apple Metal; stage profiler opt-in
+test engine::metal::qwen::tests::qwen_target_submission_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_verify_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen_mtplx::allocator::mtplx_allocations_preserve_logical_bounds_with_reference_rounding ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_heap_limits_and_gpu_lifetime ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_replays_reference_allocations ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::ar::mtplx_ar_driver_matches_reference ... ignored, requires Apple Metal; unchanged MTPLX generate_ar orchestration oracle
+test engine::metal::qwen_mtplx::array::mtplx_array_detach_clears_every_siblings_inputs_and_primitive ... ok
+test engine::metal::qwen::tests::qwen_ple_hash_contract_matches_golden_boundaries ... ok
+test engine::metal::qwen_mtplx::array::mtplx_array_materialization_keeps_geometry_and_rejects_short_storage ... ignored, requires Apple Metal; typed materialized descriptors without model weights
+test engine::metal::qwen_mtplx::array::mtplx_array_layout_preserves_dtype_shape_stride_and_materialization_flags ... ok
+test engine::metal::qwen_mtplx::array::mtplx_dtype_promotion_matches_all_reference_pairs ... ok
+test engine::kvstore::tests::longest_compatible_prefix_wins_and_hits_survive_reopen ... ok
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_dense_mask_graph_matches_reference ... ignored, requires Apple Metal; original MTPLX block mask including sentinel and tails
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_eager_array_graph_matches_reference ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_prefill_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX tiled prefill gather
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_rows_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX QSA rows-gather attention
+test engine::metal::qwen_mtplx::attention::mtplx_sdpa_fallback_graph_matches_reference ... ignored, requires Apple M5 Max; actual Qwen SDPA fallback reference
+test engine::metal::qwen_mtplx::binary::mtplx_graph_binary_preserves_reference_branches ... ignored, requires Apple Metal; binary graph broadcast/layout and donation branches
+test engine::metal::qwen_mtplx::binding::mtplx_installed_module_binding_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS and SSD policy; only small norms and four table rows evaluated
+test engine::metal::qwen_mtplx::cache::mtplx_kv_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX KVCache growth, trim and state receipts
+test engine::metal::qwen_mtplx::cache::mtplx_qsa_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX positional cache, mirror and held state aliases
+test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_scalar_bits_and_custom_kernel_identity ... ignored, requires Metal buffers and external supervisor; scalar ownership only, no GPU dispatch
+test engine::metal::qwen_mtplx::context_copy::prompt_copy_match_and_backoff_preserve_reference_policy ... ok
+test engine::metal::qwen_mtplx::decoder::mtplx_decoder_connected_attention_and_moe_advance_caches ... ignored, requires Apple Metal and external test-supervisor; complete decoder/cache smoke
+test engine::metal::qwen_mtplx::decoder::mtplx_decoder_residual_preserves_graph_order ... ok
+test engine::metal::qwen_mtplx::dense::mtplx_dense_attention_batches_match_reference ... ignored, requires Apple M5 Max; actual BF16 batched matmul reference
+test engine::metal::qwen_mtplx::dense::mtplx_dense_float_graph_matches_reference ... ignored, requires Apple M5 Max; floating Matmul factory and original dispatch
+test engine::metal::qwen_mtplx::compiled::mtplx_replay_replaces_descriptors_without_rebuilding_primitives ... ok
+test engine::metal::qwen_mtplx::dense::mtplx_qsa_score_array_graph_matches_reference ... ignored, requires Apple M5 Max; canonical QSA F32 graph with the original TF32 setting
+test engine::metal::qwen_mtplx::encoder::mtplx_empty_bindings_preserve_nil_dependencies_without_dummy_storage ... ignored, requires Apple Metal; empty original-kernel grids and nil bindings
+test engine::metal::qwen_mtplx::allocator::mtplx_allocator_cache_matches_live_reference_trace ... ok
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_commits_after_primitives_and_balances_completion ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_scope_routes_operators_and_allocator ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_shared_events_and_error_completion_order ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_untracked_dependencies_across_encoders_and_commands ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_drains_failure_preserves_cause_and_retries_only_failed_nodes ... ignored, requires Apple Metal; supervised evaluator failure and retry
+test database::tests::migrations_and_crud_work ... ok
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_releases_intermediates_and_preserves_descriptor_donation ... ignored, requires Apple Metal; graph ownership and last-consumer release
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_barrier_epochs_and_commit_thresholds ... ok
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_retains_tracer_topology_without_input_donation ... ignored, requires Apple Metal; evaluator tracer donation and retained topology
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_runs_cpu_gpu_siblings_and_async_continuations ... ignored, requires Apple Metal; tiny supervised CPU/GPU evaluator graph
+test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_scheduled_detach_and_tracer_errors ... ok
+test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ar_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_AR_REFERENCE required
+test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_tracks_stream_fences_per_leaf_and_cross_device_union ... ok
+test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_bfs_width_and_sibling_choice ... ok
+test engine::metal::qwen_mtplx::execution::mtplx_full_installed_mtp_prefill_matches_reference ... ignored, full installed MTP model; external guard and DS4_QWEN38_MTP_PREFILL_REFERENCE required
+test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ping_forward_matches_reference ... ignored, full installed model; requires external memory guard and DS4_QWEN38_FORWARD_REFERENCE
+test engine::metal::qwen_mtplx::execution::qwen_power_throttle_drains_before_sleep_and_bypasses_at_full_power ... ignored, requires Apple Metal; run under test-supervisor
+test engine::metal::qwen_mtplx::fused::mtplx_compiled_fusion_gpu_matches_reference ... ignored, requires Metal and external supervisor; generated fusion vs pinned SiLU plus multi-output/scalar dispatch
+test engine::metal::qwen_mtplx::fused::mtplx_compiled_constant_spelling ... ok
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_forward_matches_model_reference ... ignored, requires Apple Metal; actual MTPLX GDN forward and cache receipts
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_prefill_and_verify_match_model_reference ... ignored, requires Apple Metal; complete GDN Conv1d prefill and fused verify graphs
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_staged_primitives_match_reference ... ignored, requires Apple Metal; 80 actual delta-update receipts and six conv-normalization kernels
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_step_preserves_views_and_siblings ... ignored, requires Apple Metal; GDN custom primitive strided copies and sibling roots
+test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_event_snapshots_preserve_values_streams_and_shared_errors ... ignored, requires Apple Metal events, no model weights
+test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fence_coherence_preserves_empty_and_partial_word_spans ... ignored, requires Apple Metal3, model-free empty coherent dispatch
+test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fences_transfer_cpu_gpu_gpu_cpu_in_both_modes ... ignored, requires Apple Metal3, tiny model-free coherent fence kernels
+test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_forward_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX GatedResidual reference receipts
+test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_pack_and_reads_match_reference ... ignored, requires Apple Metal; original hyper primitives and runtime pack
+test engine::metal::qwen_mtplx::indexing::mtplx_graph_indexing_preserves_layouts_and_offsets ... ignored, requires Apple Metal; Select layout branches, GatherAxis views and empty arguments
+test engine::metal::qwen_mtplx::indexing::mtplx_scatter_axis_graph_matches_reference ... ignored, requires Apple Metal; pinned Bool ScatterAxis layout and broadcast paths
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_numeric_receipts ... ignored, #103 diagnostic; Metal required, eager versus compiled numeric operands
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_sampling_receipts ... ignored, #103 diagnostic; Metal required, pins currently broken sampling law and invalid rows
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_sorted_gather_receipts ... ignored, #103 diagnostic; M5 and pinned runtime shader required; initialized sentinels
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_tokenizer_receipts ... ignored, #103 diagnostic; installed tokenizer and latest oracle JSONL; no GPU
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_vision_position_receipts ... ignored, #103 diagnostic; small active attention module on Metal; mixed position boundary
+test engine::metal::qwen_mtplx::issue104::issue104_installed_ar_stream_and_state ... ignored, full pinned Bare-Speed model; external memory guard required
+test engine::metal::qwen_mtplx::issue105::issue105_batched_greedy_rejects_each_invalid_row ... ignored, Metal required; run serially under test-supervisor
+test engine::metal::qwen_mtplx::issue106::issue106_complete_state_and_checkpoint_recovery ... ignored, installed model and actual pre-fix checkpoint; external memory guard required
+test engine::metal::qwen_mtplx::issue106::issue106_upstream_exit_matrix ... ignored, installed model, scripted logits on real caches; external memory guard required
+test engine::metal::qwen_mtplx::issue108::issue108_attention_chunk_positions ... ignored, Metal and pinned MTPLX chunk fixtures; external supervisor required
+test engine::metal::qwen_mtplx::issue109::issue109_gate_phase_and_dtype_laws ... ignored, Metal and exhaustive pinned gate fixtures; external supervisor required
+test engine::metal::qwen_mtplx::issue110::issue110_actual_projection_rows_match_oracle ... ignored, installed layer weights and pinned oracle receipts; external memory supervisor required
+test engine::metal::qwen_mtplx::issue110::issue110_guard_allocation_refusal_releases_graph_and_retries ... ignored, Metal allocation refusal and retry; external supervisor required
+test engine::metal::qwen_mtplx::issue110::issue110_large_cold_and_warm_prefill ... ignored, full installed model and large partial chunks; serial memory supervisor required
+test engine::metal::qwen_mtplx::issue112_bench::issue112_hidden_prefill ... ignored, full installed model hidden-state prefill; serial supervisor required
+test engine::metal::qwen_mtplx::issue112_bench::issue112_paired_prefill ... ignored, full installed model; serial memory supervisor and explicit output directory
+test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_dequantize_matches_reference ... ignored, requires Apple Metal; original table dequantize dtype/group/bit/layout receipts
+test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_quantize_matches_reference ... ignored, requires Apple Metal; original affine quantize dtype/group/bit/layout receipts
+test engine::metal::qwen_mtplx::linear::mtplx_graph_gather_qmm_matches_reference ... ignored, requires Apple Metal; original GatherQMM array layouts and expert routing
+test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_batched_identity ... ignored, requires Apple Metal and external supervisor; strided/broadcast batched affine dispatch
+test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_matches_reference ... ignored, requires Apple M5 Max Metal; 78 pinned affine fixtures through lazy graph
+test engine::metal::qwen_mtplx::load::mtplx_safetensors_load_laziness_boundaries_and_lifetime ... ignored, requires Apple Metal; bounded CPU Load and dependent GPU graph
+test engine::metal::qwen_mtplx::mlp::mtplx_graph_shared_mlp_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX shared MLP reference receipts
+test engine::metal::qwen_mtplx::linear::sorted_gather_guard_matches_pinned_call_predicates ... ok
+test engine::metal::qwen_mtplx::mlp::mtplx_graph_split_and_compiled_silu_layouts ... ignored, requires Apple Metal; Split sibling ownership and compiled SiLU layouts
+test engine::metal::qwen_mtplx::moe::mtplx_graph_sparse_moe_matches_reference ... ignored, requires Apple M5 Max; full MoE Array graph, no model weights; use test-supervisor
+test engine::metal::qwen_mtplx::model::mtplx_model_compile_policy_and_run_boundaries ... ok
+test engine::metal::qwen_mtplx::moe_sorted::issue112_all_kernel_variants ... ignored, Metal synthetic all supported row-map variants; external supervisor
+test engine::metal::qwen_mtplx::moe_sorted::issue112_canary_failures ... ignored, Metal canary rejection and sticky fallback; external supervisor
+test engine::metal::qwen_mtplx::moe_sorted::issue112_full_model_state ... ignored, full installed model; validates all cold/warm state receipts against the pre-port oracle
+test engine::metal::qwen_mtplx::moe_sorted::issue112_installed_gate_up ... ignored, installed projection weights; external supervisor required
+test engine::metal::qwen_mtplx::moe_sorted::row_map_applicability ... ok
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_embedding_matches_reference ... ignored, requires Apple Metal; full resident NGram graph and lazy decode history
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_gpu_rows_and_history_match_reference ... ignored, requires Apple Metal; original lazy NGram GPU row graph, no table
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_resident_matches_reference ... ignored, requires Apple Metal; original NGramTable resident gather/dequantize receipts
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_table_priority ... ignored, requires Apple Metal; NGram table routing and missing-sidecar behavior
+test engine::metal::qwen_mtplx::ngram_sidecar::mtplx_ngram_sidecar_matches_reference ... ignored, requires Apple Metal; actual file-backed MTPLX sidecar/LRU/dequantization receipts
+test database::tests::chat_and_compaction_history_survive_reopen_and_session_deletion ... ok
+test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_allclose_branches ... ignored, requires Apple Metal; stage verification special values and original reduction branches
+test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_matches_reference ... ignored, requires Apple Metal; original complete stage lifecycle and file-backed tables
+test engine::metal::qwen_mtplx::normalization::mtplx_graph_norm_modules_match_reference ... ignored, requires Apple Metal; connected Qwen normalization graph golden fixtures
+test engine::metal::qwen_mtplx::normalization::mtplx_graph_rms_norm_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph RMSNorm donation, copy, broadcast and looped kernels
+test engine::metal::qwen_mtplx::normalization::mtplx_logsumexp_graph_matches_reference ... ignored, requires Apple Metal; original sampler LogSumExp branches
+test engine::metal::qwen_mtplx::normalization::mtplx_softmax_graph_storage_matches_reference ... ignored, requires Apple Metal; original Softmax type and storage branches
+test engine::metal::qwen_mtplx::ops::mtplx_concatenate_preserves_general_general_copy_semantics ... ignored, requires Apple Metal; Concatenate promotion, offsets, strided writers and validation
+test engine::metal::qwen_mtplx::ops::mtplx_owned_operations_build_and_execute_qsa_layout_graph ... ignored, requires Apple Metal; self-contained lazy QSA-layout recipe, no callback map
+test engine::metal::qwen_mtplx::ops::mtplx_slice_and_contiguous_preserve_cache_window_semantics ... ignored, requires Apple Metal; normalized Slice and Contiguous cache-window graph
+test engine::metal::qwen_mtplx::owner::issue105_product::issue105_product_ar_cleanup_and_http_retry ... ignored, installed Qwen model; serial test-supervisor with memory guard required
+test engine::metal::qwen_mtplx::owner::issue105_product::issue105_product_mtp_cleanup_and_http_retry ... ignored, installed Qwen MTP model; serial test-supervisor with memory guard required
+test engine::metal::qwen_mtplx::owner::issue106_product::issue106_product_discards_old_checkpoint_and_recovers_cold ... ignored, installed model and actual pre-fix checkpoint; external memory guard required
+test engine::metal::qwen_mtplx::owner::issue107_product::issue107_vision_product ... ignored, installed model plus vision tower; serial external memory supervisor required
+test engine::metal::qwen_mtplx::owner::issue109_product::issue109_verify_rows_and_warm_checkpoint ... ignored, installed model; serial external memory supervisor required
+test engine::metal::qwen_mtplx::fused::mtplx_compiled_emitter_matches_all_pinned_silu_bodies ... ok
+test engine::metal::qwen_mtplx::owner::issue111_product::issue111_token_prefix_cache ... ignored, installed model; serial external memory supervisor required
+test engine::metal::qwen_mtplx::owner::qwen_product_generator_chat_matches_reference ... ignored, full installed model through product Generator; external supervisor and chat reference required
+test engine::metal::qwen_mtplx::owner::qwen_vision_chat_checkpoint_preserves_image_identity ... ignored, requires installed Qwen, Metal, and DS4_QWEN_VISION_IMAGE; run under test-supervisor
+test engine::metal::qwen_mtplx::fused::mtplx_compile_fusion_preserves_parent_and_output_boundaries ... ok
+test engine::metal::qwen_mtplx::owner::qwen_mtp_progress_publishes_without_turn_completion_or_double_counting ... ok
+test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_connected_matches_reference ... ignored, requires Apple M5 Max; complete original PLELayer with real lookup
+test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_projection_and_cache_match_reference ... ignored, requires Apple M5 Max; actual MTPLX PLELayer after embedding lookup
+test engine::metal::qwen_mtplx::policy::qwen_server_policy_matches_original_functions ... ignored, requires DS4_QWEN38_POLICY_REFERENCE from pinned MTPLX; CPU-only
+test engine::metal::qwen_mtplx::prefill::mtplx_mtp_prefill_driver_matches_reference ... ignored, requires Apple Metal; original streaming MTP history prefill
+test engine::metal::qwen_mtplx::policy::sorted_moe_switch_and_family_rehearsal ... ok
+test engine::metal::qwen_mtplx::prefill::mtplx_prefill_driver_matches_reference ... ignored, requires Apple Metal; original _prefill control flow and raw cache roots
+test engine::metal::qwen_mtplx::qsa_attention::mtplx_qsa_attention_module_matches_reference ... ignored, requires Metal and unchanged whole-Attention receipts; no model download
+test engine::metal::qwen_mtplx::qsa_compile::mtplx_compiled_indexer_array_replay_matches_reference ... ignored, requires Metal and original compiled indexer receipts; no model weights
+test engine::metal::qwen_mtplx::qsa_core::mtplx_qsa_core_host_matches_reference ... ignored, requires Metal and original host-core receipts; no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_cache_donation_requires_descriptor_data_and_allocation_contract ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_completion_owns_buffers_until_gpu_finishes ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_evaluator_retains_inputs_and_siblings_not_primary_data ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::qsa_indexer::issue107::issue107_real_geometry_query_and_block_start_positions ... ignored, Metal and independently verified pinned indexer fixtures; external supervisor required
+test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_canonical_indexer_routes_match_reference ... ignored, requires Metal and original ongoing indexer routing receipts; no weights
+test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_compiled_commit_matches_reference ... ignored, requires Metal and original indexer lifecycle receipts; no model weights
+test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_prefix_matches_reference ... ignored, requires Apple Metal, original stateful QSAIndexer receipts, no model weights
+test engine::metal::qwen_mtplx::qsa_prefill::mtplx_qsa_prefill_graph_matches_reference ... ignored, requires Metal and original two-stage prefill receipts, no weights
+test engine::metal::qwen_mtplx::qsa_prepare::mtplx_qsa_prepare_graph_matches_reference ... ignored, requires Apple Metal; parameterized original QSA preparation and traced positions
+test engine::metal::qwen_mtplx::qsa_select::mtplx_qsa_fused_graph_matches_reference ... ignored, requires Apple Metal and original QSA selector receipts; no model weights
+test engine::metal::qwen_mtplx::random::mtplx_sampler_prefix_matches_reference ... ignored, requires Apple Metal; original target-prefix sampler and default RNG
+test engine::metal::qwen_mtplx::random::mtplx_sampler_resident_matches_reference ... ignored, requires Apple Metal; original MTPLX resident sampling and keyed RNG
+test engine::metal::qwen_mtplx::qsa_graph::qsa_schedule_preserves_the_selected_sibling ... ok
+test engine::metal::qwen_mtplx::policy::retired_ar_pipeline_flag_cannot_change_server_or_turn_policy ... ok
+test engine::metal::qwen_mtplx::reduce::mtplx_sampler_argmax_matches_reference ... ignored, requires Apple Metal; original MTPLX greedy ArgReduce and storage
+test engine::metal::qwen_mtplx::reduce::mtplx_sum_graph_matches_reference ... ignored, requires Apple Metal; original shared sum dispatch and QSA mean
+test engine::metal::qwen_mtplx::residency::mtplx_residency_matches_reference_lifecycle_cases ... ignored, requires Apple Metal 3 / macOS 15, no model weights
+test engine::metal::qwen_mtplx::rope::mtplx_rope_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX RoPE, static YaRN and multi-axis receipts
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_batched_matches_reference ... ignored, requires Apple Metal; original MTPLX bound and serial batch distributions
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_entry_matches_reference ... ignored, requires Apple Metal; actual MTPLX generator sampling entry
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_fallback_matches_reference ... ignored, requires Apple Metal; original sparse fallback and host dense sampler
+test engine::metal::qwen_mtplx::request::qwen_request_progress_uses_ui_frontiers_and_decode_timer ... ok
+test engine::metal::qwen_mtplx::request::qwen_request_settings_override_backend_policy ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_penalties_match_reference ... ignored, requires Apple Metal; original GPU and CPU raw-logit penalties
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_selection_matches_reference ... ignored, requires Apple Metal; original MTPLX sampler selection, no model weights
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_serial_support_matches_reference ... ignored, requires Apple Metal; actual MTPLX serial candidate route, no weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_primitive_holds_data_and_temporaries_in_fifo_order ... ignored, requires Apple Metal shared storage, no GPU dispatch or model weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_dispatch_counts_tenth_and_completes_after_failed_work ... ok
+test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_simplify_preserves_reference_parent_rules ... ok
+test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_cpu_events_propagate_errors_without_consuming_source ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::sdpa::mtplx_sdpa_vector_matches_reference ... ignored, requires Apple M5 Max; actual pinned fused SDPA kernels
+test engine::metal::qwen_mtplx::session_cache::mtplx_full_installed_chat_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_CHAT_REFERENCE required
+test engine::metal::qwen_mtplx::session_cache::mtplx_session_boundary_policy ... ok
+test engine::metal::qwen_mtplx::session_cache::mtplx_session_final_state_requires_capture_and_safe_cache ... ok
+test engine::metal::qwen_mtplx::session_checkpoint::qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state ... ignored, model-free Metal checkpoint; external supervisor and DS4_QWEN38_CHECKPOINT_DIR required
+test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_host_import_matches_reference ... ignored, model-free Metal imports; external supervisor and DS4_HOST_IMPORT_REFERENCE required
+test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_matches_original_blobs_and_restore ... ignored, tiny GPU arrays; external supervisor and DS4_SESSION_CODEC_REFERENCE required
+test engine::metal::qwen_mtplx::sort::mtplx_sort_graph_matches_reference ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_streams_preserve_fifo_errors_and_shutdown ... ok
+test engine::metal::qwen_mtplx::sparse_attention::mtplx_sparse_attention_graph_matches_reference ... ignored, requires Apple M5; actual MTPLX sparse Decode and TensorOps Prefill consumers
+test engine::metal::qwen_mtplx::speculative::mtplx_full_installed_mtp_generation_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_MTP_GENERATION_REFERENCE required
+test engine::metal::qwen_mtplx::stream::mtplx_stream_routes_queues_events_and_finalizes_all_under_pressure ... ignored, requires Apple Metal, tiny model-free two-queue copy
+test engine::metal::profile::tests::profile_reports_locality_and_lru_hits ... ok
+test engine::metal::qwen_mtplx::submission::mtplx_submission_attaches_at_flush_eval_finish_and_drop ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::unary::mtplx_graph_unary_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph unary layout, donation and kernel boundaries
+test engine::metal::qwen_mtplx::views::mtplx_dtype_conversion_and_views_preserve_values_layout_and_donation ... ignored, requires Apple Metal; original conversion/view kernels and donation in lazy eval
+test engine::metal::qwen_mtplx::views::mtplx_dynamic_slice_graph_matches_reference ... ignored, requires Metal and original dynamic slice receipts; no model weights
+test engine::metal::qwen_mtplx::views::mtplx_reshape_shares_or_copies_in_the_connected_evaluator ... ignored, requires Apple Metal; original Reshape copy kernels and shared backing
+test engine::metal::qwen_mtplx::views::mtplx_views_preserve_aliases_offsets_strides_and_gpu_output ... ignored, requires Apple Metal; connected zero-copy views and offset-aware original copy kernel
+test engine::metal::qwen_mtplx::vision::qwen_vision_tower_matches_mtplx_image ... ignored, requires Metal and pinned Qwen vision artifacts/reference values
+test engine::metal::qwen_mtplx::weights::mtplx_checkpoint_projection_fusions_match_reference ... ignored, requires Apple Metal; bounded lazy checkpoint Load and fusion
+test engine::metal::qwen_mtplx::weights::mtplx_installed_checkpoint_fusion_metadata_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS; headers only, never evaluate weights
+test engine::metal::qwen_mtplx_tests::mtplx_affine_selection_preserves_reference_boundaries ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_canonical_hyper_and_gdn_kernels_match_reference ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::stream::mtplx_stream_registry_preserves_thread_defaults_templates_and_clear ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_connected_delta_update_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gather_sort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_decode_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_forward_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_staged_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_norm_modules_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_router_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_sparse_moe_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_switch_glu_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_gate_up_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_cache_lifecycle_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_backings_and_dynamic_copies_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_routing_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_window_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_capacity_matches_reference ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_indexer_modes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_scalar_sources_match_reference ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_connected_cache_selection_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_eager_topk_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_f32_scores_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_preparation_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_routes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_producer_dispatch_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_canonical_source_bodies_preserve_pinned_hashes ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_selector_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_affine_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_argsort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_gather_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_activity_preserves_reference_pressure_and_wait_rules ... ok
+test engine::metal::tests::batch_raw_span_includes_the_chunk_and_previous_window ... ok
+test engine::metal::tests::compression_schedule_tracks_the_deepseek_model_shape ... ok
+test engine::metal::tests::decode_feature_policy_keeps_device_generations_and_rollbacks_separate ... ok
+test engine::metal::tests::decode_raw_span_tracks_the_logical_sliding_window ... ok
+test engine::metal::tests::directional_steering_matches_the_ds4_token_oracle ... ignored, requires the installed Flash GGUF, DS4 steering fixture, and a Metal device
+test engine::metal::tests::dspark_confidence_quantizes_q8_input_like_ds4 ... ok
+test engine::metal::tests::dspark_markov_activation_matches_q8_rounding ... ok
+test engine::metal::tests::dspark_matches_original_summary_cycles ... ignored, requires installed 0731/DSpark and DS4SERVER_DSPARK_REFERENCE cycle recording
+test engine::metal::tests::dspark_runs_a_target_owned_greedy_cycle ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures
+test engine::metal::tests::dspark_scheduler_matches_the_ds4_default_window ... ok
+test engine::metal::tests::dspark_verifier_selects_only_the_committed_row ... ok
+test engine::metal::tests::flash_0731_long_context_crosses_indexed_prefill_boundary ... ignored, requires a 0731 Flash GGUF and an Apple M5 device
+test engine::metal::tests::flash_0731_m5_decode_performance_gate ... ignored, requires a 0731 Flash GGUF and an Apple M5 device
+test engine::metal::tests::flash_0731_resident_and_ssd_streaming_choose_the_same_tokens ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
+test engine::metal::tests::flash_0731_runs_exact_sampled_dspark ... ignored, requires the installed 0731 target and checkpoint-specific DSpark GGUF fixtures
+test engine::metal::tests::flash_0731_ssd_streaming_maps_and_seeds_batched_prefill_layers ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
+test engine::metal::tests::generated_hotlists_match_each_model_shape ... ok
+test engine::metal::tests::half_conversion_handles_normal_and_subnormal_values ... ok
+test engine::metal::tests::pro_q4_model_spans_remain_isolated ... ok
+test engine::metal::tests::pro_resident_admission_includes_weights_context_and_scratch ... ok
+test engine::metal::tests::pro_ssd_streaming_full_layer_and_selected_batch_agree ... ignored, requires the installed DeepSeek V4 Pro GGUF and Apple Metal
+test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
+test engine::metal::tests::ssd_streaming_supports_dspark ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures
+test engine::metal::tests::streaming_memory_plan_matches_ds4_graph_formulas ... ok
+test engine::metal::vision::tests::glm53_resize_matches_reference_token_grids ... ok
+test engine::metal::qwen_mtplx::submission::mtplx_all_native_command_commits_pass_the_submission_hook ... ok
+test engine::qwen::tests::metadata_and_safetensors_boundaries_fail_closed ... ok
+test engine::qwen::tests::pinned_artifact_set_loads_and_renders_goldens ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
+test engine::qwen::tests::pinned_qwen_metadata_matches_product_without_executor_state ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
+test engine::qwen_vision::qwen_image_cache_keys_distinguish_pixels_and_preserve_prior_text ... ok
+test engine::qwen::tests::frozen_inventory_and_memory_categories_are_exact ... ok
+test engine::qwen_vision::qwen_vision_positions_and_resize_contract ... ok
+test dev_brain::tests::parser_and_index_rebuild_ignore_links_in_code ... ok
+test engine::sampling_tests::checkpoint_rebuilds_explain_compatibility_and_history_misses ... ok
+test engine::sampling_tests::checkpoint_tag_accepts_an_unchanged_prefix_before_reminders ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_distribution_and_rng_match_reference ... ok
+test engine::sampling_tests::checkpoint_headers_identify_the_exact_model ... ok
+test engine::sampling_tests::checkpoint_tag_covers_the_canonical_chat_state ... ok
+test engine::sampling_tests::ds4_sampler_real_logits_cpu_benchmark ... ignored, CPU-only; requires DS4_SAMPLER_RECORDING with the recorded GLM logits
+test engine::sampling_tests::exact_delta_sampling_accepts_or_corrects_the_draft ... ok
+test engine::sampling_tests::generated_text_drains_incomplete_bytes_before_phase_changes_and_finish ... ok
+test engine::sampling_tests::metal_executes_real_flash_token ... ignored, requires the 80 GiB Flash checkpoint and Apple Metal
+test engine::sampling_tests::generated_text_preserves_thinking_utf8_and_split_stops ... ok
+test engine::sampling_tests::qwen_rng_matches_numpy_pcg64 ... ok
+test engine::sampling_tests::qwen_sampling_uses_mtplx_global_nucleus_mass_before_top_k ... ok
+test engine::sampling_tests::resumed_prefill_crossover_matches_each_ds4_graph ... ok
+test engine::sampling_tests::sampling_probabilities_match_ds4_filter_order ... ok
+test engine::sampling_tests::split_utf8_token_bytes_are_joined_before_decoding ... ok
+test engine::sampling_tests::top_k_and_stream_stops_are_applied_before_output ... ok
+test engine::sampling_tests::workspace_instruction_identity_participates_in_append_only_kv_keys ... ok
+test engine::sampling_tests::zero_temperature_is_greedy ... ok
+test engine::tokenizer::issue111::issue111_ds4_families ... ignored, CPU-only; installed DS4 GGUFs and exact oracle IDs
+test engine::tokenizer::issue111::issue111_every_unicode_scalar_split ... ignored, CPU-only; exhaustive pinned Qwen split receipt
+test engine::tokenizer::issue111::issue111_qwen_tokens_and_requests ... ignored, CPU-only; installed Qwen tokenizer and pinned exact IDs
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_workspace_matches_reference ... ok
+test engine::tokenizer::tests::character_helpers_are_total_at_every_byte_offset ... ok
+test engine::tokenizer::tests::ds4_chat_matches_original_session_tokens ... ignored, CPU-only; requires DS4SERVER_CHAT_REFERENCE and its installed GGUF
+test engine::tokenizer::tests::qwen_checkpoint_continuation_matches_full_conversation ... ignored, requires the installed Qwen tokenizer, but no GPU or model weights
+test engine::tokenizer::tests::qwen_plain_chat_matches_mtplx_server_tokens ... ignored, requires local tokenizer and tools/qwen-chat-reference.py output; no inference
+test engine::tokenizer::tests::reasoning_prefixes_match_each_model_family_contract ... ok
+test engine::validation::tests::configured_glm53_main_fixture_passes_the_exact_layout ... ok
+test engine::validation::tests::configured_glm53_vision_fixture_passes_the_exact_layout ... ok
+test engine::tokenizer::issue111::qwen_split_boundaries ... ok
+test engine::validation::tests::installed_dspark_fixture_passes_the_target_layout ... ok
+test engine::validation::tests::mxfp4_is_limited_to_routed_experts ... ok
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_cpu_rows_and_raw_history_match_reference ... ok
+test extensions::tests::duplicate_skill_names_and_update_rollback_are_deterministic ... ok
+test dev_brain::tests::validation_names_every_drifted_source_and_requires_semantic_refresh ... ok
+test extensions::tests::hook_output_is_bounded_typed_and_event_matched ... ok
+test engine::sampling_tests::probability_ratio_speculation_preserves_the_target_distribution ... ok
+test extensions::tests::manifest_and_paths_are_validated_without_following_escapes ... ok
+test extensions::tests::pinned_ponytail_runs_all_lifecycle_modes ... ignored, downloads the pinned Ponytail reference integration
+test extensions::tests::registry_persists_enable_disable_errors_and_uninstall ... ok
+test extensions::tests::symlink_and_variable_escapes_are_rejected ... ok
+test instructions::tests::changes_removals_resume_and_compaction_are_append_only ... ok
+test engine::metal::qwen_mtplx::array::mtplx_array_releases_sibling_cycles_and_deep_input_graphs ... ok
+test instructions::tests::discovery_is_global_to_local_deduplicated_and_confined ... ok
+test instructions::tests::empty_and_repaired_baselines_do_not_repeat_prompt_content ... ok
+test instructions::tests::newly_crossed_scope_is_added_once_and_unavailable_content_is_retained ... ok
+test metrics::tests::cache_report_splits_buckets_and_folds_index_files_into_their_checkpoint ... ok
+test metrics::tests::decode_timer_excludes_later_work_and_resets_between_requests ... ok
+test metrics::tests::prompt_timing_keeps_unmeasured_components_distinct_from_zero ... ok
+test metrics::tests::removed_preview_metric_code_stays_decodable ... ok
+test metrics::tests::snapshots_track_runtime_and_server_counters ... ok
+test model::qwen::qwen_vision_is_an_independent_managed_artifact_set ... ok
+test model::qwen::tests::embedded_manifest_is_the_frozen_complete_set ... ok
+test model::qwen::tests::verification_rejects_hash_mismatch_and_honors_cancellation ... ok
+test model::transfer::tests::cancellation_keeps_the_partial_file_for_the_next_run ... ok
+test model::transfer::tests::catalog_and_checksum_verification_are_explicit ... ok
+test model::transfer::tests::managed_artifact_inventory_and_delete_include_partial_files ... ok
+test model::transfer::tests::qwen_set_verifies_reloads_and_removes_as_one_managed_artifact ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
+test instructions::tests::frames_escape_control_tags_and_enforce_utf8_byte_limits ... ok
+test model::transfer::tests::restart_resumes_at_the_existing_partial_byte ... ok
+test model::transfer::tests::verification_reports_bytes_read ... ok
+test model_eval::tests::acceleration_receipt_reports_effective_sampling_contract ... ok
+test dev_brain::tests::skill_validation_and_default_guide_restore_are_strict ... ok
+test model_eval::tests::acceleration_switch_uses_each_models_native_path ... ok
+test model_eval::tests::canary_cross_phase_samples_keep_their_start_phase ... ok
+test model_eval::tests::evaluations_exclude_overlapping_workers_and_release_on_exit ... ok
+test model_eval::tests::external_canary_observes_phases_and_finishes_on_eof ... ignored, Metal probe only; DS4SERVER_CANARY_BINARY must point to the current release binary
+test model_eval::tests::external_canary_protocol_accepts_only_known_phases ... ok
+test model_eval::tests::parses_safe_defaults_and_overrides ... ok
+test model_eval::tests::rejects_unbounded_or_invalid_limits ... ok
+test model_eval::tests::repeated_prompts_form_one_ordered_conversation ... ok
+test model_eval::tests::warmup_timing_separates_loading_without_inventing_missing_durations ... ok
+test model_eval::tests::worker_progress_events_advance_the_watchdog ... ok
+test native_edit::tests::edit_history_supports_undo_and_redo ... ok
+test native_edit::tests::paste_replays_a_complete_command_shortcut ... ok
+test native_media::tests::media_player_only_accepts_remote_http_urls ... ok
+test native_menu::tests::edit_items_are_custom_menu_events ... ok
+test native_menu::tests::user_guide_uses_unmodified_f1 ... ok
+test native_menu::tests::view_items_keep_the_selected_pane_checked ... ok
+test runtime::tests::dropping_active_generation_cancels_it ... ok
+test runtime::tests::operation_failures_use_the_matching_event ... ok
+test runtime::tests::operation_tracking_is_consistent ... ok
+test runtime::tests::runtime_panic_recovery_accepts_the_next_operation ... ok
+test server::request::tests::reasoning_requests_preserve_each_models_exact_modes ... ok
+test server::tests::canonical_tool_replay_preserves_argument_types ... ok
+test server::tests::connection_slots_enforce_the_limit_and_release_on_drop ... ok
+test server::tests::content_arrays_match_ds4_text_projection ... ok
+test server::tests::context_limit_errors_use_protocol_standard_fields ... ok
+test server::tests::cors_headers_are_opt_in_and_cover_preflight ... ok
+test server::tests::dsml_tool_stream_projects_incremental_json_arguments ... ok
+test server::tests::generated_dsml_becomes_openai_tool_calls ... ok
+test server::tests::generated_tool_parser_accepts_ds4_recovery_syntaxes ... ok
+test server::tests::model_metadata_uses_the_requested_alias_and_default_token_limit ... ok
+test server::tests::protocol_tool_results_require_live_or_replayed_call_ids ... ok
+test dev_brain::tests::source_hash_drift_excludes_page_from_authoritative_queries ... ok
+test server::tests::responses_parser_rejects_dropped_state_and_preserves_hosted_calls ... ok
+test server::tests::sampled_tool_replay_survives_server_restart ... ok
+test server::tests::streaming_errors_match_ds4_server_framing ... ok
+test dev_brain::tests::verified_skills_are_indexed_and_injected_on_demand ... ok
+test server::tests::tool_prompt_matches_ds4_server_text ... ok
+test server::tests::tool_stream_projects_ds4_recovery_syntaxes ... ok
+test settings::tests::captured_ds4_cli_options_are_uniquely_classified ... ok
+test settings::tests::effective_settings_build_one_engine_and_turn_configuration ... ok
+test settings::tests::effective_settings_preserve_unset_model_defaults ... ok
+test settings::tests::execution_settings_keep_ds4_automatic_values_and_glm_rules ... ok
+test settings::tests::kv_cache_gates_follow_the_ds4_store_rules ... ok
+test settings::tests::reasoning_selection_is_exact_and_model_specific ... ok
+test settings::tests::runtime_settings_preserve_automatic_values_and_cross_field_rules ... ok
+test settings::tests::speculative_settings_match_acceleration_defaults_and_dependencies ... ok
+test dev_brain::tests::validation_keeps_broken_links_editable_and_reports_provenance_errors ... ok
+test server::tests::streaming_prefill_keeps_the_client_connection_alive ... ok
+test dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree ... ok
+test model_eval::tests::cpu_accounting_matches_getrusage_units ... ok
+test engine::validation::tests::installed_ds4_fixture_opens_and_renders_a_prompt ... ok
+test model::transfer::tests::stalled_download_times_out_and_keeps_the_partial_file ... ok
+test server::tests::request_deadline_stops_slow_clients ... ok
+test extensions::tests::direct_hook_runner_isolated_environment_and_handles_failures ... ok
+test engine::sampling_tests::ds4_sampler_matches_original_reference_tokens_and_rng ... ok
+test agent::tests::execution_failures_and_timeouts_have_distinct_codes ... ok
+kill: 91003: No such process
+test agent::tests::dropping_session_tools_stops_live_jobs_and_removes_output ... ok
+test agent::tests::local_file_and_bash_tools_execute_in_the_project ... ok
+test engine::qwen_vision::qwen_image_preprocessing_matches_pillow_mtplx ... ok
+test extensions::tests::lifecycle_matchers_preserve_declared_event_order ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_shaders_are_pinned ... ok
+test extensions::tests::hook_failures_are_bounded_actionable_and_non_shell ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_mixed_distributions_match_reference ... ok
+
+test result: ok. 312 passed; 0 failed; 234 ignored; 0 measured; 0 filtered out; finished in 5.76s
+
+     Running unittests tools/qwen38-artifacts.rs (target/debug/deps/qwen38_artifacts-1288d410762a1a77)
+
+running 4 tests
+test tests::artifact_names_cannot_escape_the_output_directory ... ok
+test tests::affine_sample_digest_uses_packed_values_scales_and_biases ... ok
+test tests::safetensors_validation_rejects_duplicate_names_and_invalid_ranges ... ok
+test tests::artifact_download_resumes_an_authenticated_partial ... ok
+
+test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
+
+     Running unittests tools/test-supervisor.rs (target/debug/deps/test_supervisor-df80f419c8443a12)
+
+running 4 tests
+test supervisor::tests::direct_worker_arguments_are_not_rust_test_arguments ... ok
+test supervisor::tests::split_records_cannot_be_interleaved_with_watchdog_samples ... ok
+test supervisor::tests::split_probe_records_do_not_reset_progress ... ok
+ready{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+ready
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+progressprogressprogressprogressprogressprogressprogressprogresstest supervisor::tests::watchdog_stops_memory_and_silence_but_not_total_runtime ... ok
+
+test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.38s

metal/MLX-LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright © 2023 Apple Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

metal/mtplx-moe-sorted-body.metal

@@ -0,0 +1,199 @@
+// MTPLX 9882703, MIT; see MTPLX-LICENSE.txt.
+
+    constexpr int BM = 64;
+    constexpr int BN = 64;
+    constexpr int BK = 64;
+    constexpr int WN = 2;
+    constexpr short SM = 32;
+    constexpr short SN = 32;
+    constexpr short SK = 32;
+    constexpr short TM = SM / 16;
+    constexpr short TN = SN / 16;
+    constexpr short TK = SK / 16;
+    constexpr int PACK = get_pack_factor<BITS, 8>();
+    constexpr int PACK_BYTES = get_bytes_per_pack<BITS>();
+    constexpr int BK_PAD = BK + 16 / sizeof(T);
+    constexpr int ROW_BYTES = KD * PACK_BYTES / PACK;
+    constexpr int ROW_GROUPS = KD / GS;
+    constexpr int OUT_COLS = OUTC;
+    // SWIGLU: the tile's first 32 weight rows are gate rows, the last 32 the
+    // matching up rows, each half loaded by 64 threads from its own row range.
+    using WeightLoader = metal::conditional_t<
+        SWIGLU,
+        QuantizedBlockLoader<T, BN / 2, BK, BK_PAD, 1, 64, GS, BITS>,
+        QuantizedBlockLoader<T, BN, BK, BK_PAD, 1, 128, GS, BITS>>;
+
+    threadgroup T w_tile[BN * BK_PAD];
+
+    // The tile this threadgroup owns, and the one expert whose rows it holds:
+    // the last expert whose first tile is at or before it.
+    const int tile = int(threadgroup_position_in_grid.y);
+    if (tile >= tile_start[ED]) {
+        return;
+    }
+    int lo = 0;
+    int hi = ED - 1;
+    while (lo < hi) {
+        const int mid = (lo + hi + 1) >> 1;
+        if (tile_start[mid] <= tile) {
+            lo = mid;
+        } else {
+            hi = mid - 1;
+        }
+    }
+    const int expert = lo;
+    const int row0 = row_start[expert] + (tile - tile_start[expert]) * BM;
+    const int tile_rows = min(BM, row_start[expert + 1] - row0);
+    const int col0 = int(threadgroup_position_in_grid.x) * BN;
+
+    const ushort sg = ushort(simdgroup_index_in_threadgroup);
+    const ushort lane = ushort(thread_index_in_simdgroup);
+    const short tm = SM * short(sg / WN);
+    const short tn = SN * short(sg % WN);
+    // Rows of this tile that fall in this simdgroup's 32-row half (0 to 32).
+    const short rows = short(clamp(tile_rows - int(tm), 0, int(SM)));
+
+    ushort load_sg = sg;
+    const device uint8_t* w_src = (const device uint8_t*)w;
+    const device T* s_src = scales;
+    const device T* b_src = biases;
+    size_t w_row = size_t(expert) * ND + size_t(col0);
+    threadgroup T* w_dst = w_tile;
+    if (SWIGLU) {
+        // Output columns [col0 / 2, col0 / 2 + 32): the matching gate rows
+        // of w and up rows of wu (the same array at an offset when the
+        // weight is a fused [gate | up]).
+        const int up_half = int(load_sg / 2);
+        const size_t c = size_t(col0 / 2);
+        if (up_half) {
+            w_src = (const device uint8_t*)wu;
+            s_src = scales_u;
+            b_src = biases_u;
+            w_row = size_t(expert) * UD + size_t(UOFF) + c;
+        } else {
+            w_row = size_t(expert) * ND + c;
+        }
+        w_dst = w_tile + up_half * (BN / 2) * BK_PAD;
+        load_sg = load_sg % 2;
+    }
+    thread WeightLoader loader(
+        w_src + w_row * ROW_BYTES,
+        s_src + w_row * ROW_GROUPS,
+        b_src + w_row * ROW_GROUPS,
+        KD,
+        w_dst,
+        load_sg,
+        lane);
+
+    NAXTile<float, TM, TN> acc;
+    acc.clear();
+
+    // A lane holds rows (y, y + 8) and four consecutive columns from x of every
+    // 16x16 fragment (MLX's fragment coordinates).  Its four token rows are
+    // fixed for the whole K loop; rows past the tile read as zero, as MLX's
+    // bounded load does.
+    const short2 coord = BaseNAXFrag::get_coord();
+    const device T* lane_rows[TM][2];
+    bool lane_live[TM][2];
+    for (short i = 0; i < TM; i++) {
+        for (short r = 0; r < 2; r++) {
+            const short local = i * 16 + coord.y + r * 8;
+            lane_live[i][r] = local < rows;
+            const int token = lane_live[i][r] ? int(row_map[row0 + tm + local]) : 0;
+            lane_rows[i][r] = x + size_t(token) * KD + coord.x;
+        }
+    }
+
+    // The K loop, compiled twice: for a simdgroup whose 32 rows are all live
+    // (no bounds) and for a partial or empty half.
+    dispatch_bool(rows == SM, [&](auto full) {
+        for (int k = 0; k < KD / BK; k++) {
+            threadgroup_barrier(mem_flags::mem_threadgroup);
+            loader.load_unsafe();
+            threadgroup_barrier(mem_flags::mem_threadgroup);
+
+            STEEL_PRAGMA_NO_UNROLL
+            for (short kk = 0; kk < BK; kk += SK) {
+                if (decltype(full)::value || rows > 0) {
+                    NAXTile<T, TM, TK> a;
+                    NAXTile<T, TN, TK> b;
+                    volatile int keep_order;
+                    const int col = k * BK + kk;
+                    STEEL_PRAGMA_UNROLL
+                    for (short i = 0; i < TM; i++) {
+                        STEEL_PRAGMA_UNROLL
+                        for (short j = 0; j < TK; j++) {
+                            thread auto& frag = a.frag_at(i, j);
+                            STEEL_PRAGMA_UNROLL
+                            for (short r = 0; r < 2; r++) {
+                                const device T* src = lane_rows[i][r] + col + j * 16;
+                                STEEL_PRAGMA_UNROLL
+                                for (short c = 0; c < 4; c++) {
+                                    if constexpr (decltype(full)::value) {
+                                        frag[r * 4 + c] = src[c];
+                                    } else {
+                                        frag[r * 4 + c] = lane_live[i][r] ? src[c] : T(0);
+                                    }
+                                }
+                            }
+                        }
+                    }
+                    b.template load<T, BK_PAD, 1>(w_tile + tn * BK_PAD + kk);
+                    tile_matmad_nax(
+                        acc,
+                        a,
+                        metal::bool_constant<false>{},
+                        b,
+                        metal::bool_constant<true>{});
+                    (void)keep_order;
+                }
+            }
+            loader.next();
+        }
+    });
+
+    if constexpr (SWIGLU) {
+        // The up simdgroup of each 32-row half hands its values, rounded to T
+        // as the stock gather's output is, to the gate simdgroup of the same
+        // rows through the (now free) weight tile, lane to lane.  The gate
+        // simdgroup writes silu(gate) * up with MLX's own functors, rounding to
+        // T after each op as the stock chain does.
+        constexpr short VALS = TM * TN * 8;
+        static_assert(2 * 32 * VALS <= BN * BK_PAD, "the hand-off must fit the weight tile");
+        threadgroup T* handoff = w_tile + (sg / 2) * (32 * VALS) + lane * VALS;
+        const thread float* vals = acc.elems();
+        threadgroup_barrier(mem_flags::mem_threadgroup);
+        if (tn != 0) {
+            for (short e = 0; e < VALS; e++) {
+                handoff[e] = static_cast<T>(vals[e]);
+            }
+        }
+        threadgroup_barrier(mem_flags::mem_threadgroup);
+        if (tn == 0 && rows > 0) {
+            device T* out = y + size_t(row0 + tm) * OUT_COLS + size_t(col0 / 2);
+            for (short i = 0; i < TM; i++) {
+                for (short j = 0; j < TN; j++) {
+                    for (short r = 0; r < 2; r++) {
+                        const short row = i * 16 + coord.y + r * 8;
+                        if (row >= rows) {
+                            continue;
+                        }
+                        for (short c = 0; c < 4; c++) {
+                            const short e = (i * TN + j) * 8 + r * 4 + c;
+                            const T gate = static_cast<T>(vals[e]);
+                            const T act = Multiply()(gate, Sigmoid()(gate));
+                            out[size_t(row) * OUT_COLS + j * 16 + coord.x + c] =
+                                Multiply()(act, handoff[e]);
+                        }
+                    }
+                }
+            }
+        }
+    } else if (rows > 0) {
+        device T* out = y + size_t(row0 + tm) * ND + size_t(col0 + tn);
+        if (rows == SM) {
+            acc.store(out, ND);
+        } else {
+            acc.store_safe(out, ND, short2(SN, rows));
+        }
+    }

src/engine/metal/qwen_mtplx.rs

@@ -41,11 +41,14 @@ mod issue108;
 mod issue109;
 #[cfg(test)]
 mod issue110;
+#[cfg(test)]
+mod issue112_bench;
 mod linear;
 mod load;
 mod mlp;
 mod model;
 mod moe;
+mod moe_sorted;
 mod mtp;
 mod ngram;
 mod ngram_sidecar;
@@ -1200,7 +1203,7 @@ fn gather_bf16_broadcast(
     let lhs_strides = lhs_strides.map(|d| d as i64);
     let rhs_strides = rhs_strides.map(|d| d as i64);
 
-    gather_bf16_layout(
+    gather_layout(
         [
             tensor(0, w).input(elements[0]),
             tensor(1, s).input(elements[1]),
@@ -1210,6 +1213,7 @@ fn gather_bf16_broadcast(
             tensor(5, rhs).input(elements[5]),
             tensor(6, y).output(elements[6]),
         ],
+        array::Dtype::BF16,
         [m, n, k, experts, routes, routes],
         [bits, group],
         sorted,
@@ -1232,8 +1236,10 @@ fn gather_bf16_broadcast(
 
 /// Shared original GatherQMM dispatch; array batch strides are passed verbatim,
 /// while the two broadcast index layouts have jointly collapsed dimensions.
-fn gather_bf16_layout(
+#[allow(clippy::too_many_arguments)] // Original GatherQMM ABI and array layouts.
+fn gather_layout(
     bindings: [Binding<'_>; 7],
+    dtype: array::Dtype,
     [m, n, k, experts, routes, sorted_rows]: [u32; 6],
     [bits, group]: [u32; 2],
     sorted: bool,
@@ -1248,7 +1254,12 @@ fn gather_bf16_layout(
         rhs_strides,
     ]: [&[i64]; 6],
 ) -> Result<(), String> {
-    let suffix = format!("bfloat16_t_gs_{group}_b_{bits}");
+    let ty = match dtype {
+        array::Dtype::BF16 => "bfloat16_t",
+        array::Dtype::F16 => "float16_t",
+        _ => return Err("GatherQMM dtype unsupported".into()),
+    };
+    let suffix = format!("{ty}_gs_{group}_b_{bits}");
     if m == 1 && routes >= 16 && sorted && routes / experts >= 4 {
         let (bm, bn, bk, wm, wn) = if nax {
             (

src/engine/metal/qwen_mtplx/binding.rs

@@ -438,6 +438,7 @@ impl Bank {
         policy: model::CompilePolicy,
         fused_moe_decode: bool,
         fused_moe_verify: bool,
+        sorted_gather: bool,
     ) -> Result<model::TextModel<'_>, String> {
         let mut bind = Bind {
             bank: self,
@@ -445,6 +446,7 @@ impl Bank {
             order: Vec::new(),
             fused_moe_decode,
             fused_moe_verify,
+            sorted_gather,
         };
         let embedding = bind.linear("language_model.model.embed_tokens")?;
         let layers = self
@@ -503,6 +505,7 @@ struct Bind<'a> {
     order: Vec<String>,
     fused_moe_decode: bool,
     fused_moe_verify: bool,
+    sorted_gather: bool,
 }
 
 #[test]
@@ -566,7 +569,12 @@ fn mtplx_installed_module_binding_matches_reference() {
             "no eager parameter evaluation"
         );
         let mut model = bank
-            .bind(model::CompilePolicy::from_values(None, None), true, true)
+            .bind(
+                model::CompilePolicy::from_values(None, None),
+                true,
+                true,
+                true,
+            )
             .unwrap();
         assert_eq!(case["mask"], order["mask"]);
         assert_eq!(case["mtp"], order["mtp"]);
@@ -687,7 +695,12 @@ fn mtplx_installed_module_binding_matches_reference() {
                 .all(|(_, a)| !a.has_data())
         );
         let mut owned = BoundModel::try_new(bank, |bank| {
-            bank.bind(model::CompilePolicy::from_values(None, None), true, true)
+            bank.bind(
+                model::CompilePolicy::from_values(None, None),
+                true,
+                true,
+                true,
+            )
         })
         .unwrap();
         let module_address = std::ptr::from_ref(owned.borrow_dependent()).addr();
@@ -705,19 +718,29 @@ fn mtplx_installed_module_binding_matches_reference() {
         let extra = bank.parameters["language_model.model.layers.0.linear_attn.A_log"].clone();
         bank.parameters.insert("unexpected.weight".into(), extra);
         assert!(
-            bank.bind(model::CompilePolicy::from_values(None, None), true, true)
+            bank.bind(
-                .err()
+                model::CompilePolicy::from_values(None, None),
-                .unwrap()
+                true,
-                .contains("unbound checkpoint parameters")
+                true,
+                true
+            )
+            .err()
+            .unwrap()
+            .contains("unbound checkpoint parameters")
         );
         bank.parameters.remove("unexpected.weight");
         bank.parameters
             .remove("language_model.model.layers.0.linear_attn.norm.weight");
         assert!(
-            bank.bind(model::CompilePolicy::from_values(None, None), true, true)
+            bank.bind(
-                .err()
+                model::CompilePolicy::from_values(None, None),
-                .unwrap()
+                true,
-                .contains("missing parameter")
+                true,
+                true
+            )
+            .err()
+            .unwrap()
+            .contains("missing parameter")
         );
         eprintln!(
             "binding metadata, module recipes, all-parameter coverage, MTP norms and SSD table lifecycle exact"
@@ -938,6 +961,7 @@ impl<'a> Bind<'a> {
             normalize: c.norm_topk_prob,
             fused_decode: self.fused_moe_decode,
             fused_verify: self.fused_moe_verify,
+            sorted_gather: self.sorted_gather,
         };
         let attn_hyper = self.hyper(&format!("{p}.attn_hyper_connection"), true)?;
         let mlp_hyper = self.hyper(&format!("{p}.mlp_hyper_connection"), true)?;

src/engine/metal/qwen_mtplx/compile_graph.rs

@@ -565,10 +565,10 @@ fn equivalent_operation(a: &Operation, b: &Operation, x: &Array, y: &Array) -> b
         | (AffineQuantize { bits: a, group: b }, AffineQuantize { bits: x, group: y })
         | (AffineDequantize { bits: a, group: b }, AffineDequantize { bits: x, group: y })
         | (
-            GatherQmmBf16 {
+            GatherQmm {
                 bits: a, group: b, ..
             },
-            GatherQmmBf16 {
+            GatherQmm {
                 bits: x, group: y, ..
             },
         ) => (a, b) == (x, y),

src/engine/metal/qwen_mtplx/decoder.rs

@@ -299,6 +299,7 @@ fn mtplx_decoder_connected_attention_and_moe_advance_caches() {
             shared_gate: quantized(1, 2560),
             topk: 1,
             normalize: true,
+            sorted_gather: true,
             fused_decode: false,
             fused_verify: false,
         }

src/engine/metal/qwen_mtplx/execution.rs

@@ -313,6 +313,7 @@ impl Execution {
                 ),
                 flag("MTPLX_FUSED_MOE_DECODE"),
                 flag("MTPLX_FUSED_MOE_VERIFY"),
+                super::policy::sorted_gather_enabled(get),
             )
         })?;
         if !lazy {

src/engine/metal/qwen_mtplx/issue110.rs

@@ -44,7 +44,7 @@ fn unit_gather(rows: i32, streams: &Streams) -> Array {
         .unwrap(),
     )
     .unwrap();
-    linear::guarded_gather_qmm_bf16(
+    linear::guarded_gather_qmm(
         &x,
         [&w, &s, &b],
         [None, Some(&rhs)],
@@ -141,7 +141,7 @@ fn issue110_actual_projection_rows_match_oracle() {
             )
             .unwrap()
         });
-        let out = linear::guarded_gather_qmm_bf16(
+        let out = linear::guarded_gather_qmm(
             &x,
             [&layer[0], &layer[1], &layer[2]],
             [lhs.as_ref(), Some(&rhs)],
@@ -176,7 +176,7 @@ pub(super) fn record(rows: i32, pad: i32, k: i32, n: i32) {
 
 #[test]
 #[ignore = "full installed model and large partial chunks; serial memory supervisor required"]
-fn issue110_large_cold_and_warm_prefill() {
+pub(super) fn issue110_large_cold_and_warm_prefill() {
     assert_eq!(
         std::env::var("MTPLX_GDN_BOUNDARY_TAIL_INTERVAL").as_deref(),
         Ok("0")

src/engine/metal/qwen_mtplx/issue112_bench.rs

@@ -0,0 +1,180 @@
+//! Paired, serial installed-model prefill timings; state reads follow timing.
+use super::execution::Execution;
+use super::sampling::SamplerConfig;
+use super::session_cache::{DecodeOptions, TurnProgress};
+use std::cell::Cell;
+use std::time::Instant;
+
+#[test]
+#[ignore = "full installed model; serial memory supervisor and explicit output directory"]
+fn issue112_paired_prefill() {
+    let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
+    let directory = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE112_DIR").unwrap());
+    std::fs::create_dir_all(&directory).unwrap();
+    let repeats = std::env::var("DS4_ISSUE112_REPEATS")
+        .ok()
+        .map(|n| n.parse::<usize>().unwrap())
+        .unwrap_or(3);
+    let settings = crate::settings::effective_settings(
+        crate::model::ModelChoice::Qwen38FlashNext,
+        &crate::settings::GenerationPreferences {
+            context_tokens: 16384,
+            max_generated_tokens: 1,
+            temperature: Some(0.),
+            ..Default::default()
+        },
+        &crate::settings::RuntimePreferences::default(),
+        root.parent().unwrap(),
+    )
+    .unwrap();
+    let (mut execution, mut options) = Execution::open(&settings.engine).unwrap();
+    let sampler = SamplerConfig {
+        temperature: 0.,
+        ..Default::default()
+    };
+    options.sampler = sampler;
+    options.draft_sampler = sampler;
+    options.prefill.return_hidden = false;
+    options.decode = DecodeOptions::Ar(super::ar::Options {
+        max_tokens: 1,
+        seed: 1,
+        stop_tokens: &[],
+        sync_eval: false,
+        return_hidden: false,
+        capture_final_state: true,
+    });
+    options.store_min_suffix = None;
+    let tokens = |len: usize| {
+        (0..len)
+            .map(|i| [9707, 11, 1075, 374, 264, 1092, 13][i % 7])
+            .collect::<Vec<_>>()
+    };
+    for chunk in [2048, 4096] {
+        options.prefill.chunk_size = Some(chunk);
+        for rep in 0..repeats {
+            let mut prompt = tokens(4096);
+            let mut saved = None;
+            for phase in ["cold", "warm"] {
+                let start = Instant::now();
+                let ttft = Cell::new(None);
+                let out = execution
+                    .session_turn(
+                        &prompt,
+                        saved.as_ref(),
+                        &options,
+                        |event| {
+                            if let TurnProgress::Prefill(super::prefill::Progress::Settled {
+                                tokens,
+                                total,
+                            }) = event
+                            {
+                                eprintln!("#112 {chunk}/{rep}/{phase} {tokens}/{total}");
+                            }
+                            Ok(())
+                        },
+                        |_| {
+                            if ttft.get().is_none() {
+                                ttft.set(Some(start.elapsed().as_secs_f64()));
+                            }
+                            Ok(())
+                        },
+                        |_| Ok(()),
+                    )
+                    .unwrap();
+                let elapsed = start.elapsed().as_secs_f64();
+                assert!(out.final_snapshot_error.is_none());
+                let snapshot = out.snapshot.unwrap();
+                let state = super::issue104::state(&execution, &snapshot);
+                let hidden_state=snapshot.hidden.as_ref().map(|a| {
+                    let values=execution.values_f32(a).unwrap();
+                    assert!(values.iter().all(|v|v.is_finite()));
+                    serde_json::json!({"shape":a.layout().shape(),"sha256":super::super::qwen_mtplx_tests::hex_digest(&values.iter().flat_map(|v|v.to_le_bytes()).collect::<Vec<_>>())})
+                });
+                assert!(hidden_state.is_none());
+                let metrics = serde_json::json!({"chunk":chunk,"rep":rep,"phase":phase,"prompt":prompt.len(),"cached":out.cached_tokens,"ttft_seconds":ttft.get().unwrap(),"prefill_seconds":out.prefill_seconds,"prefill_tokens_per_second":(prompt.len()-out.cached_tokens) as f64/out.prefill_seconds,"elapsed_seconds":elapsed});
+                let record =
+                    serde_json::json!({"metrics":metrics,"state":state,"hidden":hidden_state});
+                std::fs::write(
+                    directory.join(format!("{chunk}-{rep}-{phase}.json")),
+                    serde_json::to_vec(&record).unwrap(),
+                )
+                .unwrap();
+                println!(
+                    "{}",
+                    serde_json::json!({"case":"paired_prefill","metrics":metrics})
+                );
+                prompt = snapshot.token_ids.clone();
+                prompt.extend(tokens(2048));
+                saved = Some(snapshot);
+            }
+        }
+    }
+}
+
+#[test]
+#[ignore = "full installed model hidden-state prefill; serial supervisor required"]
+fn issue112_hidden_prefill() {
+    let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
+    let directory = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE112_DIR").unwrap());
+    std::fs::create_dir_all(&directory).unwrap();
+    let mut settings = crate::settings::effective_settings(
+        crate::model::ModelChoice::Qwen38FlashNext,
+        &crate::settings::GenerationPreferences {
+            context_tokens: 16384,
+            max_generated_tokens: 1,
+            ..Default::default()
+        },
+        &crate::settings::RuntimePreferences::default(),
+        root.parent().unwrap(),
+    )
+    .unwrap();
+    settings.engine.speculative.glm_mtp = true;
+    let (mut execution, mut options) = Execution::open(&settings.engine).unwrap();
+    options.decode = DecodeOptions::Ar(super::ar::Options {
+        max_tokens: 1,
+        seed: 1,
+        stop_tokens: &[],
+        sync_eval: false,
+        return_hidden: true,
+        capture_final_state: true,
+    });
+    options.prefill.return_hidden = true;
+    options.store_min_suffix = None;
+    let tokens = |len: usize| {
+        (0..len)
+            .map(|i| [9707, 11, 1075, 374, 264, 1092, 13][i % 7])
+            .collect::<Vec<_>>()
+    };
+    for chunk in [2048, 4096] {
+        options.prefill.chunk_size = Some(chunk);
+        let mut saved = None;
+        let mut prompt = tokens(4096);
+        for phase in ["cold", "warm"] {
+            let snapshot = execution
+                .prefill_session(&prompt, saved.as_ref(), &options, |event| {
+                    if let super::prefill::Progress::Settled { tokens, total } = event {
+                        eprintln!("#112 hidden {chunk}/{phase} {tokens}/{total}");
+                    }
+                    Ok(())
+                })
+                .unwrap();
+            let state = super::issue104::state(&execution, &snapshot);
+            let hidden = snapshot.hidden.as_ref().unwrap();
+            let values = execution.values_f32(hidden).unwrap();
+            assert!(values.iter().all(|v| v.is_finite()));
+            let hidden = serde_json::json!({"shape":hidden.layout().shape(),"sha256":super::super::qwen_mtplx_tests::hex_digest(&values.iter().flat_map(|v|v.to_le_bytes()).collect::<Vec<_>>())});
+            let record = serde_json::json!({"case":"hidden_prefill","chunk":chunk,"phase":phase,"cached":saved.as_ref().map_or(0,|s|s.token_ids.len()),"state":state,"hidden":hidden});
+            std::fs::write(
+                directory.join(format!("{chunk}-{phase}.json")),
+                serde_json::to_vec(&record).unwrap(),
+            )
+            .unwrap();
+            println!(
+                "{}",
+                serde_json::json!({"case":"hidden_prefill","chunk":chunk,"phase":phase,"hidden":hidden})
+            );
+            prompt.extend(tokens(2048));
+            saved = Some(snapshot);
+        }
+    }
+}

src/engine/metal/qwen_mtplx/linear.rs

@@ -371,7 +371,7 @@ fn sorted_gather_guard_matches_pinned_call_predicates() {
 
 /// Guard both stock Gate/Up and Down gathers, including future fused-path
 /// fallbacks. Keep the underlying runtime primitive unchanged.
-pub(super) fn guarded_gather_qmm_bf16(
+pub(super) fn guarded_gather_qmm(
     x: &Array,
     weights: [&Array; 3],
     indices: [Option<&Array>; 2],
@@ -390,7 +390,7 @@ pub(super) fn guarded_gather_qmm_bf16(
         super::sparse_attention::nax_available(),
     );
     if pad == 0 {
-        return gather_qmm_bf16(x, weights, indices, quant, sorted, streams, stream);
+        return gather_qmm(x, weights, indices, quant, sorted, streams, stream);
     }
     let rows = shape[0];
     #[cfg(test)]
@@ -428,7 +428,7 @@ pub(super) fn guarded_gather_qmm_bf16(
             ops::concatenate(&[lhs.clone(), tail], 0, stream)
         })
         .transpose()?;
-    let out = gather_qmm_bf16(
+    let out = gather_qmm(
         &x,
         weights,
         [lhs.as_ref(), Some(&rhs)],
@@ -448,9 +448,9 @@ pub(super) fn guarded_gather_qmm_bf16(
     )
 }
 
-/// Installed affine/BF16, transposed GatherQMM from ops.cpp. Explicit index
+/// Affine BF16/F16 transposed GatherQMM from ops.cpp. Explicit index
 /// casts use the default stream, as indices_or_default does in the reference.
-pub(super) fn gather_qmm_bf16(
+pub(super) fn gather_qmm(
     x: &Array,
     [w, scales, biases]: [&Array; 3],
     indices: [Option<&Array>; 2],
@@ -466,17 +466,19 @@ pub(super) fn gather_qmm_bf16(
     let wl = w.layout();
     let sl = scales.layout();
     if stream.device() != Device::Gpu
-        || xl.dtype() != Dtype::BF16
+        || !matches!(xl.dtype(), Dtype::BF16 | Dtype::F16)
         || wl.dtype() != Dtype::U32
-        || sl.dtype() != Dtype::BF16
+        || sl.dtype() != xl.dtype()
-        || biases.layout().dtype() != Dtype::BF16
+        || biases.layout().dtype() != xl.dtype()
         || xl.shape().len() < 2
         || wl.shape().len() < 2
         || wl.shape().len() != sl.shape().len()
         || !matches!(bits, 4 | 8)
-        || !matches!(group, 32 | 64)
+        || !matches!(group, 32 | 64 | 128)
     {
-        return Err("GatherQMM requires BF16 GPU matrices and U32 affine expert weights".into());
+        return Err(
+            "GatherQMM requires BF16/F16 GPU matrices and U32 affine expert weights".into(),
+        );
     }
     let wrank = wl.shape().len();
     let k = *xl.shape().last().unwrap() as u32;
@@ -491,7 +493,9 @@ pub(super) fn gather_qmm_bf16(
     {
         return Err("GatherQMM input/weight/group shapes do not match".into());
     }
-    affine_plan([m, n, k], bits, group, true)?;
+    if [m, n, k].iter().any(|&d| d == 0 || d > i32::MAX as u32) || !k.is_multiple_of(group) {
+        return Err("invalid GatherQMM shape".into());
+    }
     let index = |a: &Array, indices: Option<&Array>| -> Result<Array, String> {
         if let Some(indices) = indices {
             if !matches!(
@@ -530,7 +534,7 @@ pub(super) fn gather_qmm_bf16(
     )?);
     let mut shape = inputs[4].layout().shape().to_vec();
     shape.extend([m as i32, n as i32]);
-    let layout = Layout::new(&shape, Dtype::BF16)?;
+    let layout = Layout::new(&shape, xl.dtype())?;
     if layout.size() > i32::MAX as usize {
         return Err("GatherQMM output size overflow".into());
     }
@@ -538,7 +542,7 @@ pub(super) fn gather_qmm_bf16(
         stream,
         &inputs,
         layout,
-        Operation::GatherQmmBf16 {
+        Operation::GatherQmm {
             bits,
             group,
             right_sorted: sorted && indices[0].is_none(),
@@ -575,7 +579,7 @@ pub(super) fn gather_evaluate(
         if prepared[0].layout().size() / k as usize != prepared[5].layout().size() {
             let mut shape = prepared[5].layout().shape().to_vec();
             shape.extend([m as i32, k as i32]);
-            let broadcast = Array::unallocated(&shape, Dtype::BF16)?;
+            let broadcast = Array::unallocated(&shape, prepared[0].layout().dtype())?;
             views::broadcast(&prepared[0], &broadcast)?;
             prepared[0] = broadcast;
         }
@@ -596,7 +600,7 @@ pub(super) fn gather_evaluate(
         views::collapse_joint(ll.shape(), &[ll.strides(), rl.strides()]);
     let arrays = [w, s, b, x, lhs, rhs, output];
     let buffers = arrays.map(Array::buffer);
-    super::gather_bf16_layout(
+    super::gather_layout(
         std::array::from_fn(|i| {
             let binding = super::tensor(i as u32, &buffers[i]).at_byte_offset(arrays[i].offset());
             if i == 6 {
@@ -605,6 +609,7 @@ pub(super) fn gather_evaluate(
                 binding.input(arrays[i].data_size())
             }
         }),
+        xl.dtype(),
         [m, n, k, experts, batches, sorted_rows],
         [bits, group],
         right_sorted,
@@ -1058,7 +1063,7 @@ fn mtplx_graph_gather_qmm_matches_reference() {
                     .unwrap();
                 }
                 let (output, construction) = capture_dispatches(|| {
-                    gather_qmm_bf16(
+                    gather_qmm(
                         &x,
                         [&w, &s, &b],
                         [left.then_some(&lhs), right.then_some(&rhs)],

src/engine/metal/qwen_mtplx/model.rs

@@ -721,7 +721,7 @@ impl TextModel<'_> {
                         O::RmsNorm(_) => "RMSNorm",
                         O::Softmax(_) => "Softmax",
                         O::QuantizedLinearBf16 { .. } => "QuantizedMatmul",
-                        O::GatherQmmBf16 { .. } => "GatherQMM",
+                        O::GatherQmm { .. } => "GatherQMM",
                         O::DenseMatmul => "Matmul",
                         O::Sum(_) => "Sum",
                         O::Sort {

src/engine/metal/qwen_mtplx/moe.rs

@@ -27,7 +27,7 @@ fn gather(
     streams: &Streams,
     stream: Stream,
 ) -> Result<Array, String> {
-    linear::guarded_gather_qmm_bf16(
+    linear::guarded_gather_qmm(
         input,
         [&layer.weight, &layer.scales, &layer.biases],
         [None, Some(ids)],
@@ -55,13 +55,19 @@ impl FeedForward {
         &self,
         input: &Array,
         indices: &Array,
+        sorted_gather: bool,
         streams: &Streams,
         stream: Stream,
     ) -> Result<Array, String> {
-        let mut x = ops::expand_dims(input, &[-2, -3], stream)?;
         let do_sort = indices.layout().size() >= 64;
+        let mut x = if do_sort {
+            input.clone()
+        } else {
+            ops::expand_dims(input, &[-2, -3], stream)?
+        };
         let mut ids = indices.clone();
         let mut inverse = None;
+        let mut row_map = None;
         if do_sort {
             let size =
                 i32::try_from(indices.layout().size()).map_err(|_| "MoE index size overflow")?;
@@ -76,33 +82,46 @@ impl FeedForward {
             let k = *input.layout().shape().last().ok_or("MoE input rank")?;
             let flat_rows = i32::try_from(input.layout().size() / k as usize)
                 .map_err(|_| "MoE row count overflow")?;
-            x = indexing::take(
+            x = ops::reshape(&x, &[flat_rows, 1, k], stream)?;
-                &ops::reshape(&x, &[flat_rows, 1, k], stream)?,
+            row_map = Some(rows);
-                &rows,
-                0,
-                stream,
-            )?;
             ids = indexing::take(&ids, &order, 0, stream)?;
         }
-        let hidden = match &self.input {
+        let hidden = if let Some(map) = row_map {
-            GateUp::Fused(gu) => {
+            let (gate, up) = match &self.input {
-                let gu = gather(&x, gu, &ids, do_sort, streams, stream)?;
+                GateUp::Fused(gu) => (gu, None),
-                let n = *gu.layout().shape().last().ok_or("MoE gate/up rank")?;
+                GateUp::Separate { gate, up } => (gate, Some(up)),
-                if n % 2 != 0 {
+            };
-                    return Err("MoE fused gate/up width must be even".into());
+            super::moe_sorted::swiglu_rows(
+                &x,
+                &map,
+                gate,
+                up,
+                &ids,
+                sorted_gather,
+                streams,
+                stream,
+            )?
+        } else {
+            match &self.input {
+                GateUp::Fused(gu) => {
+                    let gu = gather(&x, gu, &ids, do_sort, streams, stream)?;
+                    let n = *gu.layout().shape().last().ok_or("MoE gate/up rank")?;
+                    if n % 2 != 0 {
+                        return Err("MoE fused gate/up width must be even".into());
+                    }
+                    let pair = ops::split(&gu, &[n / 2], -1, stream)?;
+                    binary(
+                        &mlp::silu(&pair[0], stream)?,
+                        &pair[1],
+                        Binary::Multiply,
+                        stream,
+                    )?
+                }
+                GateUp::Separate { gate, up } => {
+                    let up = gather(&x, up, &ids, do_sort, streams, stream)?;
+                    let gate = gather(&x, gate, &ids, do_sort, streams, stream)?;
+                    mlp::swiglu(&gate, &up, stream)?
                 }
-                let pair = ops::split(&gu, &[n / 2], -1, stream)?;
-                binary(
-                    &mlp::silu(&pair[0], stream)?,
-                    &pair[1],
-                    Binary::Multiply,
-                    stream,
-                )?
-            }
-            GateUp::Separate { gate, up } => {
-                let up = gather(&x, up, &ids, do_sort, streams, stream)?;
-                let gate = gather(&x, gate, &ids, do_sort, streams, stream)?;
-                mlp::swiglu(&gate, &up, stream)?
             }
         };
         let mut output = gather(&hidden, &self.down, &ids, do_sort, streams, stream)?;
@@ -123,6 +142,7 @@ pub(super) struct SparseMoe {
     pub(super) shared_gate: QuantizedLinear,
     pub(super) topk: i32,
     pub(super) normalize: bool,
+    pub(super) sorted_gather: bool,
     pub(super) fused_decode: bool,
     pub(super) fused_verify: bool,
 }
@@ -199,7 +219,9 @@ impl SparseMoe {
                 stream,
             )?
         } else {
-            let switched = self.experts.switched(input, &indices, streams, stream)?;
+            let switched =
+                self.experts
+                    .switched(input, &indices, self.sorted_gather, streams, stream)?;
             let scores = ops::expand_dims(&scores, &[-1], stream)?;
             reduce::sum(
                 &binary(&switched, &scores, Binary::Multiply, stream)?,
@@ -413,6 +435,7 @@ fn mtplx_graph_sparse_moe_matches_reference() {
                 shared_gate: mlp::fixture_linear(1, k, 4, 64, 52),
                 topk: 10,
                 normalize: row["normalize"].as_bool().unwrap(),
+                sorted_gather: true,
                 fused_decode: row["fused"].as_bool().unwrap_or(false),
                 fused_verify: row["fused"].as_bool().unwrap_or(false),
             };

src/engine/metal/qwen_mtplx/moe_sorted.rs

@@ -0,0 +1,839 @@
+//! MTPLX 9882703 expert row-map gather, including whole-first-call canaries.
+use super::array::{Array, Dtype, Layout};
+use super::binary::{Binary, binary, floor_divide_integer};
+use super::mlp::{self, QuantizedLinear};
+use super::stream::{Stream, Streams};
+use super::{indexing, linear, ops};
+use std::collections::BTreeMap;
+use std::sync::{LazyLock, Mutex};
+
+#[derive(Default, Clone, Copy, Debug)]
+pub(super) struct Stats {
+    pub(super) calls: u64,
+    pub(super) fallbacks: u64,
+    pub(super) canaries: u64,
+    pub(super) failures: u64,
+}
+static CANARY: LazyLock<Mutex<(BTreeMap<String, bool>, Stats)>> =
+    LazyLock::new(|| Mutex::new((BTreeMap::new(), Stats::default())));
+
+#[cfg(test)]
+pub(super) fn stats() -> Stats {
+    CANARY.lock().unwrap().1
+}
+
+#[cfg(test)]
+static FAILURE: std::sync::atomic::AtomicU8 = std::sync::atomic::AtomicU8::new(0);
+
+#[derive(Clone, Copy, Debug)]
+pub(super) struct Kernel {
+    dtype: Dtype,
+    bits: u32,
+    group: u32,
+    k: i32,
+    n: i32,
+    experts: i32,
+    swiglu: bool,
+    separate: bool,
+}
+
+impl Kernel {
+    fn width(self) -> i32 {
+        if self.swiglu && !self.separate {
+            self.n / 2
+        } else {
+            self.n
+        }
+    }
+    fn name(self) -> String {
+        format!(
+            "kernel_qwen_mtplx_sorted_{}_g{}_b{}_k{}_n{}_e{}_s{}_u{}",
+            self.dtype.kernel_name(),
+            self.group,
+            self.bits,
+            self.k,
+            self.n,
+            self.experts,
+            u8::from(self.swiglu),
+            u8::from(self.separate)
+        )
+    }
+    fn source(self) -> String {
+        let ty = if self.dtype == Dtype::BF16 {
+            "bfloat16_t"
+        } else {
+            "float16_t"
+        };
+        let params = [
+            ("x", ty),
+            ("w", "uint"),
+            ("scales", ty),
+            ("biases", ty),
+            ("wu", "uint"),
+            ("scales_u", ty),
+            ("biases_u", ty),
+            ("tile_start", "int"),
+            ("row_start", "int"),
+            ("row_map", "uint"),
+        ]
+        .iter()
+        .enumerate()
+        .map(|(i, (name, ty))| format!("const device {ty}* {name} [[buffer({i})]]"))
+        .collect::<Vec<_>>()
+        .join(",\n");
+        format!(
+            "{}\nusing T={ty};\nconstant int GS={}, BITS={}, KD={}, ND={}, UD={}, UOFF={}, OUTC={}, ED={};\nconstant bool SWIGLU={};\nkernel void {}({params}, device T* y [[buffer(10)]], uint3 threadgroup_position_in_grid [[threadgroup_position_in_grid]], uint simdgroup_index_in_threadgroup [[simdgroup_index_in_threadgroup]], uint thread_index_in_simdgroup [[thread_index_in_simdgroup]]) {{\n{}\n}}",
+            include_str!("../../../../metal/mtplx-moe-sorted-header.metal"),
+            self.group,
+            self.bits,
+            self.k,
+            self.n,
+            self.n,
+            if self.swiglu && !self.separate {
+                self.n / 2
+            } else {
+                0
+            },
+            self.width(),
+            self.experts,
+            self.swiglu,
+            self.name(),
+            include_str!("../../../../metal/mtplx-moe-sorted-body.metal")
+        )
+    }
+    pub(super) fn evaluate(self, inputs: &[Array], outputs: &[Array]) -> Result<(), String> {
+        super::compile_custom(&self.name(), || {
+            #[cfg(test)]
+            if FAILURE.load(std::sync::atomic::Ordering::Relaxed) == 1 {
+                return Ok("injected invalid Metal source".into());
+            }
+            Ok(self.source())
+        })?;
+        let rows = outputs[0].layout().shape()[0] as u32;
+        let cols = if self.swiglu {
+            self.width() * 2
+        } else {
+            self.n
+        } as u32;
+        super::gdn::custom_evaluate(inputs, outputs, |bindings| {
+            super::dispatch_geometry(
+                &self.name(),
+                bindings,
+                &[],
+                [
+                    (cols / 64) * 128,
+                    rows.div_ceil(64) + self.experts as u32,
+                    1,
+                ],
+                [128, 1, 1],
+                true,
+            )
+        })
+    }
+}
+
+// Applies mirrors the affine tensor/layout predicates. QuantizedLinear always
+// carries biases and affine mode; separate packs must also share quantization.
+pub(super) fn applies(
+    tokens: &Array,
+    map: &Array,
+    gate: &QuantizedLinear,
+    up: Option<&QuantizedLinear>,
+    ids: &Array,
+    enabled: bool,
+    nax: bool,
+) -> bool {
+    if let Some(up) = up
+        && (gate.bits != up.bits
+            || gate.group != up.group
+            || [&gate.weight, &gate.scales, &gate.biases]
+                .iter()
+                .zip([&up.weight, &up.scales, &up.biases])
+                .any(|(a, b)| {
+                    a.layout().shape() != b.layout().shape()
+                        || a.layout().dtype() != b.layout().dtype()
+                }))
+    {
+        return false;
+    }
+    let x = tokens.layout();
+    let w = gate.weight.layout();
+    let s = gate.scales.layout();
+    let b = gate.biases.layout();
+    let r = map.layout();
+    let i = ids.layout();
+    enabled
+        && nax
+        && matches!(gate.bits, 4 | 8)
+        && matches!(gate.group, 32 | 64 | 128)
+        && matches!(x.dtype(), Dtype::BF16 | Dtype::F16)
+        && x.shape().len() == 3
+        && x.shape()[1] == 1
+        && s.dtype() == x.dtype()
+        && b.dtype() == x.dtype()
+        && i.shape().len() == 1
+        && r.shape() == i.shape()
+        && i.size() >= 4096
+        && r.dtype() == Dtype::U32
+        && w.shape().len() == 3
+        && w.dtype() == Dtype::U32
+        && w.shape()[1] % 64 == 0
+        && (w.shape()[2] * 32 / gate.bits as i32) % 64 == 0
+        && x.shape()[2] == w.shape()[2] * 32 / gate.bits as i32
+        && x.shape()[2] % gate.group as i32 == 0
+}
+
+fn scalar(n: i32) -> Result<Array, String> {
+    Array::new(&[], Dtype::I32, super::scalar_buffer(&n.to_le_bytes())?)
+}
+fn schedule(ids: &Array, experts: i32, stream: Stream) -> Result<(Array, Array), String> {
+    if !matches!(
+        ids.layout().dtype(),
+        Dtype::U8
+            | Dtype::U16
+            | Dtype::U32
+            | Dtype::U64
+            | Dtype::I8
+            | Dtype::I16
+            | Dtype::I32
+            | Dtype::I64
+    ) {
+        return Err("MoE expert indices must be integral".into());
+    }
+    let ids = ops::contiguous(&ops::astype(ids, Dtype::U32, false, stream)?, false, stream)?;
+    let zero = ops::zeros(&[experts], Dtype::I32, stream)?;
+    let counts = Array::make_operation_with_inputs(
+        stream,
+        &[zero, ids.clone()],
+        Layout::new(&[experts], Dtype::I32)?,
+        ops::Operation::MoeSortedCounts,
+    )?;
+    let row_start = ops::concatenate(
+        &[
+            ops::zeros(&[1], Dtype::I32, stream)?,
+            indexing::cumsum_last_i32(&counts, stream)?,
+        ],
+        0,
+        stream,
+    )?;
+    let tiles = floor_divide_integer(
+        &binary(&counts, &scalar(63)?, Binary::Add, stream)?,
+        &scalar(64)?,
+        stream,
+    )?;
+    let tile_start = ops::concatenate(
+        &[
+            ops::zeros(&[1], Dtype::I32, stream)?,
+            indexing::cumsum_last_i32(&tiles, stream)?,
+        ],
+        0,
+        stream,
+    )?;
+    Ok((tile_start, row_start))
+}
+
+pub(super) fn counts_evaluate(inputs: &[Array], output: &Array) -> Result<(), String> {
+    // Same zeros.at[ids].add(1) schedule, with integer atomic sums on the GPU.
+    // The pinned library does not export this integer scatter specialization.
+    const NAME: &str = "kernel_qwen_mtplx_sorted_counts";
+    super::compile_custom(NAME, || {
+        Ok("#include <metal_stdlib>\nusing namespace metal;\nkernel void kernel_qwen_mtplx_sorted_counts(const device uint* ids [[buffer(0)]], device atomic_int* counts [[buffer(1)]], constant uint& size [[buffer(2)]],uint i [[thread_position_in_grid]]) { if(i<size) atomic_fetch_add_explicit(counts+ids[i],1,memory_order_relaxed); }".into())
+    })?;
+    super::views::copy_gpu(&inputs[0], output, true)?;
+    let ids = &inputs[1];
+    let n = ids.layout().size() as u32;
+    super::dispatch_geometry(
+        NAME,
+        &[
+            super::tensor(0, &ids.buffer())
+                .at_byte_offset(ids.offset())
+                .input(ids.data_size()),
+            super::tensor(1, &output.buffer()).output(output.data_size()),
+            super::bytes(2, &n),
+        ],
+        &[],
+        [n, 1, 1],
+        [n.min(1024), 1, 1],
+        true,
+    )
+}
+
+#[allow(clippy::too_many_arguments)]
+fn launch(
+    tokens: &Array,
+    map: &Array,
+    gate: &QuantizedLinear,
+    up: Option<&QuantizedLinear>,
+    ids: &Array,
+    kernel: Kernel,
+    stream: Stream,
+) -> Result<Array, String> {
+    let (tiles, rows) = schedule(ids, kernel.experts, stream)?;
+    let u = up.unwrap_or(gate);
+    let input = ops::reshape(tokens, &[-1, kernel.k], stream)?;
+    let y = Array::make_operation_with_inputs(
+        stream,
+        &[
+            input,
+            gate.weight.clone(),
+            gate.scales.clone(),
+            gate.biases.clone(),
+            u.weight.clone(),
+            u.scales.clone(),
+            u.biases.clone(),
+            tiles,
+            rows,
+            map.clone(),
+        ],
+        Layout::new(&[ids.layout().shape()[0], kernel.width()], kernel.dtype)?,
+        ops::Operation::MoeSortedGather(kernel),
+    )?;
+    ops::reshape(&y, &[ids.layout().shape()[0], 1, kernel.width()], stream)
+}
+
+fn gather(
+    x: &Array,
+    q: &QuantizedLinear,
+    ids: &Array,
+    streams: &Streams,
+    stream: Stream,
+) -> Result<Array, String> {
+    linear::guarded_gather_qmm(
+        x,
+        [&q.weight, &q.scales, &q.biases],
+        [None, Some(ids)],
+        [q.bits, q.group],
+        true,
+        streams,
+        stream,
+    )
+}
+
+fn activation(gate: &Array, up: &Array, separate: bool, stream: Stream) -> Result<Array, String> {
+    if gate.layout().dtype() == Dtype::BF16 {
+        return if separate {
+            mlp::swiglu(gate, up, stream)
+        } else {
+            binary(&mlp::silu(gate, stream)?, up, Binary::Multiply, stream)
+        };
+    }
+    let graph = super::compiled::Replay::trace_compiled(&[gate.clone(), up.clone()], |a| {
+        let sig = super::unary::unary(&a[0], super::unary::Unary::Sigmoid, stream)?;
+        let silu = binary(&a[0], &sig, Binary::Multiply, stream)?;
+        Ok(vec![if separate {
+            binary(&silu, &a[1], Binary::Multiply, stream)?
+        } else {
+            silu
+        }])
+    })?;
+    let result = graph.run(&[gate.clone(), up.clone()])?.remove(0);
+    if separate {
+        Ok(result)
+    } else {
+        binary(&result, up, Binary::Multiply, stream)
+    }
+}
+
+#[allow(clippy::too_many_arguments)]
+fn stock(
+    tokens: &Array,
+    map: &Array,
+    gate: &QuantizedLinear,
+    up: Option<&QuantizedLinear>,
+    ids: &Array,
+    swiglu: bool,
+    streams: &Streams,
+    stream: Stream,
+) -> Result<Array, String> {
+    let x = indexing::take(tokens, map, 0, stream)?;
+    let y = gather(&x, gate, ids, streams, stream)?;
+    if !swiglu {
+        return Ok(y);
+    }
+    if let Some(up) = up {
+        activation(&y, &gather(&x, up, ids, streams, stream)?, true, stream)
+    } else {
+        let pair = ops::split(&y, &[y.layout().dim(-1)? / 2], -1, stream)?;
+        activation(&pair[0], &pair[1], false, stream)
+    }
+}
+
+#[allow(clippy::too_many_arguments)]
+fn checked(
+    tokens: &Array,
+    map: &Array,
+    gate: &QuantizedLinear,
+    up: Option<&QuantizedLinear>,
+    ids: &Array,
+    swiglu: bool,
+    streams: &Streams,
+    stream: Stream,
+) -> Result<Option<Array>, String> {
+    let kernel = Kernel {
+        dtype: tokens.layout().dtype(),
+        bits: gate.bits,
+        group: gate.group,
+        k: tokens.layout().dim(-1)?,
+        n: gate.weight.layout().dim(-2)?,
+        experts: gate.weight.layout().shape()[0],
+        swiglu,
+        separate: up.is_some(),
+    };
+    let key = format!(
+        "{}_{}",
+        kernel.name(),
+        if ids.layout().size() >= 64 * kernel.experts as usize {
+            "wide"
+        } else {
+            "narrow"
+        }
+    );
+    let mut state = CANARY.lock().map_err(|_| "MoE canary lock poisoned")?;
+    if state.0.get(&key) == Some(&false) {
+        state.1.fallbacks += 1;
+        return Ok(None);
+    }
+    if state.0.get(&key) == Some(&true) {
+        state.1.calls += 1;
+        drop(state);
+        return launch(tokens, map, gate, up, ids, kernel, stream).map(Some);
+    }
+    state.1.canaries += 1;
+    let probe: Result<Array, String> = (|| {
+        let ours = launch(tokens, map, gate, up, ids, kernel, stream)?;
+        let reference = stock(tokens, map, gate, up, ids, swiglu, streams, stream)?;
+        ops::evaluate(streams, &[ours.clone(), reference.clone()], stream, false)?;
+        #[cfg(test)]
+        if FAILURE.load(std::sync::atomic::Ordering::Relaxed) == 2 {
+            let mut bits = [0u8; 2];
+            ours.buffer().read(ours.offset(), &mut bits)?;
+            bits[1] ^= 0x80;
+            ours.buffer().write(ours.offset(), &bits)?;
+        }
+        #[cfg(test)]
+        if FAILURE.load(std::sync::atomic::Ordering::Relaxed) == 3 {
+            ours.buffer().write(ours.offset(), &[0, 128])?;
+            reference.buffer().write(reference.offset(), &[0, 0])?;
+        }
+        let equal = binary(
+            &ops::view(&ours, Dtype::U16, stream)?,
+            &ops::view(&reference, Dtype::U16, stream)?,
+            Binary::Equal,
+            stream,
+        )?;
+        let different = super::unary::unary(&equal, super::unary::Unary::LogicalNot, stream)?;
+        let mismatch = super::reduce::any(&different, &[0, 1, 2], false, stream)?;
+        ops::evaluate(streams, std::slice::from_ref(&mismatch), stream, false)?;
+        let mut failed = [0u8];
+        mismatch.buffer().read(mismatch.offset(), &mut failed)?;
+        if failed[0] != 0 {
+            return Err(
+                "first whole-call output differs bit for bit from the stock sorted gather".into(),
+            );
+        }
+        Ok(ours)
+    })();
+    match probe {
+        Ok(ours) => {
+            state.0.insert(key.clone(), true);
+            state.1.calls += 1;
+            eprintln!(
+                "[moe-sorted-gather] tensor-unit kernel on for {key}: first call equals stock bit for bit"
+            );
+            Ok(Some(ours))
+        }
+        Err(reason) => {
+            state.0.insert(key.clone(), false);
+            state.1.failures += 1;
+            state.1.fallbacks += 1;
+            eprintln!(
+                "[moe-sorted-gather] tensor-unit kernel off for {key}: {reason}; stock sorted gather runs instead (canary_failures={})",
+                state.1.failures
+            );
+            Ok(None)
+        }
+    }
+}
+
+#[allow(clippy::too_many_arguments)]
+pub(super) fn swiglu_rows(
+    tokens: &Array,
+    map: &Array,
+    gate: &QuantizedLinear,
+    up: Option<&QuantizedLinear>,
+    ids: &Array,
+    enabled: bool,
+    streams: &Streams,
+    stream: Stream,
+) -> Result<Array, String> {
+    let eligible = applies(
+        tokens,
+        map,
+        gate,
+        up,
+        ids,
+        enabled,
+        super::sparse_attention::nax_available(),
+    );
+    if eligible && let Some(y) = checked(tokens, map, gate, up, ids, true, streams, stream)? {
+        return Ok(y);
+    }
+    // The fused-weight wrapper retries its plain row-map gather if the SwiGLU
+    // canary failed. Separate SwitchGLU instead returns to its original chain.
+    if eligible
+        && up.is_none()
+        && let Some(gu) = checked(tokens, map, gate, None, ids, false, streams, stream)?
+    {
+        let pair = ops::split(&gu, &[gu.layout().dim(-1)? / 2], -1, stream)?;
+        return activation(&pair[0], &pair[1], false, stream);
+    }
+    if let Some(up) = up {
+        let x = indexing::take(tokens, map, 0, stream)?;
+        let up = gather(&x, up, ids, streams, stream)?;
+        let gate = gather(&x, gate, ids, streams, stream)?;
+        return activation(&gate, &up, true, stream);
+    }
+    stock(tokens, map, gate, None, ids, true, streams, stream)
+}
+
+#[test]
+#[ignore = "Metal synthetic all supported row-map variants; external supervisor"]
+fn issue112_all_kernel_variants() {
+    use super::super::{
+        gpu::Context,
+        qwen_mtplx_tests::{hex_digest, pattern},
+    };
+    super::configure_sources().unwrap();
+    let _context = Context::open_qwen(0).unwrap();
+    let streams = Streams::new(Some(super::allocator::Allocator::new().unwrap()));
+    let gpu = streams.default_stream(super::stream::Device::Gpu).unwrap();
+    let e = 128;
+    let n = 256;
+    let k = 256;
+    let tokens = 512;
+    let read = |a: &Array| {
+        ops::evaluate(&streams, std::slice::from_ref(a), gpu, false).unwrap();
+        let mut v = vec![0; a.layout().nbytes()];
+        a.buffer().read(a.offset(), &mut v).unwrap();
+        v
+    };
+    let u32s = |v: Vec<u32>| {
+        Array::new(
+            &[v.len() as i32],
+            Dtype::U32,
+            super::scalar_buffer(&v.into_iter().flat_map(u32::to_le_bytes).collect::<Vec<_>>())
+                .unwrap(),
+        )
+        .unwrap()
+    };
+    for dtype in [Dtype::BF16, Dtype::F16] {
+        for bits in [4, 8] {
+            for group in [32, 64, 128] {
+                let pack = |salt| {
+                    let mut q = mlp::fixture_linear(e * n, k, bits, group, salt);
+                    for a in [&mut q.weight, &mut q.scales, &mut q.biases] {
+                        let width = a.layout().dim(-1).unwrap();
+                        *a = ops::reshape(a, &[e as i32, n as i32, width], gpu).unwrap();
+                        if a.layout().dtype() == Dtype::BF16 {
+                            *a = ops::astype(a, dtype, false, gpu).unwrap();
+                        }
+                    }
+                    q
+                };
+                let gate = pack(7);
+                let up = pack(9);
+                let x = ops::astype(
+                    &Array::new(
+                        &[tokens, 1, k as i32],
+                        Dtype::BF16,
+                        pattern(tokens as u32 * k, 8),
+                    )
+                    .unwrap(),
+                    dtype,
+                    false,
+                    gpu,
+                )
+                .unwrap();
+                for separate in [false, true] {
+                    for rows in [4096, 8193, 32769] {
+                        eprintln!(
+                            "row-map {dtype:?} b{bits}/g{group} separate={separate} rows={rows}"
+                        );
+                        let ids = u32s(
+                            (0..rows)
+                                .map(|r| (r * (e - 3) / rows) / 2 * 2 + 1)
+                                .collect(),
+                        );
+                        let map = u32s((0..rows).map(|r| (r * 13) % tokens as u32).collect());
+                        let ids = if rows == 32769 {
+                            ops::astype(&ids, Dtype::I64, false, gpu).unwrap()
+                        } else {
+                            ids
+                        };
+                        let map = if rows == 8193 {
+                            let repeated = ops::repeat(&map, 2, 0, gpu).unwrap();
+                            ops::slice(&repeated, &[0], &[(rows * 2) as i32], &[2], gpu).unwrap()
+                        } else {
+                            map
+                        };
+                        let up = separate.then_some(&up);
+                        assert!(applies(&x, &map, &gate, up, &ids, true, true));
+                        assert!(!applies(&x, &map, &gate, up, &ids, false, true));
+                        assert!(!applies(&x, &map, &gate, up, &ids, true, false));
+                        let before = stats();
+                        let y =
+                            swiglu_rows(&x, &map, &gate, up, &ids, true, &streams, gpu).unwrap();
+                        let got = read(&y);
+                        let reference =
+                            read(&stock(&x, &map, &gate, up, &ids, true, &streams, gpu).unwrap());
+                        assert_eq!(got, reference);
+                        let after = stats();
+                        assert_eq!(after.failures, before.failures, "{after:?}");
+                        assert_eq!(after.calls, before.calls + 1);
+                        println!(
+                            "{}",
+                            serde_json::json!({"case":"synthetic","dtype":format!("{dtype:?}"),"bits":bits,"group":group,"separate":separate,"rows":rows,"sha256":hex_digest(&got),"canaries":after.canaries,"calls":after.calls})
+                        );
+                    }
+                }
+            }
+        }
+    }
+}
+
+#[test]
+#[ignore = "Metal canary rejection and sticky fallback; external supervisor"]
+fn issue112_canary_failures() {
+    use super::super::{gpu::Context, qwen_mtplx_tests::pattern};
+    super::configure_sources().unwrap();
+    let _context = Context::open_qwen(0).unwrap();
+    let streams = Streams::new(Some(super::allocator::Allocator::new().unwrap()));
+    let gpu = streams.default_stream(super::stream::Device::Gpu).unwrap();
+    let rows = 4096;
+    let e = 16;
+    let n = 128;
+    let k = 256;
+    let u32s = |v: Vec<u32>| {
+        Array::new(
+            &[v.len() as i32],
+            Dtype::U32,
+            super::scalar_buffer(&v.into_iter().flat_map(u32::to_le_bytes).collect::<Vec<_>>())
+                .unwrap(),
+        )
+        .unwrap()
+    };
+    let x = Array::new(&[1, 1, k], Dtype::BF16, pattern(k as u32, 8)).unwrap();
+    let ids = u32s((0..rows).map(|r| r * e / rows).collect());
+    let map = u32s(vec![0; rows as usize]);
+    let read = |a: &Array| {
+        ops::evaluate(&streams, std::slice::from_ref(a), gpu, false).unwrap();
+        let mut v = vec![0; a.layout().nbytes()];
+        a.buffer().read(a.offset(), &mut v).unwrap();
+        v
+    };
+    for (failure, group) in [(1, 32), (2, 64), (3, 128)] {
+        for separate in [false, true] {
+            let mut q = mlp::fixture_linear(e * n, k as u32, 4, group, 7);
+            for a in [&mut q.weight, &mut q.scales, &mut q.biases] {
+                let width = a.layout().dim(-1).unwrap();
+                *a = ops::reshape(a, &[e as i32, n as i32, width], gpu).unwrap();
+            }
+            let up = separate.then_some(&q);
+            let reference = read(&stock(&x, &map, &q, up, &ids, true, &streams, gpu).unwrap());
+            let before = stats();
+            FAILURE.store(failure, std::sync::atomic::Ordering::Relaxed);
+            let actual = read(&swiglu_rows(&x, &map, &q, up, &ids, true, &streams, gpu).unwrap());
+            FAILURE.store(0, std::sync::atomic::Ordering::Relaxed);
+            assert_eq!(actual, reference);
+            let rejected = stats();
+            assert_eq!(
+                rejected.failures - before.failures,
+                if separate { 1 } else { 2 }
+            );
+            assert_eq!(
+                read(&swiglu_rows(&x, &map, &q, up, &ids, true, &streams, gpu).unwrap()),
+                reference
+            );
+            let after = stats();
+            assert_eq!(after.canaries, rejected.canaries);
+            assert_eq!(after.calls, before.calls);
+            assert_eq!(after.failures, rejected.failures);
+            println!(
+                "{}",
+                serde_json::json!({"case":"sticky_fallback","failure":failure,"separate":separate,"canaries":after.canaries,"failures":after.failures,"fallbacks":after.fallbacks,"exact":true})
+            );
+        }
+    }
+}
+
+#[test]
+fn row_map_applicability() {
+    let a = |shape: &[i32], dtype| Array::unallocated(shape, dtype).unwrap();
+    let q = QuantizedLinear {
+        weight: a(&[128, 256, 32], Dtype::U32),
+        scales: a(&[128, 256, 4], Dtype::BF16),
+        biases: a(&[128, 256, 4], Dtype::BF16),
+        bits: 4,
+        group: 64,
+    };
+    let x = a(&[512, 1, 256], Dtype::BF16);
+    for rows in [1, 40, 63, 64, 4095, 4096, 32769] {
+        let map = a(&[rows], Dtype::U32);
+        let ids = a(&[rows], Dtype::U32);
+        assert_eq!(applies(&x, &map, &q, None, &ids, true, true), rows >= 4096);
+        assert!(!applies(&x, &map, &q, None, &ids, false, true));
+        assert!(!applies(&x, &map, &q, None, &ids, true, false));
+    }
+    let ids = a(&[4096], Dtype::U32);
+    for map in [
+        a(&[4096], Dtype::I32),
+        a(&[1, 4096], Dtype::U32),
+        a(&[4095], Dtype::U32),
+    ] {
+        assert!(!applies(&x, &map, &q, None, &ids, true, true));
+    }
+    for x in [
+        a(&[512, 1, 256], Dtype::F32),
+        a(&[512, 256], Dtype::BF16),
+        a(&[512, 2, 256], Dtype::BF16),
+        a(&[512, 1, 128], Dtype::BF16),
+    ] {
+        assert!(!applies(&x, &ids, &q, None, &ids, true, true));
+    }
+    let mut bad = q.clone();
+    bad.group = 16;
+    assert!(!applies(&x, &ids, &bad, None, &ids, true, true));
+    bad = q.clone();
+    bad.bits = 2;
+    assert!(!applies(&x, &ids, &bad, None, &ids, true, true));
+    bad = q.clone();
+    bad.scales = a(&[128, 256, 4], Dtype::F16);
+    assert!(!applies(&x, &ids, &bad, None, &ids, true, true));
+    bad = q.clone();
+    bad.weight = a(&[128, 255, 32], Dtype::U32);
+    assert!(!applies(&x, &ids, &bad, None, &ids, true, true));
+    assert!(!applies(&x, &ids, &q, Some(&bad), &ids, true, true));
+}
+
+#[test]
+#[ignore = "installed projection weights; external supervisor required"]
+fn issue112_installed_gate_up() {
+    use super::super::{
+        gpu::Context,
+        qwen_mtplx_tests::{hex_digest, pattern},
+    };
+    super::configure_sources().unwrap();
+    let _context = Context::open_qwen(0).unwrap();
+    let streams = Streams::new(Some(super::allocator::Allocator::new().unwrap()));
+    let gpu = streams.default_stream(super::stream::Device::Gpu).unwrap();
+    let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
+    let weights = super::weights::load_shards(&root, &streams).unwrap();
+    let parts = |name: &str| {
+        ["weight", "scales", "biases"].map(|p| {
+            weights[&format!("language_model.model.layers.0.mlp.switch_mlp.{name}.{p}")].clone()
+        })
+    };
+    let gate = parts("gate_proj");
+    let up = parts("up_proj");
+    let gu = std::array::from_fn(|i| {
+        ops::concatenate(&[gate[i].clone(), up[i].clone()], 1, gpu).unwrap()
+    });
+    let pack = |[weight, scales, biases]: [Array; 3]| QuantizedLinear {
+        weight,
+        scales,
+        biases,
+        bits: 4,
+        group: 64,
+    };
+    let fused = pack(gu);
+    let gate = pack(gate);
+    let up = pack(up);
+    let u32s = |v: Vec<u32>| {
+        Array::new(
+            &[v.len() as i32],
+            Dtype::U32,
+            super::scalar_buffer(&v.into_iter().flat_map(u32::to_le_bytes).collect::<Vec<_>>())
+                .unwrap(),
+        )
+        .unwrap()
+    };
+    let read = |a: &Array| {
+        ops::evaluate(&streams, std::slice::from_ref(a), gpu, false).unwrap();
+        let mut v = vec![0; a.layout().nbytes()];
+        a.buffer().read(a.offset(), &mut v).unwrap();
+        v
+    };
+    for tokens in [410, 2048, 3277, 4095, 4096, 8192] {
+        let rows = tokens * 10;
+        let x = Array::new(
+            &[tokens as i32, 1, 2560],
+            Dtype::BF16,
+            pattern(tokens * 2560, 7),
+        )
+        .unwrap();
+        let ids = u32s((0..rows).map(|r| r * 512 / rows).collect());
+        let map = u32s((0..rows).map(|r| r % tokens).collect());
+        let mut packed = None;
+        for separate in [false, true] {
+            let (g, u) = if separate {
+                (&gate, Some(&up))
+            } else {
+                (&fused, None)
+            };
+            let before = stats();
+            let y = swiglu_rows(&x, &map, g, u, &ids, true, &streams, gpu).unwrap();
+            let got = read(&y);
+            assert_eq!(
+                got,
+                read(&stock(&x, &map, g, u, &ids, true, &streams, gpu).unwrap())
+            );
+            let hash = hex_digest(&got);
+            if let Some(packed) = &packed {
+                assert_eq!(&hash, packed);
+            } else {
+                packed = Some(hash.clone());
+            }
+            let after = stats();
+            assert_eq!(after.failures, before.failures);
+            assert_eq!(after.calls, before.calls + 1);
+            println!(
+                "{}",
+                serde_json::json!({"case":"installed_gate_up","tokens":tokens,"rows":rows,"separate":separate,"sha256":hash,"canaries":after.canaries,"calls":after.calls,"exact":true})
+            );
+        }
+    }
+}
+
+#[test]
+#[ignore = "full installed model; validates all cold/warm state receipts against the pre-port oracle"]
+fn issue112_full_model_state() {
+    let before = stats();
+    super::issue110::issue110_large_cold_and_warm_prefill();
+    let after = stats();
+    assert!(after.calls > before.calls);
+    assert_eq!(after.failures, before.failures);
+    let directory = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE110_DIR").unwrap());
+    for chunk in [2048, 4096] {
+        for phase in ["cold", "warm"] {
+            let actual: serde_json::Value = serde_json::from_slice(
+                &std::fs::read(directory.join(format!("{chunk}-{phase}.json"))).unwrap(),
+            )
+            .unwrap();
+            let expected: serde_json::Value = serde_json::from_slice(
+                &std::fs::read(
+                    std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(format!(
+                        "docs/issue110-sorted-gather/{chunk}-{phase}-state.json"
+                    )),
+                )
+                .unwrap(),
+            )
+            .unwrap();
+            assert_eq!(actual["state"], expected["state"]);
+            assert_eq!(actual["cached"], expected["cached"]);
+        }
+    }
+    println!(
+        "{}",
+        serde_json::json!({"case":"model_state","calls":after.calls,"canaries":after.canaries,"failures":after.failures,"cold_warm_exact":true})
+    );
+}

src/engine/metal/qwen_mtplx/ops.rs

@@ -6,6 +6,8 @@ use super::stream::{Device, Stream, Streams};
 use super::{eval, views};
 
 pub(super) enum Operation {
+    MoeSortedGather(super::moe_sorted::Kernel),
+    MoeSortedCounts,
     Vision(super::vision_kernels::Kernel),
     Compiled(super::fused::Expression),
     Load(super::load::Load),
@@ -60,7 +62,7 @@ pub(super) enum Operation {
         bits: u32,
         group: u32,
     },
-    GatherQmmBf16 {
+    GatherQmm {
         bits: u32,
         group: u32,
         right_sorted: bool,
@@ -134,6 +136,9 @@ pub(super) enum Operation {
 impl Operation {
     fn evaluate(&self, input: &Array, output: &Array, stream: Stream) -> Result<(), String> {
         match self {
+            Self::MoeSortedGather(_) | Self::MoeSortedCounts => {
+                Err("sorted MoE requires captured inputs".into())
+            }
             Self::Compiled(_) => Err("compiled expression requires all inputs/outputs".into()),
             Self::QsaCompiledScalar(_) => {
                 Err("compiled scalar requires its captured inputs".into())
@@ -215,7 +220,7 @@ impl Operation {
             | Self::ArangeI64 => Err("invalid indexing arity".into()),
             Self::Vision(_) | Self::RmsNorm(_) => Err("RMSNorm requires two inputs".into()),
             Self::QuantizedLinearBf16 { .. } => Err("quantized linear requires four inputs".into()),
-            Self::GatherQmmBf16 { .. } => Err("GatherQMM requires six inputs".into()),
+            Self::GatherQmm { .. } => Err("GatherQMM requires six inputs".into()),
             Self::AffineQuantize { .. } => Err("quantize requires three sibling outputs".into()),
             Self::AffineDequantize { .. } => Err("dequantize requires three inputs".into()),
             Self::HyperRead(_) | Self::HyperV3R1 | Self::HyperV3R2 => {
@@ -262,6 +267,12 @@ pub(super) fn evaluate(
                 .operation()
                 .ok_or("array has no executable primitive")?;
             match &*operation {
+                Operation::MoeSortedGather(kernel) => {
+                    return kernel.evaluate(inputs, outputs);
+                }
+                Operation::MoeSortedCounts => {
+                    return super::moe_sorted::counts_evaluate(inputs, &outputs[0]);
+                }
                 Operation::Vision(k) => {
                     return super::vision_kernels::evaluate(*k, inputs, &outputs[0]);
                 }
@@ -526,7 +537,7 @@ pub(super) fn evaluate(
                 }
                 return super::linear::evaluate(inputs, &outputs[0], *bits, *group);
             }
-            if let Operation::GatherQmmBf16 {
+            if let Operation::GatherQmm {
                 bits,
                 group,
                 right_sorted,

src/engine/metal/qwen_mtplx/policy.rs

@@ -25,6 +25,11 @@ fn falsey(value: Option<&str>) -> bool {
     })
 }
 
+pub(super) fn sorted_gather_enabled(get: impl Fn(&'static str) -> Option<String>) -> bool {
+    !falsey(get("MTPLX_MOE_SORTED_GATHER_KERNEL").as_deref())
+        && !truthy(get("MTPLX_FORCE_GPU_FAMILY_FALLBACK").as_deref())
+}
+
 fn strict_bool(name: &str, raw: Option<String>) -> Result<bool, String> {
     match raw
         .as_deref()
@@ -323,3 +328,19 @@ fn qwen_server_policy_matches_original_functions() {
     }
     assert_eq!(count, 268);
 }
+
+#[test]
+fn sorted_moe_switch_and_family_rehearsal() {
+    assert!(sorted_gather_enabled(|_| None));
+    for off in ["0", " false ", "NO", "off"] {
+        assert!(!sorted_gather_enabled(|key| (key
+            == "MTPLX_MOE_SORTED_GATHER_KERNEL")
+            .then(|| off.to_owned())));
+    }
+    for force in ["1", " true ", "YES", "on"] {
+        assert!(!sorted_gather_enabled(|key| (key
+            == "MTPLX_FORCE_GPU_FAMILY_FALLBACK")
+            .then(|| force.to_owned())));
+    }
+    assert!(sorted_gather_enabled(|_| Some("unexpected".into())));
+}

tools/issue112-extract-metal.py

@@ -0,0 +1,19 @@
+"""Regenerate only the vendored GPU shader closure from the pinned oracle."""
+import subprocess,sys
+from pathlib import Path
+checkout=Path(sys.argv[1])
+assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
+sys.path.insert(0,str(checkout))
+import mlx.core as mx
+from mtplx.kernels import moe_sorted_gather_nax as k
+assert mx.__version__=='0.32.2'
+root=k._include_root();seen=set();out=[]
+for rel in (k._PREAMBLE_HEADER,*k._KERNEL_HEADERS):k._inline(root,rel,seen,out)
+for rel,name in [('mlx/backend/metal/kernels/unary_ops.h','Sigmoid'),('mlx/backend/metal/kernels/binary_ops.h','Multiply')]:
+ functor=k._functor(root,rel,name)
+ assert functor is not None
+ out.append(functor)
+metal=Path(__file__).resolve().parents[1]/'metal'
+preamble='// MLX 0.32.2 GPU support, Copyright Apple Inc., MIT; see MLX-LICENSE.txt.\n// Flattened only the GPU header closure used by MTPLX 9882703 row-map gather.\n'
+(metal/'mtplx-moe-sorted-header.metal').write_text(preamble+'\n'.join(out)+'\nusing namespace mlx::steel;\n')
+(metal/'mtplx-moe-sorted-body.metal').write_text('// MTPLX 9882703, MIT; see MTPLX-LICENSE.txt.\n'+k._SOURCE)

tools/issue112-installed-reference.py

@@ -0,0 +1,38 @@
+"""Installed gate/up row-map oracle, using the identical native fixture inputs."""
+import hashlib,json,subprocess,sys
+from pathlib import Path
+checkout,root,native=map(Path,sys.argv[1:4])
+assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
+sys.path.insert(0,str(checkout))
+import mlx.core as mx
+import numpy as np
+from mtplx.kernels import moe_sorted_gather_nax as kernel
+assert mx.__version__=='0.32.2'
+index=json.loads((root/'model.safetensors.index.json').read_text())['weight_map']
+base='language_model.model.layers.0.mlp.switch_mlp.'
+weights=mx.load(str(root/index[base+'gate_proj.weight']))
+def parts(proj):return [weights[base+proj+'.'+part] for part in ['weight','scales','biases']]
+gate=parts('gate_proj');up=parts('up_proj')
+gu=[mx.concatenate(a,axis=1) for a in zip(gate,up)]
+mx.eval(*gu,*gate,*up)
+def digest(a):
+ mx.eval(a)
+ return hashlib.sha256(np.asarray(a.view(mx.uint16)).tobytes()).hexdigest()
+count=0
+for line in native.read_text().splitlines():
+ if '"case":"installed_gate_up"' not in line:continue
+ case=json.loads(line[line.index('{'):]);count+=1
+ tokens,rows=case['tokens'],case['rows']
+ v=((np.arange(tokens*2560,dtype=np.uint32)*17+7*13)%257).astype(np.int32)-128
+ x=mx.array((v.astype(np.float32)/128).reshape(tokens,1,2560)).astype(mx.bfloat16)
+ row_map=mx.arange(rows,dtype=mx.uint32)%tokens
+ ids=mx.array(np.arange(rows,dtype=np.uint32)*512//rows)
+ g,u=(gate,up) if case['separate'] else (gu,None)
+ kw=dict(group_size=64,bits=4,swiglu=True,up=u)
+ y=kernel.gather_rows_qmm(x,row_map,*g,ids,**kw)
+ stock=kernel._stock(x,row_map,*g,ids,**kw)
+ assert y is not None and digest(y)==digest(stock)==case['sha256'],case
+ print(json.dumps(dict(case='installed_oracle',native=case,exact=True,kernel=kernel.stats())),flush=True)
+ del x,row_map,ids,y,stock
+ mx.clear_cache()
+assert count==12,count

tools/issue112-model-reference.py

@@ -0,0 +1,58 @@
+"""Pinned full-model cold/warm prefill, including explicit hidden-state capture."""
+import argparse,hashlib,json,os,subprocess,sys
+from pathlib import Path
+parser=argparse.ArgumentParser(description=__doc__)
+for name in ['checkout','root','native','out']:parser.add_argument(name,type=Path)
+parser.add_argument('--hidden',action='store_true')
+a=parser.parse_args()
+assert subprocess.check_output(['git','-C',str(a.checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
+sys.path.insert(0,str(a.checkout))
+from mtplx.server.openai import _server_runtime_env_overrides
+os.environ.update(_server_runtime_env_overrides(argparse.Namespace(model=str(a.root),generation_mode='mtp' if a.hidden else 'ar'),None))
+os.environ['MTPLX_SUSTAINED_PREFILL']='1'
+import mlx.core as mx
+import numpy as np
+from mlx_lm.utils import load_model
+from mtplx.models import qwen4_exp
+from mtplx.runtime import MTPLXRuntime,_load_tokenizer_resilient
+from mtplx.sampling import SamplerConfig
+from mtplx.generation import generate_ar,restore_or_prefill_prompt_state
+from mtplx.cache_state import snapshot_cache,snapshot_cache_lazy_hybrid
+from mtplx.session_bank import SessionBank,_lazy_snapshot_enabled
+from mtplx.kernels import moe_sorted_gather_nax as kernel
+assert mx.__version__=='0.32.2'
+a.out.mkdir(parents=True,exist_ok=True)
+model,config=load_model(a.root,lazy=False,strict=True,get_model_classes=lambda **_:(qwen4_exp.Model,qwen4_exp.ModelArgs))
+model.post_weight_load(a.root)
+tokenizer=_load_tokenizer_resilient(a.root,config)
+from mtplx.mtp_patch import MTPContract
+if a.hidden: assert model.attach_mtp(a.root)
+rt=MTPLXRuntime(model,tokenizer,a.root,a.hidden,MTPContract() if a.hidden else None)
+def fingerprint(v):
+ values=np.asarray(v.astype(mx.float32)).astype('<f4')
+ assert np.isfinite(values).all()
+ return dict(shape=list(v.shape),sha256=hashlib.sha256(values.tobytes()).hexdigest())
+for chunk in [2048,4096]:
+ os.environ['MTPLX_PREFILL_CHUNK_SIZE']=str(chunk)
+ bank=SessionBank()
+ for phase in ['cold','warm']:
+  name=f'{chunk}-{phase}' if a.hidden else f'{chunk}-0-{phase}'
+  expected=json.loads((a.native/(name+'.json')).read_text())
+  prompt=expected['state']['ids'] if a.hidden else expected['state']['ids'][:expected['metrics']['prompt']]
+  if a.hidden:
+   result=restore_or_prefill_prompt_state(rt,prompt,mtp_history_policy='cycle',session_bank=bank,session_id='issue112',capture_hidden=True,store_prefix_snapshot=False,prefill_callback=lambda event:print(json.dumps(dict(case='progress',chunk=chunk,phase=phase,event=event),default=str),flush=True))
+   cache,logits,hidden,ids,cached=result.trunk_cache,result.logits,result.hidden,prompt,result.cached_tokens
+  else:
+   result=generate_ar(rt,prompt,max_tokens=1,sampler=SamplerConfig(temperature=0.,top_p=.95,top_k=20),seed=1,stop_token_ids=set(),capture_final_state=True,session_bank=bank,session_id='issue112',store_prefix_snapshot=False,token_callback=lambda ids:print(json.dumps(dict(case='tokens',chunk=chunk,phase=phase,ids=ids)),flush=True))
+   final=result.final_state
+   assert final.safe_to_commit
+   cache,logits,hidden,ids,cached=final.final_trunk_cache,final.final_logits,final.final_hidden,prompt+result.tokens,result.stats.cached_tokens
+  states=[[None if v is None else fingerprint(v) for v in state] if state is not None else None for state in (snapshot_cache_lazy_hybrid(cache) if _lazy_snapshot_enabled() else snapshot_cache(cache)).states]
+  record=dict(state=dict(ids=ids,logits=fingerprint(logits),states=states),hidden=None if hidden is None else fingerprint(hidden))
+  (a.out/(name+'.json')).write_text(json.dumps(record))
+  assert record['state']==expected['state'],(chunk,phase,'state differs')
+  assert record['hidden']==expected['hidden'],(chunk,phase,'hidden differs')
+  assert cached==(expected['cached'] if a.hidden else expected['metrics']['cached'])
+  print(json.dumps(dict(case='model_oracle',chunk=chunk,phase=phase,hidden=a.hidden,exact=True,cached=cached,kernel=kernel.stats())),flush=True)
+  if phase=='cold':
+   assert bank.put(runtime=rt,token_ids=ids,cache=cache,logits=logits,hidden=hidden,hidden_variant='post_norm',session_id='issue112',mtp_history_policy='cycle',snapshot_epoch=len(ids)) is not None

tools/issue112-reference.py

@@ -0,0 +1,45 @@
+"""Pinned row-map kernel, guarded stock chain and native synthetic bit receipts."""
+import hashlib,json,subprocess,sys
+from pathlib import Path
+checkout,native=map(Path,sys.argv[1:3])
+assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
+sys.path.insert(0,str(checkout))
+import mlx.core as mx
+import numpy as np
+from mtplx.kernels import moe_sorted_gather_nax as kernel
+assert mx.__version__=='0.32.2'
+def pattern(count,salt,scale=1):
+ a=((np.arange(count,dtype=np.uint32)*17+salt*13)%257).astype(np.int32)-128
+ return mx.array(a.astype(np.float32)/128*scale).astype(mx.bfloat16)
+def pack(e,n,k,bits,group,salt,dtype):
+ words=np.arange(e*n*k//32*bits,dtype=np.uint32)*np.uint32(2654435761)+np.uint32(12345)
+ return (mx.array(words.reshape(e,n,k*bits//32)),pattern(e*n*k//group,salt,1/64).reshape(e,n,k//group).astype(dtype),pattern(e*n*k//group,salt+1,1/64).reshape(e,n,k//group).astype(dtype))
+def digest(a):
+ mx.eval(a)
+ return hashlib.sha256(np.asarray(a.view(mx.uint16)).tobytes()).hexdigest()
+count=0
+for line in native.read_text().splitlines():
+ if '"case":"synthetic"' not in line:continue
+ case=json.loads(line[line.index('{'):])
+ count+=1
+ dtype={'BF16':mx.bfloat16,'F16':mx.float16}[case['dtype']]
+ bits,group,rows=case['bits'],case['group'],case['rows']
+ gate=pack(128,256,256,bits,group,7,dtype)
+ up=pack(128,256,256,bits,group,9,dtype) if case['separate'] else None
+ x=pattern(512*256,8).reshape(512,1,256).astype(dtype)
+ r=np.arange(rows,dtype=np.uint32)
+ ids=mx.array((r*125//rows)//2*2+1)
+ row_map=mx.array((r*13)%512)
+ assert kernel.applies(x,row_map,*gate,ids,group_size=group,bits=bits,mode='affine',up=up)
+ kwargs=dict(group_size=group,bits=bits,swiglu=True,up=up)
+ before=kernel.stats()
+ y=kernel.gather_rows_qmm(x,row_map,*gate,ids,**kwargs)
+ assert y is not None,kernel.stats()
+ stock=kernel._stock(x,row_map,*gate,ids,**kwargs)
+ actual=digest(y)
+ assert actual==digest(stock)==case['sha256'],(case,actual,digest(stock))
+ print(json.dumps(dict(case='synthetic_oracle',native=case,exact=True,kernel_before=before,kernel_after=kernel.stats())),flush=True)
+ del x,row_map,ids,gate,up,y,stock
+ mx.clear_cache()
+
+assert count==72,count