diff --git a/docs/issue113-head-chunk/README.md b/docs/issue113-head-chunk/README.md
new file mode 100644
index 0000000..41f7708
--- /dev/null
+++ b/docs/issue113-head-chunk/README.md
@@ -0,0 +1,123 @@
+# Qwen head-chunked SDPA (#113)
+
+The dense Qwen attention consumer now follows pinned MTPLX `_verify_sdpa`:
+when enabled (default) and `rows * GQA > 32` with `rows <= 32`, split query
+heads inside each KV group into `min(GQA, 32 / rows)` heads, preserve remainder
+heads, call the existing SDPA primitive with that group's K/V, then concatenate
+in original head order. The first engagement is logged. Explicit false/off/no/0
+keeps the prior consumer. Policy is resolved through the existing Qwen lookup;
+no process environment mutation or shared-family SDPA change was introduced.
+
+Oracle: MTPLX `9882703f3105363ddc37eca9f97aa09a1d387112` / MLX 0.32.2.
+Artifact: Bare-Speed `74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`; environment
+and artifact fingerprints in ../issue104-ar/environment.json. Pre-port native
+commit: `7b894fbfeed81d2944b9cf479e60638ef04987c7`.
+
+## Exact scope found during review
+
+The wrapper's name does not restrict it to verification. It is the final dense
+consumer after sparse/gather branches in `Attention.__call__`, in any phase.
+Thus short prefills also use it. Existing sparse selections, gathered consumers,
+gates, projection order, RoPE and cache offsets are unchanged.
+
+The pinned `_sdpa_head_chunked` passes `mask` **unchanged** to each subcall. It
+does not slice a full-query-head mask. Broadcast-compatible masks work; a mask
+whose head dimension cannot broadcast to a chunk is rejected. The fixtures
+preserve these errors as well as valid head-specific masks. This resolves the
+issue body's tentative mask-slicing wording against the actual pinned source.
+
+For installed head dimension 256, MLX 0.32.2 uses the vector primitive only
+through eight query rows. The wrapper still splits heads for eligible rows
+9–32, and those subcalls retain the existing fallback primitive. The port does
+not force a vector route merely because the wrapper docstring describes it.
+Wide rows above 32 pass through unchanged.
+
+## Cache compatibility
+
+Checkpoint identity now records the head-chunk policy. Missing metadata means
+the actual old unchunked route. A policy difference rejects all MTP states and
+AR states with four or more tokens, before reading or allocating tensor blobs.
+MTP can retain a short frontier from a wider verify window; an AR prefix of at
+most three tokens cannot have executed a three-row prefill body and remains
+restorable. Existing format and unrelated identity checks remain intact. Policy
+changes in both directions are covered, including current/legacy metadata.
+
+The product test rejects an actual pre-port checkpoint produced during #109,
+without changing the live session, then saves/reopens corrected state and
+compares the complete warm frontier. The explicit-off control restores that
+same actual old file exactly. Fake tiny checkpoint fixtures additionally cover
+AR/MTP × 1/3/4-token frontiers × both producing/consuming policies and missing
+metadata; affected files stop after header/manifest reads.
+
+## Isolated oracle evidence
+
+- 520 independent plan cases cover boundary/invalid head counts and rows.
+- 216 installed-dimension attention cases include single/grouped/ungrouped
+  heads, remainder heads, batch two, offset/strided Q/K/V, no/bool/additive/
+  two-dimensional/full-head/chunk-local masks, default on and explicit off,
+  short KV, and contexts through 4097 tokens. All 159 successful outputs match
+  raw BF16 bits and all 57 invalid cases match oracle rejection. Native graph
+  subcall shapes, fused primitive counts, vector dispatches and fallback
+  softmax counts match the independently captured oracle route.
+- The exact reported request (`Count from one to ten.`, low reasoning, seed 1,
+  greedy, depth 3, max 12, context-copy disabled) is checked cold, warm and after
+  disk reopen. Default and off controls have separate native/reference runs.
+  Every verify input row, output logit/hidden byte, emitted token, final array
+  and saved trunk/MTP cache bit is compared; no zero-sign or numeric tolerance
+  exception is allowed in this issue's oracle tool.
+
+Comparisons use the same snapshot stage as the server: lazy hybrid trunk and
+cloned MTP head. Taking an extra deep clone while measuring a lazy key array
+would add zero and change signed-zero bits. Warm suffix prefill's final hidden
+row is compared after the same final-row selection used by the upstream owner
+(`generation.py::_prefill_restored_prompt_suffix`), not against its unsliced
+multi-row intermediate. Verify-row arrays themselves are compared in full.
+
+## Reproduction
+
+Generate the independent fixture using isolated MLX 0.32.2 Python:
+`tools/issue113-reference.py PINNED_CHECKOUT tests/fixtures/mtplx-head-chunk.json`.
+Build the release test executable with `cargo test --release --all-features
+--bin ds4-server issue113 --no-run`, then copy the executable outside `target`
+before running it. Run `issue113_plan_and_policy_match_pinned_oracle` normally.
+
+Run the ignored tests serially under `target/debug/test-supervisor MEMORY_MIB
+60 120 --command TEST_BINARY FILTER --ignored --nocapture --test-threads=1`:
+
+- `issue113_head_chunk_outputs_and_dispatch`: 4096 MiB.
+- `qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state`: 4096 MiB,
+  with `DS4_QWEN38_CHECKPOINT_DIR` set to an empty output directory.
+- `issue113_default_verify_and_checkpoint`: 114688 MiB, with
+  `DS4_QWEN38_ARTIFACTS=MODEL_DIR`, `MTPLX_CONTEXT_COPY=0`,
+  `DS4_ISSUE109_DIR=OUTPUT_DIRECTORY` (shared real-request harness), and
+  `DS4_ISSUE113_OLD=ACTUAL_PRE_PORT_CHECKPOINT`. Repeat separately with
+  `MTPLX_QWEN4_VERIFY_SDPA_HEAD_CHUNK=0` for the off control.
+
+For each model run, use isolated reference Python to execute
+`tools/issue113-model-reference.py PINNED_CHECKOUT MODEL_DIR NATIVE_OUTPUT
+ORACLE_OUTPUT`, with the matching default/off switch. Keep child output
+separate from supervisor resource logs to avoid interleaved JSON records.
+
+## Completion review
+
+The four-row reproduction `[760,1156,369,9859]` now takes four vector calls
+(8+4 query heads for each of two KV groups). Its logits and hidden arrays differ
+from the old ordinary fallback and match the default pinned oracle exactly.
+Both cold and warm turns have four verify forwards and 12 emitted tokens; the
+warm turn restores 58 tokens. All verify rows, prefill final-row state, final
+logits/hidden, trunk arrays and MTP arrays match raw reference bits. Disk reopen
+reproduces the entire warm state. The off route matches all retained #109
+pre-port verify, final-logit/hidden and full cache-array evidence byte for byte.
+
+Regression controls pass for the QSA eager selector, dense masks, tiled prefill
+gather, row gather, unchunked SDPA, complete QSA modules with TF32 off/on, and
+exhaustive gate/injection laws. Installed DeepSeek resident-session switching
+and GLM verifier/recurrent-state controls pass. Review confirms no production
+DS4/GLM code or shader changed and no shared-family helper was redirected.
+This is the pinned Qwen execution route, not a DS4Server-only feature.
+
+All commit gates pass: formatting, warning-free Clippy, signed application
+bundle, and all-features tests (313 application + 8 tool tests, 236 explicitly
+ignored hardware/reference tests). Relevant ignored tests ran separately under
+the resource supervisor; their receipts and the complete gate outputs are
+stored here. No implementation or acceptance item is stubbed or deferred.
diff --git a/docs/issue113-head-chunk/bundle.log b/docs/issue113-head-chunk/bundle.log
new file mode 100644
index 0000000..7e534c6
--- /dev/null
+++ b/docs/issue113-head-chunk/bundle.log
@@ -0,0 +1,8 @@
+cargo build --release --all-targets --all-features
+   Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
+    Finished `release` profile [optimized] target(s) in 21.46s
+warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
+note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
+cargo packager --release --formats app
+codesign --force --deep --sign - --timestamp=none target/release/DS4Server.app
+target/release/DS4Server.app: replacing existing signature
diff --git a/docs/issue113-head-chunk/checkpoint.log b/docs/issue113-head-chunk/checkpoint.log
new file mode 100644
index 0000000..d6f7a85
--- /dev/null
+++ b/docs/issue113-head-chunk/checkpoint.log
@@ -0,0 +1,11 @@
+running 1 test
+test engine::metal::qwen_mtplx::session_checkpoint::qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
+ds4: Metal 4 tensor API enabled for Tensor kernels
+ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
+ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx
+#113 SDPA policy: legacy/current, both directions, AR/MTP, 1/3/4-token frontiers; exact compatible restore and incompatible rejection before tensor reads
+Qwen sorted-gather checkpoint policy: old short state exact, possibly affected long state rejected before restore
+Qwen checkpoint: exact roundtrip, cancellation/corruption controls, old MTP rejected and old AR restored
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 0.13s
diff --git a/docs/issue113-head-chunk/clippy.log b/docs/issue113-head-chunk/clippy.log
new file mode 100644
index 0000000..ab3368f
--- /dev/null
+++ b/docs/issue113-head-chunk/clippy.log
@@ -0,0 +1,4 @@
+    Checking ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
+    Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.49s
+warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
+note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
diff --git a/docs/issue113-head-chunk/deepseek.log b/docs/issue113-head-chunk/deepseek.log
new file mode 100644
index 0000000..361be84
--- /dev/null
+++ b/docs/issue113-head-chunk/deepseek.log
@@ -0,0 +1,10 @@
+
+running 1 test
+test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
+ds4: Metal 4 tensor API enabled for Tensor kernels
+ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
+ds4: Metal model views created in 1.885 ms, residency requested in 9117.538 ms, warmup 3.450 ms (mapped 82697.67 MiB from offset 5.09 MiB)
+ds4: Metal mapped mmaped model as 2 overlapping shared buffers
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 9.69s
diff --git a/docs/issue113-head-chunk/fmt.log b/docs/issue113-head-chunk/fmt.log
new file mode 100644
index 0000000..e69de29
diff --git a/docs/issue113-head-chunk/glm.log b/docs/issue113-head-chunk/glm.log
new file mode 100644
index 0000000..c041c5e
--- /dev/null
+++ b/docs/issue113-head-chunk/glm.log
@@ -0,0 +1,11 @@
+ds4: GLM MTP draft 5733 at position 12 in 2.3 ms
+verify2 scalar boundary cap=2 cancelled=true passed
+verify2 prefill 0
+verify2 prefill 4
+verify2 prefill 8
+verify2 prefill 12
+ds4: GLM MTP draft 5733 at position 12 in 2.3 ms
+engine-lifetime counters across new and restored chats passed
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 11.91s
diff --git a/docs/issue113-head-chunk/mtplx_qsa_dense_mask_graph_matches_reference.log b/docs/issue113-head-chunk/mtplx_qsa_dense_mask_graph_matches_reference.log
new file mode 100644
index 0000000..bae838c
--- /dev/null
+++ b/docs/issue113-head-chunk/mtplx_qsa_dense_mask_graph_matches_reference.log
@@ -0,0 +1,11 @@
+MTPLX canonical qsa_dense_mask_8_17_7_8_False_False: exact
+MTPLX canonical qsa_dense_mask_8_17_7_8_False_True: exact
+MTPLX canonical qsa_dense_mask_8_17_7_8_True_False: exact
+MTPLX canonical qsa_dense_mask_8_17_7_8_True_True: exact
+MTPLX canonical qsa_dense_mask_0_0_4_0_False_False: exact
+MTPLX canonical qsa_dense_mask_0_0_4_0_False_True: exact
+MTPLX canonical qsa_dense_mask_0_0_4_0_True_False: exact
+MTPLX canonical qsa_dense_mask_0_0_4_0_True_True: exact
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 0.75s
diff --git a/docs/issue113-head-chunk/mtplx_qsa_eager_array_graph_matches_reference.log b/docs/issue113-head-chunk/mtplx_qsa_eager_array_graph_matches_reference.log
new file mode 100644
index 0000000..f797c67
--- /dev/null
+++ b/docs/issue113-head-chunk/mtplx_qsa_eager_array_graph_matches_reference.log
@@ -0,0 +1,11 @@
+MTPLX canonical qsa_eager_graph_float32_16_tf1: exact
+MTPLX canonical qsa_eager_graph_float32_17_tf1: exact
+MTPLX canonical qsa_eager_graph_float32_18_tf1: exact
+MTPLX canonical qsa_eager_graph_float32_19_tf1: exact
+MTPLX canonical qsa_eager_graph_float32_20_tf1: exact
+MTPLX canonical qsa_eager_graph_float32_21_tf1: exact
+MTPLX canonical qsa_eager_graph_float32_22_tf1: exact
+MTPLX canonical qsa_eager_graph_float32_23_tf1: exact
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 1.06s
diff --git a/docs/issue113-head-chunk/mtplx_qsa_prefill_gather_graph_matches_reference.log b/docs/issue113-head-chunk/mtplx_qsa_prefill_gather_graph_matches_reference.log
new file mode 100644
index 0000000..6d68d52
--- /dev/null
+++ b/docs/issue113-head-chunk/mtplx_qsa_prefill_gather_graph_matches_reference.log
@@ -0,0 +1,11 @@
+MTPLX canonical qsa_prefill_gather_h2_s7_p0_r1_t4_strided0: exact
+MTPLX canonical qsa_prefill_gather_h2_s7_p0_r1_t4_strided1: exact
+MTPLX canonical qsa_prefill_gather_h2_s7_p0_r1_t64_strided0: exact
+MTPLX canonical qsa_prefill_gather_h2_s7_p0_r1_t64_strided1: exact
+MTPLX canonical qsa_prefill_gather_h2_s2048_p0_r4_t64_strided0: exact
+MTPLX canonical qsa_prefill_gather_h2_s2048_p0_r4_t64_strided1: exact
+MTPLX canonical qsa_prefill_gather_h2_s2048_p0_r4_t2048_strided0: exact
+MTPLX canonical qsa_prefill_gather_h2_s2048_p0_r4_t2048_strided1: exact
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 0.51s
diff --git a/docs/issue113-head-chunk/mtplx_qsa_rows_gather_graph_matches_reference.log b/docs/issue113-head-chunk/mtplx_qsa_rows_gather_graph_matches_reference.log
new file mode 100644
index 0000000..543bceb
--- /dev/null
+++ b/docs/issue113-head-chunk/mtplx_qsa_rows_gather_graph_matches_reference.log
@@ -0,0 +1,11 @@
+MTPLX canonical qsa_rows_attention_h24_s64_t113_k17_offset: exact
+MTPLX canonical qsa_rows_attention_h24_s64_t113_k17_strided: exact
+MTPLX canonical qsa_rows_attention_h2_s3_t33_k8_plain: exact
+MTPLX canonical qsa_rows_attention_h2_s3_t33_k8_offset: exact
+MTPLX canonical qsa_rows_attention_h2_s3_t33_k8_strided: exact
+MTPLX canonical qsa_rows_attention_h2_s16_t1057_k1025_plain: exact
+MTPLX canonical qsa_rows_attention_h2_s16_t1057_k1025_offset: exact
+MTPLX canonical qsa_rows_attention_h2_s16_t1057_k1025_strided: exact
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 0.25s
diff --git a/docs/issue113-head-chunk/mtplx_sdpa_fallback_graph_matches_reference.log b/docs/issue113-head-chunk/mtplx_sdpa_fallback_graph_matches_reference.log
new file mode 100644
index 0000000..c1b99ed
--- /dev/null
+++ b/docs/issue113-head-chunk/mtplx_sdpa_fallback_graph_matches_reference.log
@@ -0,0 +1,11 @@
+MTPLX canonical sdpa_fallback_B1_h24_r2048_t2048_bool: exact
+MTPLX canonical sdpa_fallback_B1_h24_r2048_t2048_add: exact
+MTPLX canonical sdpa_fallback_B2_h24_r3_t17_none: exact
+MTPLX canonical sdpa_fallback_B2_h24_r3_t17_bool: exact
+MTPLX canonical sdpa_fallback_B2_h24_r3_t17_add: exact
+MTPLX canonical sdpa_fallback_B1_h2_r16_t33_none: exact
+MTPLX canonical sdpa_fallback_B1_h2_r16_t33_bool: exact
+MTPLX canonical sdpa_fallback_B1_h2_r16_t33_add: exact
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 0.74s
diff --git a/docs/issue113-head-chunk/off-product/cold-arrays.json b/docs/issue113-head-chunk/off-product/cold-arrays.json
new file mode 100644
index 0000000..1e1f269
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/cold-arrays.json
@@ -0,0 +1 @@
+{"trunk":[[{"shape":[1,3,10240],"sha256":"1dadce7f74c7b48ad423170b8884bf5024c3d6dd91b2f9d800afca7d32679d7b","value_sha256":"1dadce7f74c7b48ad423170b8884bf5024c3d6dd91b2f9d800afca7d32679d7b"},{"shape":[1,48,128,128],"sha256":"d7436faf9d50e278f7fa095e9c217467020a6f534a48195775c212caa0ece52d","value_sha256":"d7436faf9d50e278f7fa095e9c217467020a6f534a48195775c212caa0ece52d"}],[{"shape":[1,3,10240],"sha256":"ca1b3cf93598b62575f01aea05ff0d0f087c00fdf1b63654626ef5ec3487715a","value_sha256":"ca1b3cf93598b62575f01aea05ff0d0f087c00fdf1b63654626ef5ec3487715a"},{"shape":[1,48,128,128],"sha256":"f1364bb8165b9189279b7769e4081f213c562f4e0541acc5962cf2ae9f42a831","value_sha256":"f1364bb8165b9189279b7769e4081f213c562f4e0541acc5962cf2ae9f42a831"},{"shape":[1,9,10240],"sha256":"c9110c820fc39f8c19c770594c15e0997693015090a3c7af4b6c8ec9a31ebc4e","value_sha256":"c9110c820fc39f8c19c770594c15e0997693015090a3c7af4b6c8ec9a31ebc4e"},{"shape":[1,2],"sha256":"9febaa06152dac026dba5f2c1151c11ff6a1b2f00e92278cdfd2905c87cf2c2f","value_sha256":"9febaa06152dac026dba5f2c1151c11ff6a1b2f00e92278cdfd2905c87cf2c2f"}],[{"shape":[1,3,10240],"sha256":"320fdcf9c7ab899133952e093d1f3de613ba5581b3aaad45c4ad7b1b4e43f1f3","value_sha256":"320fdcf9c7ab899133952e093d1f3de613ba5581b3aaad45c4ad7b1b4e43f1f3"},{"shape":[1,48,128,128],"sha256":"8cae4ba20fd0e54f8da130e5f8681ece7c54672b7a6404800cded8e64a977e08","value_sha256":"8cae4ba20fd0e54f8da130e5f8681ece7c54672b7a6404800cded8e64a977e08"}],[{"shape":[1,2,58,256],"sha256":"404510256f371f0bc2e7b8af2d1f7eb2ae0322d42eb9a0ee5827a9a5c8d489c0","value_sha256":"404510256f371f0bc2e7b8af2d1f7eb2ae0322d42eb9a0ee5827a9a5c8d489c0"},{"shape":[1,2,58,256],"sha256":"f14afc7fe1c5e4c62c84121e804e14aa484db378861904d4e234ffeef10252ef","value_sha256":"f14afc7fe1c5e4c62c84121e804e14aa484db378861904d4e234ffeef10252ef"},{"shape":[1,58,128],"sha256":"ca2d0b2f09e87fab8d0a6aede7073ec53bc37b584851f0342c76244f05ca3e12","value_sha256":"ca2d0b2f09e87fab8d0a6aede7073ec53bc37b584851f0342c76244f05ca3e12"},{"shape":[1,14,128],"sha256":"a609a4f52c9b3dc0fb5d6554019786bebecdad220bf5ae6b40d4939821f12c8b","value_sha256":"a609a4f52c9b3dc0fb5d6554019786bebecdad220bf5ae6b40d4939821f12c8b"}],[{"shape":[1,3,10240],"sha256":"0c5bdce8d9cbd2a24162139582f1889f0413632feafdba4e79cdd88353e6abc2","value_sha256":"0c5bdce8d9cbd2a24162139582f1889f0413632feafdba4e79cdd88353e6abc2"},{"shape":[1,48,128,128],"sha256":"d47c0c965fb6c41a7e1824144ef61546a7e40ead8d8606822173e21a17a4fc0f","value_sha256":"d47c0c965fb6c41a7e1824144ef61546a7e40ead8d8606822173e21a17a4fc0f"}],[{"shape":[1,3,10240],"sha256":"66014c1bce9cbbd223ec6af2f5517de66f71ce443721088167dad0c0186289fb","value_sha256":"66014c1bce9cbbd223ec6af2f5517de66f71ce443721088167dad0c0186289fb"},{"shape":[1,48,128,128],"sha256":"2593987c9ca9aeb30910e7eb69f8d6bd61c1e035861915cbe27b0838e26f3098","value_sha256":"2593987c9ca9aeb30910e7eb69f8d6bd61c1e035861915cbe27b0838e26f3098"}],[{"shape":[1,3,10240],"sha256":"45a3dbbf3dc7a79b7db631fe4c2802d6e1f4433f4b03e961df072147e8c242a2","value_sha256":"45a3dbbf3dc7a79b7db631fe4c2802d6e1f4433f4b03e961df072147e8c242a2"},{"shape":[1,48,128,128],"sha256":"e266e3456f01268de8868258a750ee18c5f8b1ff4391f647aec60a39d56f98a0","value_sha256":"e266e3456f01268de8868258a750ee18c5f8b1ff4391f647aec60a39d56f98a0"}],[{"shape":[1,2,58,256],"sha256":"ba60b2860a5717f46a4a7ffef0257799f96acb37de1a00ba88ddfc149bf44ca2","value_sha256":"ba60b2860a5717f46a4a7ffef0257799f96acb37de1a00ba88ddfc149bf44ca2"},{"shape":[1,2,58,256],"sha256":"57d040a722a1427e9b0744ee652aecca9ec30cc7b617e3db9534e772d79f42e4","value_sha256":"57d040a722a1427e9b0744ee652aecca9ec30cc7b617e3db9534e772d79f42e4"},{"shape":[1,58,128],"sha256":"84d339a2f8558893f3d73e751d5ee3d81bb1f3b1184c0b35555bf5456fdc5a22","value_sha256":"84d339a2f8558893f3d73e751d5ee3d81bb1f3b1184c0b35555bf5456fdc5a22"},{"shape":[1,14,128],"sha256":"a5378133f448c4b6380dc4711420d3b069dd8491ffc797fd16e023294405b707","value_sha256":"a5378133f448c4b6380dc4711420d3b069dd8491ffc797fd16e023294405b707"}],[{"shape":[1,3,10240],"sha256":"e6d4afab81c6a58f08c8d252991e91d25d2d60e497c3fc00614451edd0ea908c","value_sha256":"e6d4afab81c6a58f08c8d252991e91d25d2d60e497c3fc00614451edd0ea908c"},{"shape":[1,48,128,128],"sha256":"f94349b9898b0d1a5770f4aca791d4c114abad341641688012de2e2fcaa6237c","value_sha256":"f94349b9898b0d1a5770f4aca791d4c114abad341641688012de2e2fcaa6237c"}],[{"shape":[1,3,10240],"sha256":"54d2e63c68f9fc1412178f3f21bcf5897ab228967e01f709ed9d5ba0e6d0c016","value_sha256":"54d2e63c68f9fc1412178f3f21bcf5897ab228967e01f709ed9d5ba0e6d0c016"},{"shape":[1,48,128,128],"sha256":"868155f0fd6a62a46decb4f87c3e2f27310f9da38796fe0a8d926844248c7259","value_sha256":"868155f0fd6a62a46decb4f87c3e2f27310f9da38796fe0a8d926844248c7259"}],[{"shape":[1,3,10240],"sha256":"0073cc8754020915aedaad03803669a61281e18cb2dd389cb19ef7b5ed769b84","value_sha256":"0073cc8754020915aedaad03803669a61281e18cb2dd389cb19ef7b5ed769b84"},{"shape":[1,48,128,128],"sha256":"abd539c013e58f489c23c48dfc168be7a9e2145deee859432cb83d0d8764e2df","value_sha256":"abd539c013e58f489c23c48dfc168be7a9e2145deee859432cb83d0d8764e2df"}],[{"shape":[1,2,58,256],"sha256":"d855a9efa61fd043cf03b444aefe90a1fcefa5af03770819333ec695af328dc5","value_sha256":"d855a9efa61fd043cf03b444aefe90a1fcefa5af03770819333ec695af328dc5"},{"shape":[1,2,58,256],"sha256":"e2d0094873c5a47590a465e5c06fa7cab90e6ae99cb2eba7df88340755be2725","value_sha256":"e2d0094873c5a47590a465e5c06fa7cab90e6ae99cb2eba7df88340755be2725"},{"shape":[1,58,128],"sha256":"4ad0c060f771f328465a3dd5a6333d34fb9aa4cb9052c8c7811ab2dbd17d088e","value_sha256":"4ad0c060f771f328465a3dd5a6333d34fb9aa4cb9052c8c7811ab2dbd17d088e"},{"shape":[1,14,128],"sha256":"2e108a4d5870102d8e2d4a9b69fe55507b0e75f12455792f2b05610fa305d638","value_sha256":"2e108a4d5870102d8e2d4a9b69fe55507b0e75f12455792f2b05610fa305d638"}],[{"shape":[1,3,10240],"sha256":"8e0affe3773cbb974718f2a1dce2e75f60cc29c8290617c559b5f93122ae763d","value_sha256":"8e0affe3773cbb974718f2a1dce2e75f60cc29c8290617c559b5f93122ae763d"},{"shape":[1,48,128,128],"sha256":"8293399e05778a26da5207d0dd00d1802ab2314ec846951bccdea731df48e8a7","value_sha256":"8293399e05778a26da5207d0dd00d1802ab2314ec846951bccdea731df48e8a7"}],[{"shape":[1,3,10240],"sha256":"7b9ed3fd1bf6ea7c486f62899e64152b9d24bc948a26fd02920f8ab823c077ea","value_sha256":"7b9ed3fd1bf6ea7c486f62899e64152b9d24bc948a26fd02920f8ab823c077ea"},{"shape":[1,48,128,128],"sha256":"8ee1a98bfc2a5d85fef8c57b94d166755b3766c0fa223c635dbdc013e512c810","value_sha256":"8ee1a98bfc2a5d85fef8c57b94d166755b3766c0fa223c635dbdc013e512c810"}],[{"shape":[1,3,10240],"sha256":"9daf0b317a2cd918d02808a7b7ffc304e4ca9217b12dc8921984c9e5dc4acb44","value_sha256":"9daf0b317a2cd918d02808a7b7ffc304e4ca9217b12dc8921984c9e5dc4acb44"},{"shape":[1,48,128,128],"sha256":"bb3e8804e600bc41b69b14e4f89339e876f975fcd8e4caad4318ef88e424fd85","value_sha256":"bb3e8804e600bc41b69b14e4f89339e876f975fcd8e4caad4318ef88e424fd85"}],[{"shape":[1,2,58,256],"sha256":"85ce674b27b84da8c85b6f50d944f7173c2fcfed8bb112cd518aa52fed92beea","value_sha256":"85ce674b27b84da8c85b6f50d944f7173c2fcfed8bb112cd518aa52fed92beea"},{"shape":[1,2,58,256],"sha256":"1a7b2de350076386b7499577e28d627164afe1b2e00f7a2fbbc1b1ea3dae35ab","value_sha256":"1a7b2de350076386b7499577e28d627164afe1b2e00f7a2fbbc1b1ea3dae35ab"},{"shape":[1,58,128],"sha256":"fa5d6207583355c27b91d17b333b6f672d62a64dbab9ee3f787619a70a924c07","value_sha256":"fa5d6207583355c27b91d17b333b6f672d62a64dbab9ee3f787619a70a924c07"},{"shape":[1,14,128],"sha256":"f1033fd4afbbe2909dada065e00234e58a3ad7a9d92eade7fd112c0adab4af24","value_sha256":"f1033fd4afbbe2909dada065e00234e58a3ad7a9d92eade7fd112c0adab4af24"}],[{"shape":[1,3,10240],"sha256":"95c15e85c7cbc93aade6e85ba021ba4a32383a7bd6e94cfa85d6376a2f1baff1","value_sha256":"95c15e85c7cbc93aade6e85ba021ba4a32383a7bd6e94cfa85d6376a2f1baff1"},{"shape":[1,48,128,128],"sha256":"8b5e6b8b2ca9f2d347a64581632db8fa98bf0ee2e0c5de6d8889964c6739b7d7","value_sha256":"8b5e6b8b2ca9f2d347a64581632db8fa98bf0ee2e0c5de6d8889964c6739b7d7"}],[{"shape":[1,3,10240],"sha256":"2509502b2ec7afecab230e5c532f1a034a8f0f42c2fafa57163c6da3ec308eed","value_sha256":"2509502b2ec7afecab230e5c532f1a034a8f0f42c2fafa57163c6da3ec308eed"},{"shape":[1,48,128,128],"sha256":"e73f022d9beaba67a5ebd2b54f0ac2ab982323756bc0876489c2256d5685e744","value_sha256":"e73f022d9beaba67a5ebd2b54f0ac2ab982323756bc0876489c2256d5685e744"}],[{"shape":[1,3,10240],"sha256":"46817f7782dd86d4c6375cfd76bf18b6a0661ba43ff7a4360040729f076b8b81","value_sha256":"46817f7782dd86d4c6375cfd76bf18b6a0661ba43ff7a4360040729f076b8b81"},{"shape":[1,48,128,128],"sha256":"d4dd6ee143fc4bcc68d6b60c80953207082cbbc366f598a6f8cf7a00932dcb14","value_sha256":"d4dd6ee143fc4bcc68d6b60c80953207082cbbc366f598a6f8cf7a00932dcb14"}],[{"shape":[1,2,58,256],"sha256":"6a03bc8cfe966b40da35c7cc0d97190aa7572994ce59d4a5e023b927d166e51e","value_sha256":"6a03bc8cfe966b40da35c7cc0d97190aa7572994ce59d4a5e023b927d166e51e"},{"shape":[1,2,58,256],"sha256":"f7e76be69fb49db9993cea131269476cf55502affa51295f51d53d788d0eae72","value_sha256":"f7e76be69fb49db9993cea131269476cf55502affa51295f51d53d788d0eae72"},{"shape":[1,58,128],"sha256":"238df2964205645bc0012bd0588e233c099c4c2408d141168e239a4224e14e04","value_sha256":"238df2964205645bc0012bd0588e233c099c4c2408d141168e239a4224e14e04"},{"shape":[1,14,128],"sha256":"724418b117e12a9d59dba8f43c3bef4ee800e5322e4c0dd8fe1b8bfcc5defee4","value_sha256":"724418b117e12a9d59dba8f43c3bef4ee800e5322e4c0dd8fe1b8bfcc5defee4"}],[{"shape":[1,3,10240],"sha256":"4531ea10269697248e743041130a121ac1f566039470cbda939dd1f079736e0f","value_sha256":"4531ea10269697248e743041130a121ac1f566039470cbda939dd1f079736e0f"},{"shape":[1,48,128,128],"sha256":"70096b5b2f0dd778bb52980ea38cb6ce2b94749c34a9aa3cfc539b5b89366025","value_sha256":"70096b5b2f0dd778bb52980ea38cb6ce2b94749c34a9aa3cfc539b5b89366025"}],[{"shape":[1,3,10240],"sha256":"9c3a219512060c453885f0ed299d7be1618048969f0f94bd14764b464713db39","value_sha256":"9c3a219512060c453885f0ed299d7be1618048969f0f94bd14764b464713db39"},{"shape":[1,48,128,128],"sha256":"822028dfd9c7160f6fceee8551486db596349e2aa57def487716fd3254c64635","value_sha256":"822028dfd9c7160f6fceee8551486db596349e2aa57def487716fd3254c64635"}],[{"shape":[1,3,10240],"sha256":"df10516aa1338cefae8b842fdb557a2027f9aa1957577ab7750f46b5f86c54af","value_sha256":"df10516aa1338cefae8b842fdb557a2027f9aa1957577ab7750f46b5f86c54af"},{"shape":[1,48,128,128],"sha256":"36241df4f65de72b6050bde3b46196ba996c835aaedaf94ad00e3b3868204ad6","value_sha256":"36241df4f65de72b6050bde3b46196ba996c835aaedaf94ad00e3b3868204ad6"}],[{"shape":[1,2,58,256],"sha256":"69cc83b8890a2ea9a636d4546ce3ed3d8759b9d0a15477e4c8badc431e4c7fe8","value_sha256":"69cc83b8890a2ea9a636d4546ce3ed3d8759b9d0a15477e4c8badc431e4c7fe8"},{"shape":[1,2,58,256],"sha256":"ddd32a1c551437c14edd919cb138578fdc1c5a4a62faeb0faa0c771c27687b38","value_sha256":"ddd32a1c551437c14edd919cb138578fdc1c5a4a62faeb0faa0c771c27687b38"},{"shape":[1,58,128],"sha256":"ba4978b3867a9ef49ca295fd75e39f42a6ab4d5e22a0105578e324506bb469dc","value_sha256":"ba4978b3867a9ef49ca295fd75e39f42a6ab4d5e22a0105578e324506bb469dc"},{"shape":[1,14,128],"sha256":"5664914f31c67f9c2eb8e84c91ba326de6b9a520408b13bec1591b7a9c99d405","value_sha256":"5664914f31c67f9c2eb8e84c91ba326de6b9a520408b13bec1591b7a9c99d405"}],[{"shape":[1,3,10240],"sha256":"39dbb01b30f070663709cb9b93662699270bbc46fc7c72ea6e9d79899d17c215","value_sha256":"39dbb01b30f070663709cb9b93662699270bbc46fc7c72ea6e9d79899d17c215"},{"shape":[1,48,128,128],"sha256":"148a32eac2abf5d7891bdff58650dfb116a5a7c4d59cc44b6b95cdf1ac99e18f","value_sha256":"148a32eac2abf5d7891bdff58650dfb116a5a7c4d59cc44b6b95cdf1ac99e18f"}],[{"shape":[1,3,10240],"sha256":"e06ef379ef9e77086ebffeec6a8b7739a9534c853e06be61daeb4280665dec79","value_sha256":"e06ef379ef9e77086ebffeec6a8b7739a9534c853e06be61daeb4280665dec79"},{"shape":[1,48,128,128],"sha256":"0522563654646539e1e80e0c8f29f6c3145a2e9d5e90d6326e19e3d521474376","value_sha256":"0522563654646539e1e80e0c8f29f6c3145a2e9d5e90d6326e19e3d521474376"}],[{"shape":[1,3,10240],"sha256":"e4dd86d4eeb57ac1afe7f8f4e2f84735fca091939861e3fbe96eff76d2c098ac","value_sha256":"e4dd86d4eeb57ac1afe7f8f4e2f84735fca091939861e3fbe96eff76d2c098ac"},{"shape":[1,48,128,128],"sha256":"bfcbb36270a75ae6c09fddf04b6ca00c419408378aba21ff2236284ebccba433","value_sha256":"bfcbb36270a75ae6c09fddf04b6ca00c419408378aba21ff2236284ebccba433"}],[{"shape":[1,2,58,256],"sha256":"0d374a75bb1a9563bc20f3d4c4b884807e2305c2b14ef36e26c0f342101a0fe3","value_sha256":"0d374a75bb1a9563bc20f3d4c4b884807e2305c2b14ef36e26c0f342101a0fe3"},{"shape":[1,2,58,256],"sha256":"99f0b58866d5217b257e4147c8adf3cb6975c69e21f11b4e927edfc8c0b02f4e","value_sha256":"99f0b58866d5217b257e4147c8adf3cb6975c69e21f11b4e927edfc8c0b02f4e"},{"shape":[1,58,128],"sha256":"0ab3ab6cd8923167204f8f39a6a12d75d9e247243bff4414798dc634eecb265b","value_sha256":"0ab3ab6cd8923167204f8f39a6a12d75d9e247243bff4414798dc634eecb265b"},{"shape":[1,14,128],"sha256":"6ad6f2c1bd2b1d84d2df923a71f822e0d2a9ba91298ecc8235a872a1ff02a1e5","value_sha256":"6ad6f2c1bd2b1d84d2df923a71f822e0d2a9ba91298ecc8235a872a1ff02a1e5"}],[{"shape":[1,3,10240],"sha256":"d23b61a1a941a4df10f433a664061ace644e7182f98a44ba0109dd569e67c886","value_sha256":"d23b61a1a941a4df10f433a664061ace644e7182f98a44ba0109dd569e67c886"},{"shape":[1,48,128,128],"sha256":"ec5f8e87caa2700663046ef72156240c54add5f7a9754f751fafc1d53c4dcfaf","value_sha256":"ec5f8e87caa2700663046ef72156240c54add5f7a9754f751fafc1d53c4dcfaf"}],[{"shape":[1,3,10240],"sha256":"13d6ca0d90f0064d0241e25cbada81e70f15b39816a4903e2e9df950df50b756","value_sha256":"13d6ca0d90f0064d0241e25cbada81e70f15b39816a4903e2e9df950df50b756"},{"shape":[1,48,128,128],"sha256":"83a250a7f63996f11a75a100d09b90bd519de318646ba9aff097c21f7e715629","value_sha256":"83a250a7f63996f11a75a100d09b90bd519de318646ba9aff097c21f7e715629"}],[{"shape":[1,3,10240],"sha256":"881d586c399fc7adcb5f6a41619b2c7e93bdd056a09dcfa407c51d64e0e1b976","value_sha256":"881d586c399fc7adcb5f6a41619b2c7e93bdd056a09dcfa407c51d64e0e1b976"},{"shape":[1,48,128,128],"sha256":"8f40c358d8d8e2dbe72dc04035ad9e03a296e84b32f82244b43d5000f6b77949","value_sha256":"8f40c358d8d8e2dbe72dc04035ad9e03a296e84b32f82244b43d5000f6b77949"}],[{"shape":[1,2,58,256],"sha256":"700d59adcb347511033e6cc84b88692e2138b2d79391a34fff7bb79e98fdaced","value_sha256":"700d59adcb347511033e6cc84b88692e2138b2d79391a34fff7bb79e98fdaced"},{"shape":[1,2,58,256],"sha256":"45090dc2a4a48cce5a8749876331ae4b5f6c52c4c78e0195fc45ecfda3521ae1","value_sha256":"45090dc2a4a48cce5a8749876331ae4b5f6c52c4c78e0195fc45ecfda3521ae1"},{"shape":[1,58,128],"sha256":"7c3616d29fa3a5e38f2d65e75b4a3b60cbe5673428895c1706cbbe8dc3e1b3b4","value_sha256":"7c3616d29fa3a5e38f2d65e75b4a3b60cbe5673428895c1706cbbe8dc3e1b3b4"},{"shape":[1,14,128],"sha256":"490cfaf1631904b133dcb1ef6639af8361a18dc44ff6d16e467a194b57594a2b","value_sha256":"490cfaf1631904b133dcb1ef6639af8361a18dc44ff6d16e467a194b57594a2b"}],[{"shape":[1,3,10240],"sha256":"395b24b1c8561ad79de90d370b614b12e3fb3becb32f305dc4552f7bcd6dcd15","value_sha256":"395b24b1c8561ad79de90d370b614b12e3fb3becb32f305dc4552f7bcd6dcd15"},{"shape":[1,48,128,128],"sha256":"88f8113cc2c0314be162c3118c750aa66e791daaa18e305a7a5411031fd66a71","value_sha256":"88f8113cc2c0314be162c3118c750aa66e791daaa18e305a7a5411031fd66a71"}],[{"shape":[1,3,10240],"sha256":"3a54a9e444aa715b7afcf7aee09994a648951b006e4d48789374e6ef3c12889e","value_sha256":"3a54a9e444aa715b7afcf7aee09994a648951b006e4d48789374e6ef3c12889e"},{"shape":[1,48,128,128],"sha256":"e13d5286974864fb579dd5dd4f16cc972824a78f0d9d7513db33aaccf4902e4d","value_sha256":"e13d5286974864fb579dd5dd4f16cc972824a78f0d9d7513db33aaccf4902e4d"}],[{"shape":[1,3,10240],"sha256":"1bb2a69a6651a97d6f34910b5b30fcb93cfc5eb4038ee68122a76c1d38e09132","value_sha256":"1bb2a69a6651a97d6f34910b5b30fcb93cfc5eb4038ee68122a76c1d38e09132"},{"shape":[1,48,128,128],"sha256":"728437ce6a32dfa4a9352266b9dae2efdc327b6e9298ff2cd04f5bdde1ceac9e","value_sha256":"728437ce6a32dfa4a9352266b9dae2efdc327b6e9298ff2cd04f5bdde1ceac9e"}],[{"shape":[1,2,58,256],"sha256":"6a1c56e7333fc86e4e11bb265e99274ff727408f9ab32508f374198d4e8553c7","value_sha256":"44b87b49e9f9853e6a3cb5efcefe1f25ff4f06d56e247847d0054de3aab3fab4"},{"shape":[1,2,58,256],"sha256":"58937333f6ee9a30561802d7d287caee8d40fe5b481d31f80f694aab76bc8d42","value_sha256":"58937333f6ee9a30561802d7d287caee8d40fe5b481d31f80f694aab76bc8d42"},{"shape":[1,58,128],"sha256":"e1a78a7e3c30a628023848a71912783754128437f634aa121eea20f0a5eb27e1","value_sha256":"e1a78a7e3c30a628023848a71912783754128437f634aa121eea20f0a5eb27e1"},{"shape":[1,14,128],"sha256":"b7c598edf1ad579211d48d3f6d105202ed1266cbd42205df9ab794398adb2729","value_sha256":"b7c598edf1ad579211d48d3f6d105202ed1266cbd42205df9ab794398adb2729"}],[{"shape":[1,3,10240],"sha256":"653c93430930c95e053e5e75e3068dbab3bb4729fc49d82ba378792a07e5d8b3","value_sha256":"653c93430930c95e053e5e75e3068dbab3bb4729fc49d82ba378792a07e5d8b3"},{"shape":[1,48,128,128],"sha256":"03681ea450a4a1734cdf9e4e4fdf3253a8fbf551de70f3e45100f423138ef20a","value_sha256":"03681ea450a4a1734cdf9e4e4fdf3253a8fbf551de70f3e45100f423138ef20a"}],[{"shape":[1,3,10240],"sha256":"13d0f0882c67b19ae78bb751c2104a64e1d2ec5e9ca397f61cb9eb50045ac66b","value_sha256":"13d0f0882c67b19ae78bb751c2104a64e1d2ec5e9ca397f61cb9eb50045ac66b"},{"shape":[1,48,128,128],"sha256":"6ae4fc5e4b5a19ec13699c31e60784e7c8a0c79c2894a1967de2bcb6d24062e4","value_sha256":"6ae4fc5e4b5a19ec13699c31e60784e7c8a0c79c2894a1967de2bcb6d24062e4"}],[{"shape":[1,3,10240],"sha256":"ba4f5a7d79b0b134d529e94568049e441e129e4239274d6580790b86930c5f13","value_sha256":"ba4f5a7d79b0b134d529e94568049e441e129e4239274d6580790b86930c5f13"},{"shape":[1,48,128,128],"sha256":"1c7295715f726120f544708050a1e952a543a2e75ffe1b6c6317eba5be594e1e","value_sha256":"1c7295715f726120f544708050a1e952a543a2e75ffe1b6c6317eba5be594e1e"}],[{"shape":[1,2,58,256],"sha256":"76c1c37db1e5e29e0ca1cf9d3d2a83abb5912b89f1a780dd583280bcb0b72834","value_sha256":"76c1c37db1e5e29e0ca1cf9d3d2a83abb5912b89f1a780dd583280bcb0b72834"},{"shape":[1,2,58,256],"sha256":"406cd8e26e9fdc3980517753308f22ef96cfc5f6e594be321693f1735a4649b2","value_sha256":"406cd8e26e9fdc3980517753308f22ef96cfc5f6e594be321693f1735a4649b2"},{"shape":[1,58,128],"sha256":"d9455de2858b3e7ece5df9eea9ca961328acaa359abe4f9e49053143f443aec6","value_sha256":"d9455de2858b3e7ece5df9eea9ca961328acaa359abe4f9e49053143f443aec6"},{"shape":[1,14,128],"sha256":"de9cc43b6b076d56c23fa89efe8ae677a61dcec872d2dbbe35e332dad6567f9e","value_sha256":"de9cc43b6b076d56c23fa89efe8ae677a61dcec872d2dbbe35e332dad6567f9e"}],[{"shape":[1,3,10240],"sha256":"5f043e9fb8a29117b711c29c68b232e69e03cf6e3103064bd6ab67a74285235e","value_sha256":"5f043e9fb8a29117b711c29c68b232e69e03cf6e3103064bd6ab67a74285235e"},{"shape":[1,48,128,128],"sha256":"30fee423125f5f9a182f01f200c31e474b2a626ccafdf069a4f4548129d2d1ff","value_sha256":"30fee423125f5f9a182f01f200c31e474b2a626ccafdf069a4f4548129d2d1ff"}],[{"shape":[1,3,10240],"sha256":"4f3c8c2688a9660bb254051656b2ba771a38acad5fb7f7fe46fdd21457a857c8","value_sha256":"4f3c8c2688a9660bb254051656b2ba771a38acad5fb7f7fe46fdd21457a857c8"},{"shape":[1,48,128,128],"sha256":"067f420ad6951dfff064c2a2d26537074cf1b8a6fa3850a5c80a78677503f890","value_sha256":"067f420ad6951dfff064c2a2d26537074cf1b8a6fa3850a5c80a78677503f890"}],[{"shape":[1,3,10240],"sha256":"bb73667194e07aae5cc3d30fa2fbc15cf921c78d7016ae27bfc97a8bb0cc09f7","value_sha256":"bb73667194e07aae5cc3d30fa2fbc15cf921c78d7016ae27bfc97a8bb0cc09f7"},{"shape":[1,48,128,128],"sha256":"98837a4082cf48204396c03042b18b84c6bafc6b398ebd6852f45af660c0f0d3","value_sha256":"98837a4082cf48204396c03042b18b84c6bafc6b398ebd6852f45af660c0f0d3"}],[{"shape":[1,2,58,256],"sha256":"325b425383bd9a7a6e914af7e54cc595a14d9987cbe7c308c96ed5beb6a8a63f","value_sha256":"325b425383bd9a7a6e914af7e54cc595a14d9987cbe7c308c96ed5beb6a8a63f"},{"shape":[1,2,58,256],"sha256":"05ec6026468751e837c9b2f129b6b3330827d6102526c09fcbfb9a87b49d0560","value_sha256":"05ec6026468751e837c9b2f129b6b3330827d6102526c09fcbfb9a87b49d0560"},{"shape":[1,58,128],"sha256":"3bbef13d2f478d4def15e6b15869d999e09cf07f02a499af24066784a72b9edb","value_sha256":"3bbef13d2f478d4def15e6b15869d999e09cf07f02a499af24066784a72b9edb"},{"shape":[1,14,128],"sha256":"1abcb54411b56c34a0cd0bfa4ca920256c61bd50fcff14a2140d12de6e6cf348","value_sha256":"1abcb54411b56c34a0cd0bfa4ca920256c61bd50fcff14a2140d12de6e6cf348"}],[{"shape":[1,3,10240],"sha256":"3e3ffd492867d8445bc1b9b42b426565db732e5add86a85a24041e1aed1d50b5","value_sha256":"3e3ffd492867d8445bc1b9b42b426565db732e5add86a85a24041e1aed1d50b5"},{"shape":[1,48,128,128],"sha256":"dafa3585c9acccb5914ade9251e5533df9ffd0f05380a2e99ad342041dcc0b98","value_sha256":"dafa3585c9acccb5914ade9251e5533df9ffd0f05380a2e99ad342041dcc0b98"}],[{"shape":[1,3,10240],"sha256":"c20f1e23c969c6e7f59c8fa7781d868a2b19aa6ae0b66cda658cdcd29a68cda2","value_sha256":"c20f1e23c969c6e7f59c8fa7781d868a2b19aa6ae0b66cda658cdcd29a68cda2"},{"shape":[1,48,128,128],"sha256":"a0dde68ef7790d55314dc9db6e547a886b8d6870ebcf84eb4662d89f128d6b4a","value_sha256":"a0dde68ef7790d55314dc9db6e547a886b8d6870ebcf84eb4662d89f128d6b4a"}],[{"shape":[1,3,10240],"sha256":"ec38527615247d9ca522db4f8fcfeb558bf9a234b07e8799b290e305c69bcce6","value_sha256":"ec38527615247d9ca522db4f8fcfeb558bf9a234b07e8799b290e305c69bcce6"},{"shape":[1,48,128,128],"sha256":"779f4cfcb1fd123814363768cce1a803e1a8c0f01c7ec304df93ad568f623463","value_sha256":"779f4cfcb1fd123814363768cce1a803e1a8c0f01c7ec304df93ad568f623463"}],[{"shape":[1,2,58,256],"sha256":"6ab15b2e1edadf080918d1c74f5616d540039bb1ab0f5ced4dafd8e2a4a66da4","value_sha256":"6ab15b2e1edadf080918d1c74f5616d540039bb1ab0f5ced4dafd8e2a4a66da4"},{"shape":[1,2,58,256],"sha256":"3f056e38b54925776587ff6144c3fb7240f43b28691697cdf6643fb1d848892d","value_sha256":"3f056e38b54925776587ff6144c3fb7240f43b28691697cdf6643fb1d848892d"},{"shape":[1,58,128],"sha256":"c2a0dcb5ad4087827d26739fa0a0296de5df94b3764e02b603da06014979c555","value_sha256":"c2a0dcb5ad4087827d26739fa0a0296de5df94b3764e02b603da06014979c555"},{"shape":[1,14,128],"sha256":"525e7e0ca6455d6931c5c945dd43e90aaf4dabf2d02a0d52a60d02006cefb2d8","value_sha256":"525e7e0ca6455d6931c5c945dd43e90aaf4dabf2d02a0d52a60d02006cefb2d8"}]],"mtp":[[{"shape":[1,2,57,256],"sha256":"7f1debf8246bc37d767a9eca4263fd440b2e2e261e4aec689bfb7890a626beaf","value_sha256":"7f1debf8246bc37d767a9eca4263fd440b2e2e261e4aec689bfb7890a626beaf"},{"shape":[1,2,57,256],"sha256":"0bb507525d4209dfac234f774b301f6d154195afd5c78fd7d0fdad55b715008f","value_sha256":"0bb507525d4209dfac234f774b301f6d154195afd5c78fd7d0fdad55b715008f"},{"shape":[1,57,128],"sha256":"2f47eb6af4a4c2d8442aa1322a1d910bcd90ce9e8006ad19d1dd6d4bfeb6d453","value_sha256":"2f47eb6af4a4c2d8442aa1322a1d910bcd90ce9e8006ad19d1dd6d4bfeb6d453"},{"shape":[1,14,128],"sha256":"1ecb828fa7a6f7528b38d75459ef4e79a81089b1adf01319d3bbe58b3b93fecd","value_sha256":"1ecb828fa7a6f7528b38d75459ef4e79a81089b1adf01319d3bbe58b3b93fecd"}]]}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/cold.json b/docs/issue113-head-chunk/off-product/cold.json
new file mode 100644
index 0000000..012b13e
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/cold.json
@@ -0,0 +1 @@
+{"case":"cold","prompt":[248045,8678,198,24342,286,4879,369,716,310,3238,13,13262,678,7047,9522,321,10419,11,6992,5774,310,279,16198,1973,24366,24150,362,13,248046,198,248045,846,198,2427,494,799,310,5600,13,248046,198,248045,74455,198,248068,198],"tokens":[760,1156,6587,728,310,1698,494,799,310,5600,13,1061],"cached":0,"state_sha256":"eb61c20e4a89bb55c0284d86587966af49d6c601e5421b50eff0762ecf6842b1"}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/cold/0.json b/docs/issue113-head-chunk/off-product/cold/0.json
new file mode 100644
index 0000000..007148a
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/cold/0.json
@@ -0,0 +1 @@
+{"index":0,"ids":[760,1156,369,9859],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"16ff543d6e28cdbe7dbd56891e061ddd4b04291a9ca25bdf1f4437b4e3c5528d"},"hidden":{"shape":[1,4,10240],"sha256":"b9cc3aa8c913e508793b7486f77a10e7acd4698436060b1b6bf8abfbd59edee6"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/cold/1.json b/docs/issue113-head-chunk/off-product/cold/1.json
new file mode 100644
index 0000000..4c07aa3
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/cold/1.json
@@ -0,0 +1 @@
+{"index":1,"ids":[6587,728,310,1698],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"24e58b3561aac59d52bf55183bc9b911432481e9613a5b98f1442945bfd6903a"},"hidden":{"shape":[1,4,10240],"sha256":"eddb5de99ea61034c3f0eb46ef1a515cae10c92333d069afce68cc4a5e5f9e3e"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/cold/2.json b/docs/issue113-head-chunk/off-product/cold/2.json
new file mode 100644
index 0000000..dffac93
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/cold/2.json
@@ -0,0 +1 @@
+{"index":2,"ids":[494,799,310,5600],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"f5f2c33a36fce4088f222ec7448389865dd74ea4c017e4e2e49f157693a33d64"},"hidden":{"shape":[1,4,10240],"sha256":"330d68ed62e77a12e38fe7677ffd2b9548f655c7d33e6e1607e431d1b8957e86"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/cold/3.json b/docs/issue113-head-chunk/off-product/cold/3.json
new file mode 100644
index 0000000..290e3f6
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/cold/3.json
@@ -0,0 +1 @@
+{"index":3,"ids":[13,1061],"verify":true,"logits":{"shape":[1,2,248320],"sha256":"f44498bbf6e6e9d8d909c0e4db24b5ef9bda5866b4a6a14e188865432538643c"},"hidden":{"shape":[1,2,10240],"sha256":"dec8b787a562878d091c854235120732e98f5781751b6d6a428e5cd02228deb0"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/reopened-arrays.json b/docs/issue113-head-chunk/off-product/reopened-arrays.json
new file mode 100644
index 0000000..df8962a
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/reopened-arrays.json
@@ -0,0 +1 @@
+{"trunk":[[{"shape":[1,3,10240],"sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84","value_sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84"},{"shape":[1,48,128,128],"sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf","value_sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf"}],[{"shape":[1,3,10240],"sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232","value_sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232"},{"shape":[1,48,128,128],"sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf","value_sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf"},{"shape":[1,9,10240],"sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08","value_sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08"},{"shape":[1,2],"sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152","value_sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152"}],[{"shape":[1,3,10240],"sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2","value_sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2"},{"shape":[1,48,128,128],"sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34","value_sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34"}],[{"shape":[1,2,88,256],"sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f","value_sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f"},{"shape":[1,2,88,256],"sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24","value_sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24"},{"shape":[1,88,128],"sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748","value_sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748"},{"shape":[1,22,128],"sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a","value_sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a"}],[{"shape":[1,3,10240],"sha256":"431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda","value_sha256":"431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda"},{"shape":[1,48,128,128],"sha256":"c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f","value_sha256":"c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f"}],[{"shape":[1,3,10240],"sha256":"f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a","value_sha256":"f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a"},{"shape":[1,48,128,128],"sha256":"8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f","value_sha256":"8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f"}],[{"shape":[1,3,10240],"sha256":"3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d","value_sha256":"3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d"},{"shape":[1,48,128,128],"sha256":"bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f","value_sha256":"bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f"}],[{"shape":[1,2,88,256],"sha256":"9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b","value_sha256":"9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b"},{"shape":[1,2,88,256],"sha256":"b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba","value_sha256":"b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba"},{"shape":[1,88,128],"sha256":"491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae","value_sha256":"491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae"},{"shape":[1,22,128],"sha256":"104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb","value_sha256":"104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb"}],[{"shape":[1,3,10240],"sha256":"e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57","value_sha256":"e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57"},{"shape":[1,48,128,128],"sha256":"0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43","value_sha256":"0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43"}],[{"shape":[1,3,10240],"sha256":"f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0","value_sha256":"f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0"},{"shape":[1,48,128,128],"sha256":"75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52","value_sha256":"75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52"}],[{"shape":[1,3,10240],"sha256":"c23b7e524522714aaf620c9e0bbb6b1d3db3585971f0cffafd7fe7e2018176dc","value_sha256":"c23b7e524522714aaf620c9e0bbb6b1d3db3585971f0cffafd7fe7e2018176dc"},{"shape":[1,48,128,128],"sha256":"4ce5d0905ec5e5ae834215c16b25d810a5ef6e54a6d7d96f95e477209775ddaa","value_sha256":"4ce5d0905ec5e5ae834215c16b25d810a5ef6e54a6d7d96f95e477209775ddaa"}],[{"shape":[1,2,88,256],"sha256":"7803bca47639bdcb533d5985f37930fb74cd112882f8cfa26513bc297e1245fe","value_sha256":"7803bca47639bdcb533d5985f37930fb74cd112882f8cfa26513bc297e1245fe"},{"shape":[1,2,88,256],"sha256":"40bb50faa5d147165f88cd5ba2379dc395f4f296c66246499b9efd03c9eef7ac","value_sha256":"40bb50faa5d147165f88cd5ba2379dc395f4f296c66246499b9efd03c9eef7ac"},{"shape":[1,88,128],"sha256":"6e20787bdd02bd9615279eac42cf9f2664430aeb8cdd8c93c6a9881556722493","value_sha256":"6e20787bdd02bd9615279eac42cf9f2664430aeb8cdd8c93c6a9881556722493"},{"shape":[1,22,128],"sha256":"c17459f225244530703a28076f3006f7e53d50e728ecb79ae4f5b968b594d96d","value_sha256":"c17459f225244530703a28076f3006f7e53d50e728ecb79ae4f5b968b594d96d"}],[{"shape":[1,3,10240],"sha256":"8e1e94ade5f7453693e4d99f003d190122a3cd69b0bddca1a1fc2409d9a32b7d","value_sha256":"8e1e94ade5f7453693e4d99f003d190122a3cd69b0bddca1a1fc2409d9a32b7d"},{"shape":[1,48,128,128],"sha256":"976efc317438cb07b1e6d80ce6ae12d8e0095a51fa7b490b8a84a8e0d6b79399","value_sha256":"976efc317438cb07b1e6d80ce6ae12d8e0095a51fa7b490b8a84a8e0d6b79399"}],[{"shape":[1,3,10240],"sha256":"6153601aa51b613aa617cb823b2d5e582847afba59eac2c190a96ea465ad77e1","value_sha256":"6153601aa51b613aa617cb823b2d5e582847afba59eac2c190a96ea465ad77e1"},{"shape":[1,48,128,128],"sha256":"077106a1cc1cf3e3d24ba44378f788cab30d80fd38aef87fcf957a99b662379d","value_sha256":"077106a1cc1cf3e3d24ba44378f788cab30d80fd38aef87fcf957a99b662379d"}],[{"shape":[1,3,10240],"sha256":"d020e70e7cab233eb7d188fa42e975f0a20cdce167f52db6043933e583f7c280","value_sha256":"d020e70e7cab233eb7d188fa42e975f0a20cdce167f52db6043933e583f7c280"},{"shape":[1,48,128,128],"sha256":"c05cdc62e13b7c238ad02e2a4a81286dc5d0fd8cd9c91af6c8ea6380d203f080","value_sha256":"c05cdc62e13b7c238ad02e2a4a81286dc5d0fd8cd9c91af6c8ea6380d203f080"}],[{"shape":[1,2,88,256],"sha256":"be1eed7cb68714c49599bea49ffdc831b5caac8444781088d21c87892445d2d7","value_sha256":"be1eed7cb68714c49599bea49ffdc831b5caac8444781088d21c87892445d2d7"},{"shape":[1,2,88,256],"sha256":"abe8a391f5a932dee7718ea3abe5419e015750b85a522856f1a59f435d5688dc","value_sha256":"abe8a391f5a932dee7718ea3abe5419e015750b85a522856f1a59f435d5688dc"},{"shape":[1,88,128],"sha256":"c1cd1549e2032d2bc28d2d0c9f1bfeb608f81fe8abe2b18b77a1658a4c0ed9c4","value_sha256":"c1cd1549e2032d2bc28d2d0c9f1bfeb608f81fe8abe2b18b77a1658a4c0ed9c4"},{"shape":[1,22,128],"sha256":"072bc306a1b4a56332dc90d889c42e6b2037a820330da0b897815a3750c00043","value_sha256":"072bc306a1b4a56332dc90d889c42e6b2037a820330da0b897815a3750c00043"}],[{"shape":[1,3,10240],"sha256":"7f4500e05eb6e52b25bd9405ccaae398fcc426835af0550599175d0785694154","value_sha256":"7f4500e05eb6e52b25bd9405ccaae398fcc426835af0550599175d0785694154"},{"shape":[1,48,128,128],"sha256":"e33affece02b985e41ea0d4ce467679226e9f9b5f55c4001cdbf8ebcb087db40","value_sha256":"e33affece02b985e41ea0d4ce467679226e9f9b5f55c4001cdbf8ebcb087db40"}],[{"shape":[1,3,10240],"sha256":"d133a043fa3abeaf2498e4336142e34a6d3222e4b2eec943931e97eb060f5657","value_sha256":"d133a043fa3abeaf2498e4336142e34a6d3222e4b2eec943931e97eb060f5657"},{"shape":[1,48,128,128],"sha256":"465ca838c8b5abbaac2b32a5f2f204ebe674d20ff50ed9b3532e81ede40c1164","value_sha256":"465ca838c8b5abbaac2b32a5f2f204ebe674d20ff50ed9b3532e81ede40c1164"}],[{"shape":[1,3,10240],"sha256":"379bfa2a27e16bc7d2d1213c0d145307954a9d4d36d8c5f5a6cb9b4f172b400c","value_sha256":"379bfa2a27e16bc7d2d1213c0d145307954a9d4d36d8c5f5a6cb9b4f172b400c"},{"shape":[1,48,128,128],"sha256":"be38588e2a1191fdccc176952eb809d41b8c40d874bc9f160adae245abe93015","value_sha256":"be38588e2a1191fdccc176952eb809d41b8c40d874bc9f160adae245abe93015"}],[{"shape":[1,2,88,256],"sha256":"c3c637457aeabb3d95909c7fd35e0a4024aef118af9dec6b369a50bf409cc735","value_sha256":"c3c637457aeabb3d95909c7fd35e0a4024aef118af9dec6b369a50bf409cc735"},{"shape":[1,2,88,256],"sha256":"efc834d804fa9f0958e2323b5c77dc22e134ee9130e211e67cf46e713fc303d1","value_sha256":"efc834d804fa9f0958e2323b5c77dc22e134ee9130e211e67cf46e713fc303d1"},{"shape":[1,88,128],"sha256":"3b59d52c473c7b1710f17b8672fd1ad1611d201e357a5a61e2f4272194654d0e","value_sha256":"3b59d52c473c7b1710f17b8672fd1ad1611d201e357a5a61e2f4272194654d0e"},{"shape":[1,22,128],"sha256":"ba538ce500987345986122428d8884b73b26e0c94466e3e5651d725a1e9c6623","value_sha256":"ba538ce500987345986122428d8884b73b26e0c94466e3e5651d725a1e9c6623"}],[{"shape":[1,3,10240],"sha256":"09026ef1a4aeb90aeeba7b23b21453e76852d97c609e40a04ecbd40e1c4f387d","value_sha256":"09026ef1a4aeb90aeeba7b23b21453e76852d97c609e40a04ecbd40e1c4f387d"},{"shape":[1,48,128,128],"sha256":"1b34b1e411aed8f13f9416722c35131648f0fb7752a1b0fdbc43ad56a3b089f8","value_sha256":"1b34b1e411aed8f13f9416722c35131648f0fb7752a1b0fdbc43ad56a3b089f8"}],[{"shape":[1,3,10240],"sha256":"a0b8dfe443b66d38cfc29af210f2461229d947985d56eecb00d35b8ead4a6758","value_sha256":"a0b8dfe443b66d38cfc29af210f2461229d947985d56eecb00d35b8ead4a6758"},{"shape":[1,48,128,128],"sha256":"23d008ba78432fc50967f3f9219912ecc50a745246b1562cd047304650910631","value_sha256":"23d008ba78432fc50967f3f9219912ecc50a745246b1562cd047304650910631"}],[{"shape":[1,3,10240],"sha256":"9b2393f3353a7c0c38906fa33cb081828590f7e095a047c32201ec4b171a3cc6","value_sha256":"9b2393f3353a7c0c38906fa33cb081828590f7e095a047c32201ec4b171a3cc6"},{"shape":[1,48,128,128],"sha256":"ea13bedee4d597e350a7b9d26965b06e996d3fb60a56c4a1a0823af6f56bdc29","value_sha256":"ea13bedee4d597e350a7b9d26965b06e996d3fb60a56c4a1a0823af6f56bdc29"}],[{"shape":[1,2,88,256],"sha256":"7f609422a62de424ecbbdbdfbe89a87e8a990103c3d3ddaf2b27213942b89c33","value_sha256":"7f609422a62de424ecbbdbdfbe89a87e8a990103c3d3ddaf2b27213942b89c33"},{"shape":[1,2,88,256],"sha256":"3f6d942ae01b4536df664a306d021a26e4c0c5103385baf2a39470f0f2498c08","value_sha256":"3f6d942ae01b4536df664a306d021a26e4c0c5103385baf2a39470f0f2498c08"},{"shape":[1,88,128],"sha256":"6f4ff9a35dd06bab89858195b4c10201258ee1d38361cd57fed9d47224aafffa","value_sha256":"6f4ff9a35dd06bab89858195b4c10201258ee1d38361cd57fed9d47224aafffa"},{"shape":[1,22,128],"sha256":"418e9f68c997dc6d3565ed3331b22edbededa67d22fedf1adf60ea913bd1102a","value_sha256":"418e9f68c997dc6d3565ed3331b22edbededa67d22fedf1adf60ea913bd1102a"}],[{"shape":[1,3,10240],"sha256":"71aa2645f2ce221cc2b5b0ecdb57cf7f511d8520b642f46f5656b35b3446c68b","value_sha256":"71aa2645f2ce221cc2b5b0ecdb57cf7f511d8520b642f46f5656b35b3446c68b"},{"shape":[1,48,128,128],"sha256":"27fe6db7a664bd3f479907384daed9c203d272d9d805680bf60f5823589522f5","value_sha256":"27fe6db7a664bd3f479907384daed9c203d272d9d805680bf60f5823589522f5"}],[{"shape":[1,3,10240],"sha256":"86f5a1132abfc21f4263d1e5781bd664bb5e0515c99c99262f5ece1309d31146","value_sha256":"86f5a1132abfc21f4263d1e5781bd664bb5e0515c99c99262f5ece1309d31146"},{"shape":[1,48,128,128],"sha256":"85d28d66cd18e5520f89273d1030d60aee4815d1979da53296ca6ebcb15eaea7","value_sha256":"85d28d66cd18e5520f89273d1030d60aee4815d1979da53296ca6ebcb15eaea7"}],[{"shape":[1,3,10240],"sha256":"0c0e755df21fddcc7c6c18f91ff0c28798c7f6db5d7ae3c1ccada82b75783c49","value_sha256":"0c0e755df21fddcc7c6c18f91ff0c28798c7f6db5d7ae3c1ccada82b75783c49"},{"shape":[1,48,128,128],"sha256":"a1f173e5880541f5db8e8c59075ec15f67229eb0a14e6fc1586201f4a5367d6e","value_sha256":"a1f173e5880541f5db8e8c59075ec15f67229eb0a14e6fc1586201f4a5367d6e"}],[{"shape":[1,2,88,256],"sha256":"67e00cb7fc89c35ad41c88339922d7ebed49abc11fcee6e7992daf4b79463f45","value_sha256":"67e00cb7fc89c35ad41c88339922d7ebed49abc11fcee6e7992daf4b79463f45"},{"shape":[1,2,88,256],"sha256":"c8c4c07315a42e3d3bcb61a33402a99b6b9f0b897ddddd682cca5ca9b3c5759b","value_sha256":"c8c4c07315a42e3d3bcb61a33402a99b6b9f0b897ddddd682cca5ca9b3c5759b"},{"shape":[1,88,128],"sha256":"13309057cb8436d0541acf49399a408b9035283a3e8ee00a36a2631d6feb9854","value_sha256":"13309057cb8436d0541acf49399a408b9035283a3e8ee00a36a2631d6feb9854"},{"shape":[1,22,128],"sha256":"c2bda4e0c3881a16568e45307b3907f5d9beaf96eddf085a560a767684525031","value_sha256":"c2bda4e0c3881a16568e45307b3907f5d9beaf96eddf085a560a767684525031"}],[{"shape":[1,3,10240],"sha256":"1e1db788f66c83daab3bc9f4ebf8e21e73297710a725207b9bc91530ac6ec073","value_sha256":"1e1db788f66c83daab3bc9f4ebf8e21e73297710a725207b9bc91530ac6ec073"},{"shape":[1,48,128,128],"sha256":"d42708090309446c157f583cdd98d184bbcf3d7582dcd56d232a279cacf53d03","value_sha256":"d42708090309446c157f583cdd98d184bbcf3d7582dcd56d232a279cacf53d03"}],[{"shape":[1,3,10240],"sha256":"c0afb77f8c6e3c141a0c0140224ec1a05f514331e76195185d74763ef6a3dcf1","value_sha256":"c0afb77f8c6e3c141a0c0140224ec1a05f514331e76195185d74763ef6a3dcf1"},{"shape":[1,48,128,128],"sha256":"d6a947a09b92b149efb68abe9f29fcce54471ab9c388aa7dbbd113c7e88f6275","value_sha256":"d6a947a09b92b149efb68abe9f29fcce54471ab9c388aa7dbbd113c7e88f6275"}],[{"shape":[1,3,10240],"sha256":"910b4e1eb82d9964f242034afd4420a420c37fb47b7ff344ba7b1d9a56e8762f","value_sha256":"910b4e1eb82d9964f242034afd4420a420c37fb47b7ff344ba7b1d9a56e8762f"},{"shape":[1,48,128,128],"sha256":"1144c0faae284f0059a58041a49152c2bb554f2936852bbd0a520f45098b936a","value_sha256":"1144c0faae284f0059a58041a49152c2bb554f2936852bbd0a520f45098b936a"}],[{"shape":[1,2,88,256],"sha256":"35cdbd0aded1c0ba01994a011832c2431488d440aa03f2f6123b9b84ddf1362d","value_sha256":"35cdbd0aded1c0ba01994a011832c2431488d440aa03f2f6123b9b84ddf1362d"},{"shape":[1,2,88,256],"sha256":"74166f058c8026611b093c3ca3629c2462ecd5c506b5da39a64e08436d597a3b","value_sha256":"74166f058c8026611b093c3ca3629c2462ecd5c506b5da39a64e08436d597a3b"},{"shape":[1,88,128],"sha256":"ec566fb641c4ba35da9341de02e170624a2da3616819e6a1cf4f4395454cc386","value_sha256":"ec566fb641c4ba35da9341de02e170624a2da3616819e6a1cf4f4395454cc386"},{"shape":[1,22,128],"sha256":"f2a66ce5e3986b883434ba9c91e8bbefc099bdc4011aada8ed89a4cefbc4cd16","value_sha256":"f2a66ce5e3986b883434ba9c91e8bbefc099bdc4011aada8ed89a4cefbc4cd16"}],[{"shape":[1,3,10240],"sha256":"a361723352d381d4456b0488309d7fc3a670b9557f0293da8323b1b1170b9c71","value_sha256":"a361723352d381d4456b0488309d7fc3a670b9557f0293da8323b1b1170b9c71"},{"shape":[1,48,128,128],"sha256":"19dd73c3b6450f2ae672d602046f8fdc159e7dc8d178befe72fe8c95829004a7","value_sha256":"19dd73c3b6450f2ae672d602046f8fdc159e7dc8d178befe72fe8c95829004a7"}],[{"shape":[1,3,10240],"sha256":"dd9b66e88449a34f1f346b367c5a689f72e2549a24622ad1e04876978c1a49f6","value_sha256":"dd9b66e88449a34f1f346b367c5a689f72e2549a24622ad1e04876978c1a49f6"},{"shape":[1,48,128,128],"sha256":"da16afcaa1e209ffb6f08639c70e56c3d3b3fe88449fa5700d4c2374f545ffc3","value_sha256":"da16afcaa1e209ffb6f08639c70e56c3d3b3fe88449fa5700d4c2374f545ffc3"}],[{"shape":[1,3,10240],"sha256":"8309772a5f9abc9a257f4dca60d98744cae411e8cd5b59f38132bb9a90c9f160","value_sha256":"8309772a5f9abc9a257f4dca60d98744cae411e8cd5b59f38132bb9a90c9f160"},{"shape":[1,48,128,128],"sha256":"f3a26845be01884066372bfb0d5b54f1e19c3927da4a84e59ce714a68b69ceaf","value_sha256":"f3a26845be01884066372bfb0d5b54f1e19c3927da4a84e59ce714a68b69ceaf"}],[{"shape":[1,2,88,256],"sha256":"061f8fe052fe21c8505782c220e03d09ab31deb93e7987709c55cd5dc3b778bd","value_sha256":"98186c7fec2498072f93b621a7f5b14456d4a67b0548a006353d87b4e4c2e89f"},{"shape":[1,2,88,256],"sha256":"e2c9efd039839aeec320ba45c64b4e8851e941b7300ef43b3b767d7a6fc3f81a","value_sha256":"e2c9efd039839aeec320ba45c64b4e8851e941b7300ef43b3b767d7a6fc3f81a"},{"shape":[1,88,128],"sha256":"4bf22f9cc812cbd24c68193dca9e687922f11cd4f14456157745987762e1a5d3","value_sha256":"4bf22f9cc812cbd24c68193dca9e687922f11cd4f14456157745987762e1a5d3"},{"shape":[1,22,128],"sha256":"be37a2a222ed9aa0bfffc831ce64c51d50adf8da9e8f865758f0ea199abed00c","value_sha256":"be37a2a222ed9aa0bfffc831ce64c51d50adf8da9e8f865758f0ea199abed00c"}],[{"shape":[1,3,10240],"sha256":"05517c814a7757ddd5a9576f70b54d74ae147035464fab8cbb9f85effa33f350","value_sha256":"05517c814a7757ddd5a9576f70b54d74ae147035464fab8cbb9f85effa33f350"},{"shape":[1,48,128,128],"sha256":"d97ab5b464249daf10022fb42384c16568ffafde0db8a75c209f6b9d569f1733","value_sha256":"d97ab5b464249daf10022fb42384c16568ffafde0db8a75c209f6b9d569f1733"}],[{"shape":[1,3,10240],"sha256":"405fddf03a32c3d17029d69ab68db301f89652a513e9d8a55f3b6aa5b7e6942d","value_sha256":"405fddf03a32c3d17029d69ab68db301f89652a513e9d8a55f3b6aa5b7e6942d"},{"shape":[1,48,128,128],"sha256":"0f195fc269e9c513e7e43dabce7c0446f259f576f329fcaf97b64f4596e4df20","value_sha256":"0f195fc269e9c513e7e43dabce7c0446f259f576f329fcaf97b64f4596e4df20"}],[{"shape":[1,3,10240],"sha256":"72d77520b60e0279c3e51cfd0be2dae0740a4c502279b4bc4e81de5f283cdbe5","value_sha256":"72d77520b60e0279c3e51cfd0be2dae0740a4c502279b4bc4e81de5f283cdbe5"},{"shape":[1,48,128,128],"sha256":"5397183714e9f7e4b1d5b55ef9ae553c1cffa2c8bab284425a2a4e701c7ff73c","value_sha256":"5397183714e9f7e4b1d5b55ef9ae553c1cffa2c8bab284425a2a4e701c7ff73c"}],[{"shape":[1,2,88,256],"sha256":"29bf2a7e968d01f9e5944a545bdbfa18d55a4f7be1f196c7bcb7e763d31facb7","value_sha256":"29bf2a7e968d01f9e5944a545bdbfa18d55a4f7be1f196c7bcb7e763d31facb7"},{"shape":[1,2,88,256],"sha256":"d0a7d3c46f6f41ecd2a0cb57be29f2c174bf8e4ad442d448773e819a8564e335","value_sha256":"d0a7d3c46f6f41ecd2a0cb57be29f2c174bf8e4ad442d448773e819a8564e335"},{"shape":[1,88,128],"sha256":"5331b2be56a2cb81895f7fa0f0c39d1696f7e32540e2c77e9d40c1e22d23ac8d","value_sha256":"5331b2be56a2cb81895f7fa0f0c39d1696f7e32540e2c77e9d40c1e22d23ac8d"},{"shape":[1,22,128],"sha256":"9cef226bd40e273913f9cbadc580e770da1479955964bb2903aa53d6476fa840","value_sha256":"9cef226bd40e273913f9cbadc580e770da1479955964bb2903aa53d6476fa840"}],[{"shape":[1,3,10240],"sha256":"30ce86b21de9483ec50ec7eabce466e9594ea6c98c9a8beb0afb83348d05e0eb","value_sha256":"30ce86b21de9483ec50ec7eabce466e9594ea6c98c9a8beb0afb83348d05e0eb"},{"shape":[1,48,128,128],"sha256":"3b2f080d8ae3ebebfb70a4611a186fa1fedff9dbca28998066d835dc222c652d","value_sha256":"3b2f080d8ae3ebebfb70a4611a186fa1fedff9dbca28998066d835dc222c652d"}],[{"shape":[1,3,10240],"sha256":"f78baa9905f133d407c9c0b93957dd51229a13854e53e50d03c282d07866e768","value_sha256":"f78baa9905f133d407c9c0b93957dd51229a13854e53e50d03c282d07866e768"},{"shape":[1,48,128,128],"sha256":"0da307f15747719d4b52b2b072cbfd78c3597dd363a8ab5f7abc32536eccfd66","value_sha256":"0da307f15747719d4b52b2b072cbfd78c3597dd363a8ab5f7abc32536eccfd66"}],[{"shape":[1,3,10240],"sha256":"733c7b59cb4044de99c54495cacd03f04f036460cf2b51590d4fe95c7d13aa1d","value_sha256":"733c7b59cb4044de99c54495cacd03f04f036460cf2b51590d4fe95c7d13aa1d"},{"shape":[1,48,128,128],"sha256":"59f1f772b4c4a9e9b022952c286ec0df1d31e70d91890a227607e5b900a995af","value_sha256":"59f1f772b4c4a9e9b022952c286ec0df1d31e70d91890a227607e5b900a995af"}],[{"shape":[1,2,88,256],"sha256":"6387f267821783cba0e6718bb2644c736e384d8f36e0c7138b0be7606c3baf8c","value_sha256":"6387f267821783cba0e6718bb2644c736e384d8f36e0c7138b0be7606c3baf8c"},{"shape":[1,2,88,256],"sha256":"fe5b8510d94e73fb4c998cd9612064917dc7058849d0f1c67b534cf3791bc934","value_sha256":"fe5b8510d94e73fb4c998cd9612064917dc7058849d0f1c67b534cf3791bc934"},{"shape":[1,88,128],"sha256":"eb577791c6743ff0e48840da830393e4f24ab4d9938e98efcd49780758c3393a","value_sha256":"eb577791c6743ff0e48840da830393e4f24ab4d9938e98efcd49780758c3393a"},{"shape":[1,22,128],"sha256":"77b12c33a82a7d53f68da4d44d1db4c65335b7d5ec8c64f8d547f07ad7451c02","value_sha256":"77b12c33a82a7d53f68da4d44d1db4c65335b7d5ec8c64f8d547f07ad7451c02"}],[{"shape":[1,3,10240],"sha256":"e752995d1329444a5a9e67590876bc32ce30558ae9361a78bc5843c82733050f","value_sha256":"e752995d1329444a5a9e67590876bc32ce30558ae9361a78bc5843c82733050f"},{"shape":[1,48,128,128],"sha256":"df1a591a776abf17e05f8bbd32d1b811135f223c22e15addc4ed68dd01ad8493","value_sha256":"df1a591a776abf17e05f8bbd32d1b811135f223c22e15addc4ed68dd01ad8493"}],[{"shape":[1,3,10240],"sha256":"9624416c4d3593bd93d1b06785772b1f7df6643668840d532a3bee30aafa4042","value_sha256":"9624416c4d3593bd93d1b06785772b1f7df6643668840d532a3bee30aafa4042"},{"shape":[1,48,128,128],"sha256":"a7ede72d50c40571f83c46de94c660fa36d8ff812b05d0132e17571e3bbc3589","value_sha256":"a7ede72d50c40571f83c46de94c660fa36d8ff812b05d0132e17571e3bbc3589"}],[{"shape":[1,3,10240],"sha256":"f8911ad582ec57faad4441c7e0deb866a9278632d4504f788a86ce9a6d0c08ab","value_sha256":"f8911ad582ec57faad4441c7e0deb866a9278632d4504f788a86ce9a6d0c08ab"},{"shape":[1,48,128,128],"sha256":"b60e89e7467e58bfc5a0ff18bfda1c1f42a18bd37b05b1efd3d765a94aea23e5","value_sha256":"b60e89e7467e58bfc5a0ff18bfda1c1f42a18bd37b05b1efd3d765a94aea23e5"}],[{"shape":[1,2,88,256],"sha256":"161f854d9d75d6b25f61a9f5c6c8ed397ba91ad3ef7f8a134841dfdfcddfa513","value_sha256":"161f854d9d75d6b25f61a9f5c6c8ed397ba91ad3ef7f8a134841dfdfcddfa513"},{"shape":[1,2,88,256],"sha256":"9edf9661611655bad494ab069dc10755ba1adcd810278718a0461b4c6304041c","value_sha256":"9edf9661611655bad494ab069dc10755ba1adcd810278718a0461b4c6304041c"},{"shape":[1,88,128],"sha256":"810459f986074c122023ad0c1c659576deb14dbfdad7354675bac8543e53bd4e","value_sha256":"810459f986074c122023ad0c1c659576deb14dbfdad7354675bac8543e53bd4e"},{"shape":[1,22,128],"sha256":"17d21af4459d8d3afb2ce6c7f26bf482e2c8343aa425b0b2a4c8bd1a81da8899","value_sha256":"17d21af4459d8d3afb2ce6c7f26bf482e2c8343aa425b0b2a4c8bd1a81da8899"}]],"mtp":[[{"shape":[1,2,87,256],"sha256":"675223d7488e0b09910db850033d605b09afd67adab910691cc5c43096472352","value_sha256":"675223d7488e0b09910db850033d605b09afd67adab910691cc5c43096472352"},{"shape":[1,2,87,256],"sha256":"8aec54ec77cccccca2e80e320aa4a326d41962f98f8fe62e07cf818aa70281d9","value_sha256":"8aec54ec77cccccca2e80e320aa4a326d41962f98f8fe62e07cf818aa70281d9"},{"shape":[1,87,128],"sha256":"824fd057d8b840f5b8a1eb84e53210b17b75bc7a99280538c694dbb1590f346d","value_sha256":"824fd057d8b840f5b8a1eb84e53210b17b75bc7a99280538c694dbb1590f346d"},{"shape":[1,21,128],"sha256":"85deeac841e2b9fdbf303b9977b7241f56008c00eb07fe0e8a307bdd346b321b","value_sha256":"85deeac841e2b9fdbf303b9977b7241f56008c00eb07fe0e8a307bdd346b321b"}]]}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/reopened.json b/docs/issue113-head-chunk/off-product/reopened.json
new file mode 100644
index 0000000..d035a34
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/reopened.json
@@ -0,0 +1 @@
+{"case":"reopened","prompt":[248045,8678,198,24342,286,4879,369,716,310,3238,13,13262,678,7047,9522,321,10419,11,6992,5774,310,279,16198,1973,24366,24150,362,13,248046,198,248045,846,198,2427,494,799,310,5600,13,248046,198,248045,74455,198,248068,198,760,1156,6587,728,310,1698,494,799,310,5600,13,1061,198,248069,271,248046,198,248045,846,198,22791,24217,13,248046,198,248045,74455,198,248068,198],"tokens":[760,1156,6587,728,310,2962,24217,13,353,8334,29014,494],"cached":58,"state_sha256":"e6cb7355aaa92a3569a365c94c55aa7a50bd731dc04f6fa320da8520d182e2f8"}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/reopened/0.json b/docs/issue113-head-chunk/off-product/reopened/0.json
new file mode 100644
index 0000000..4e5e56d
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/reopened/0.json
@@ -0,0 +1 @@
+{"index":0,"ids":[760,1156,6587,728],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"dbab0d12f25f2be2234d58d45f86219af62deb20e22b0d73a77aa5e7b722d082"},"hidden":{"shape":[1,4,10240],"sha256":"8f9aebc38d1a3e550bed6a4663eedac7a5ed3ce61d836d1cecdfa11da67ebd0d"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/reopened/1.json b/docs/issue113-head-chunk/off-product/reopened/1.json
new file mode 100644
index 0000000..6ff2118
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/reopened/1.json
@@ -0,0 +1 @@
+{"index":1,"ids":[310,2962,24217,494],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"d6b8e6afe3c6f5af068f240e0e69b3875311aac6929b3d5a38ab3aeaa1bf227c"},"hidden":{"shape":[1,4,10240],"sha256":"52f20217ed43d8e59ddbe037cb77f56cc3d8ef473fd1963948fff6faf8ad4212"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/reopened/2.json b/docs/issue113-head-chunk/off-product/reopened/2.json
new file mode 100644
index 0000000..1b67c10
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/reopened/2.json
@@ -0,0 +1 @@
+{"index":2,"ids":[13,353,2582,3804],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"048e9343d4268d98455a01f7c73ee47af78bc811ef9e7680a72cae59336b9989"},"hidden":{"shape":[1,4,10240],"sha256":"8e8b07fb20baa33ad64cc76ff64403a434d520c627a182a224cf033a5a14e5d4"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/reopened/3.json b/docs/issue113-head-chunk/off-product/reopened/3.json
new file mode 100644
index 0000000..2c6b980
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/reopened/3.json
@@ -0,0 +1 @@
+{"index":3,"ids":[8334,29014,494],"verify":true,"logits":{"shape":[1,3,248320],"sha256":"82d671c6ac7b3a0952e60ef554ca670562fd01ef25c37bd10bc27025768ea0f2"},"hidden":{"shape":[1,3,10240],"sha256":"d1bc90f284364eae8bac80a5da6ab64c9c76d6994bb09ed463a17d67e25c016e"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/warm-arrays.json b/docs/issue113-head-chunk/off-product/warm-arrays.json
new file mode 100644
index 0000000..df8962a
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/warm-arrays.json
@@ -0,0 +1 @@
+{"trunk":[[{"shape":[1,3,10240],"sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84","value_sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84"},{"shape":[1,48,128,128],"sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf","value_sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf"}],[{"shape":[1,3,10240],"sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232","value_sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232"},{"shape":[1,48,128,128],"sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf","value_sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf"},{"shape":[1,9,10240],"sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08","value_sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08"},{"shape":[1,2],"sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152","value_sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152"}],[{"shape":[1,3,10240],"sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2","value_sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2"},{"shape":[1,48,128,128],"sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34","value_sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34"}],[{"shape":[1,2,88,256],"sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f","value_sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f"},{"shape":[1,2,88,256],"sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24","value_sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24"},{"shape":[1,88,128],"sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748","value_sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748"},{"shape":[1,22,128],"sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a","value_sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a"}],[{"shape":[1,3,10240],"sha256":"431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda","value_sha256":"431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda"},{"shape":[1,48,128,128],"sha256":"c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f","value_sha256":"c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f"}],[{"shape":[1,3,10240],"sha256":"f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a","value_sha256":"f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a"},{"shape":[1,48,128,128],"sha256":"8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f","value_sha256":"8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f"}],[{"shape":[1,3,10240],"sha256":"3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d","value_sha256":"3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d"},{"shape":[1,48,128,128],"sha256":"bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f","value_sha256":"bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f"}],[{"shape":[1,2,88,256],"sha256":"9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b","value_sha256":"9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b"},{"shape":[1,2,88,256],"sha256":"b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba","value_sha256":"b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba"},{"shape":[1,88,128],"sha256":"491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae","value_sha256":"491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae"},{"shape":[1,22,128],"sha256":"104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb","value_sha256":"104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb"}],[{"shape":[1,3,10240],"sha256":"e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57","value_sha256":"e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57"},{"shape":[1,48,128,128],"sha256":"0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43","value_sha256":"0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43"}],[{"shape":[1,3,10240],"sha256":"f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0","value_sha256":"f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0"},{"shape":[1,48,128,128],"sha256":"75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52","value_sha256":"75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52"}],[{"shape":[1,3,10240],"sha256":"c23b7e524522714aaf620c9e0bbb6b1d3db3585971f0cffafd7fe7e2018176dc","value_sha256":"c23b7e524522714aaf620c9e0bbb6b1d3db3585971f0cffafd7fe7e2018176dc"},{"shape":[1,48,128,128],"sha256":"4ce5d0905ec5e5ae834215c16b25d810a5ef6e54a6d7d96f95e477209775ddaa","value_sha256":"4ce5d0905ec5e5ae834215c16b25d810a5ef6e54a6d7d96f95e477209775ddaa"}],[{"shape":[1,2,88,256],"sha256":"7803bca47639bdcb533d5985f37930fb74cd112882f8cfa26513bc297e1245fe","value_sha256":"7803bca47639bdcb533d5985f37930fb74cd112882f8cfa26513bc297e1245fe"},{"shape":[1,2,88,256],"sha256":"40bb50faa5d147165f88cd5ba2379dc395f4f296c66246499b9efd03c9eef7ac","value_sha256":"40bb50faa5d147165f88cd5ba2379dc395f4f296c66246499b9efd03c9eef7ac"},{"shape":[1,88,128],"sha256":"6e20787bdd02bd9615279eac42cf9f2664430aeb8cdd8c93c6a9881556722493","value_sha256":"6e20787bdd02bd9615279eac42cf9f2664430aeb8cdd8c93c6a9881556722493"},{"shape":[1,22,128],"sha256":"c17459f225244530703a28076f3006f7e53d50e728ecb79ae4f5b968b594d96d","value_sha256":"c17459f225244530703a28076f3006f7e53d50e728ecb79ae4f5b968b594d96d"}],[{"shape":[1,3,10240],"sha256":"8e1e94ade5f7453693e4d99f003d190122a3cd69b0bddca1a1fc2409d9a32b7d","value_sha256":"8e1e94ade5f7453693e4d99f003d190122a3cd69b0bddca1a1fc2409d9a32b7d"},{"shape":[1,48,128,128],"sha256":"976efc317438cb07b1e6d80ce6ae12d8e0095a51fa7b490b8a84a8e0d6b79399","value_sha256":"976efc317438cb07b1e6d80ce6ae12d8e0095a51fa7b490b8a84a8e0d6b79399"}],[{"shape":[1,3,10240],"sha256":"6153601aa51b613aa617cb823b2d5e582847afba59eac2c190a96ea465ad77e1","value_sha256":"6153601aa51b613aa617cb823b2d5e582847afba59eac2c190a96ea465ad77e1"},{"shape":[1,48,128,128],"sha256":"077106a1cc1cf3e3d24ba44378f788cab30d80fd38aef87fcf957a99b662379d","value_sha256":"077106a1cc1cf3e3d24ba44378f788cab30d80fd38aef87fcf957a99b662379d"}],[{"shape":[1,3,10240],"sha256":"d020e70e7cab233eb7d188fa42e975f0a20cdce167f52db6043933e583f7c280","value_sha256":"d020e70e7cab233eb7d188fa42e975f0a20cdce167f52db6043933e583f7c280"},{"shape":[1,48,128,128],"sha256":"c05cdc62e13b7c238ad02e2a4a81286dc5d0fd8cd9c91af6c8ea6380d203f080","value_sha256":"c05cdc62e13b7c238ad02e2a4a81286dc5d0fd8cd9c91af6c8ea6380d203f080"}],[{"shape":[1,2,88,256],"sha256":"be1eed7cb68714c49599bea49ffdc831b5caac8444781088d21c87892445d2d7","value_sha256":"be1eed7cb68714c49599bea49ffdc831b5caac8444781088d21c87892445d2d7"},{"shape":[1,2,88,256],"sha256":"abe8a391f5a932dee7718ea3abe5419e015750b85a522856f1a59f435d5688dc","value_sha256":"abe8a391f5a932dee7718ea3abe5419e015750b85a522856f1a59f435d5688dc"},{"shape":[1,88,128],"sha256":"c1cd1549e2032d2bc28d2d0c9f1bfeb608f81fe8abe2b18b77a1658a4c0ed9c4","value_sha256":"c1cd1549e2032d2bc28d2d0c9f1bfeb608f81fe8abe2b18b77a1658a4c0ed9c4"},{"shape":[1,22,128],"sha256":"072bc306a1b4a56332dc90d889c42e6b2037a820330da0b897815a3750c00043","value_sha256":"072bc306a1b4a56332dc90d889c42e6b2037a820330da0b897815a3750c00043"}],[{"shape":[1,3,10240],"sha256":"7f4500e05eb6e52b25bd9405ccaae398fcc426835af0550599175d0785694154","value_sha256":"7f4500e05eb6e52b25bd9405ccaae398fcc426835af0550599175d0785694154"},{"shape":[1,48,128,128],"sha256":"e33affece02b985e41ea0d4ce467679226e9f9b5f55c4001cdbf8ebcb087db40","value_sha256":"e33affece02b985e41ea0d4ce467679226e9f9b5f55c4001cdbf8ebcb087db40"}],[{"shape":[1,3,10240],"sha256":"d133a043fa3abeaf2498e4336142e34a6d3222e4b2eec943931e97eb060f5657","value_sha256":"d133a043fa3abeaf2498e4336142e34a6d3222e4b2eec943931e97eb060f5657"},{"shape":[1,48,128,128],"sha256":"465ca838c8b5abbaac2b32a5f2f204ebe674d20ff50ed9b3532e81ede40c1164","value_sha256":"465ca838c8b5abbaac2b32a5f2f204ebe674d20ff50ed9b3532e81ede40c1164"}],[{"shape":[1,3,10240],"sha256":"379bfa2a27e16bc7d2d1213c0d145307954a9d4d36d8c5f5a6cb9b4f172b400c","value_sha256":"379bfa2a27e16bc7d2d1213c0d145307954a9d4d36d8c5f5a6cb9b4f172b400c"},{"shape":[1,48,128,128],"sha256":"be38588e2a1191fdccc176952eb809d41b8c40d874bc9f160adae245abe93015","value_sha256":"be38588e2a1191fdccc176952eb809d41b8c40d874bc9f160adae245abe93015"}],[{"shape":[1,2,88,256],"sha256":"c3c637457aeabb3d95909c7fd35e0a4024aef118af9dec6b369a50bf409cc735","value_sha256":"c3c637457aeabb3d95909c7fd35e0a4024aef118af9dec6b369a50bf409cc735"},{"shape":[1,2,88,256],"sha256":"efc834d804fa9f0958e2323b5c77dc22e134ee9130e211e67cf46e713fc303d1","value_sha256":"efc834d804fa9f0958e2323b5c77dc22e134ee9130e211e67cf46e713fc303d1"},{"shape":[1,88,128],"sha256":"3b59d52c473c7b1710f17b8672fd1ad1611d201e357a5a61e2f4272194654d0e","value_sha256":"3b59d52c473c7b1710f17b8672fd1ad1611d201e357a5a61e2f4272194654d0e"},{"shape":[1,22,128],"sha256":"ba538ce500987345986122428d8884b73b26e0c94466e3e5651d725a1e9c6623","value_sha256":"ba538ce500987345986122428d8884b73b26e0c94466e3e5651d725a1e9c6623"}],[{"shape":[1,3,10240],"sha256":"09026ef1a4aeb90aeeba7b23b21453e76852d97c609e40a04ecbd40e1c4f387d","value_sha256":"09026ef1a4aeb90aeeba7b23b21453e76852d97c609e40a04ecbd40e1c4f387d"},{"shape":[1,48,128,128],"sha256":"1b34b1e411aed8f13f9416722c35131648f0fb7752a1b0fdbc43ad56a3b089f8","value_sha256":"1b34b1e411aed8f13f9416722c35131648f0fb7752a1b0fdbc43ad56a3b089f8"}],[{"shape":[1,3,10240],"sha256":"a0b8dfe443b66d38cfc29af210f2461229d947985d56eecb00d35b8ead4a6758","value_sha256":"a0b8dfe443b66d38cfc29af210f2461229d947985d56eecb00d35b8ead4a6758"},{"shape":[1,48,128,128],"sha256":"23d008ba78432fc50967f3f9219912ecc50a745246b1562cd047304650910631","value_sha256":"23d008ba78432fc50967f3f9219912ecc50a745246b1562cd047304650910631"}],[{"shape":[1,3,10240],"sha256":"9b2393f3353a7c0c38906fa33cb081828590f7e095a047c32201ec4b171a3cc6","value_sha256":"9b2393f3353a7c0c38906fa33cb081828590f7e095a047c32201ec4b171a3cc6"},{"shape":[1,48,128,128],"sha256":"ea13bedee4d597e350a7b9d26965b06e996d3fb60a56c4a1a0823af6f56bdc29","value_sha256":"ea13bedee4d597e350a7b9d26965b06e996d3fb60a56c4a1a0823af6f56bdc29"}],[{"shape":[1,2,88,256],"sha256":"7f609422a62de424ecbbdbdfbe89a87e8a990103c3d3ddaf2b27213942b89c33","value_sha256":"7f609422a62de424ecbbdbdfbe89a87e8a990103c3d3ddaf2b27213942b89c33"},{"shape":[1,2,88,256],"sha256":"3f6d942ae01b4536df664a306d021a26e4c0c5103385baf2a39470f0f2498c08","value_sha256":"3f6d942ae01b4536df664a306d021a26e4c0c5103385baf2a39470f0f2498c08"},{"shape":[1,88,128],"sha256":"6f4ff9a35dd06bab89858195b4c10201258ee1d38361cd57fed9d47224aafffa","value_sha256":"6f4ff9a35dd06bab89858195b4c10201258ee1d38361cd57fed9d47224aafffa"},{"shape":[1,22,128],"sha256":"418e9f68c997dc6d3565ed3331b22edbededa67d22fedf1adf60ea913bd1102a","value_sha256":"418e9f68c997dc6d3565ed3331b22edbededa67d22fedf1adf60ea913bd1102a"}],[{"shape":[1,3,10240],"sha256":"71aa2645f2ce221cc2b5b0ecdb57cf7f511d8520b642f46f5656b35b3446c68b","value_sha256":"71aa2645f2ce221cc2b5b0ecdb57cf7f511d8520b642f46f5656b35b3446c68b"},{"shape":[1,48,128,128],"sha256":"27fe6db7a664bd3f479907384daed9c203d272d9d805680bf60f5823589522f5","value_sha256":"27fe6db7a664bd3f479907384daed9c203d272d9d805680bf60f5823589522f5"}],[{"shape":[1,3,10240],"sha256":"86f5a1132abfc21f4263d1e5781bd664bb5e0515c99c99262f5ece1309d31146","value_sha256":"86f5a1132abfc21f4263d1e5781bd664bb5e0515c99c99262f5ece1309d31146"},{"shape":[1,48,128,128],"sha256":"85d28d66cd18e5520f89273d1030d60aee4815d1979da53296ca6ebcb15eaea7","value_sha256":"85d28d66cd18e5520f89273d1030d60aee4815d1979da53296ca6ebcb15eaea7"}],[{"shape":[1,3,10240],"sha256":"0c0e755df21fddcc7c6c18f91ff0c28798c7f6db5d7ae3c1ccada82b75783c49","value_sha256":"0c0e755df21fddcc7c6c18f91ff0c28798c7f6db5d7ae3c1ccada82b75783c49"},{"shape":[1,48,128,128],"sha256":"a1f173e5880541f5db8e8c59075ec15f67229eb0a14e6fc1586201f4a5367d6e","value_sha256":"a1f173e5880541f5db8e8c59075ec15f67229eb0a14e6fc1586201f4a5367d6e"}],[{"shape":[1,2,88,256],"sha256":"67e00cb7fc89c35ad41c88339922d7ebed49abc11fcee6e7992daf4b79463f45","value_sha256":"67e00cb7fc89c35ad41c88339922d7ebed49abc11fcee6e7992daf4b79463f45"},{"shape":[1,2,88,256],"sha256":"c8c4c07315a42e3d3bcb61a33402a99b6b9f0b897ddddd682cca5ca9b3c5759b","value_sha256":"c8c4c07315a42e3d3bcb61a33402a99b6b9f0b897ddddd682cca5ca9b3c5759b"},{"shape":[1,88,128],"sha256":"13309057cb8436d0541acf49399a408b9035283a3e8ee00a36a2631d6feb9854","value_sha256":"13309057cb8436d0541acf49399a408b9035283a3e8ee00a36a2631d6feb9854"},{"shape":[1,22,128],"sha256":"c2bda4e0c3881a16568e45307b3907f5d9beaf96eddf085a560a767684525031","value_sha256":"c2bda4e0c3881a16568e45307b3907f5d9beaf96eddf085a560a767684525031"}],[{"shape":[1,3,10240],"sha256":"1e1db788f66c83daab3bc9f4ebf8e21e73297710a725207b9bc91530ac6ec073","value_sha256":"1e1db788f66c83daab3bc9f4ebf8e21e73297710a725207b9bc91530ac6ec073"},{"shape":[1,48,128,128],"sha256":"d42708090309446c157f583cdd98d184bbcf3d7582dcd56d232a279cacf53d03","value_sha256":"d42708090309446c157f583cdd98d184bbcf3d7582dcd56d232a279cacf53d03"}],[{"shape":[1,3,10240],"sha256":"c0afb77f8c6e3c141a0c0140224ec1a05f514331e76195185d74763ef6a3dcf1","value_sha256":"c0afb77f8c6e3c141a0c0140224ec1a05f514331e76195185d74763ef6a3dcf1"},{"shape":[1,48,128,128],"sha256":"d6a947a09b92b149efb68abe9f29fcce54471ab9c388aa7dbbd113c7e88f6275","value_sha256":"d6a947a09b92b149efb68abe9f29fcce54471ab9c388aa7dbbd113c7e88f6275"}],[{"shape":[1,3,10240],"sha256":"910b4e1eb82d9964f242034afd4420a420c37fb47b7ff344ba7b1d9a56e8762f","value_sha256":"910b4e1eb82d9964f242034afd4420a420c37fb47b7ff344ba7b1d9a56e8762f"},{"shape":[1,48,128,128],"sha256":"1144c0faae284f0059a58041a49152c2bb554f2936852bbd0a520f45098b936a","value_sha256":"1144c0faae284f0059a58041a49152c2bb554f2936852bbd0a520f45098b936a"}],[{"shape":[1,2,88,256],"sha256":"35cdbd0aded1c0ba01994a011832c2431488d440aa03f2f6123b9b84ddf1362d","value_sha256":"35cdbd0aded1c0ba01994a011832c2431488d440aa03f2f6123b9b84ddf1362d"},{"shape":[1,2,88,256],"sha256":"74166f058c8026611b093c3ca3629c2462ecd5c506b5da39a64e08436d597a3b","value_sha256":"74166f058c8026611b093c3ca3629c2462ecd5c506b5da39a64e08436d597a3b"},{"shape":[1,88,128],"sha256":"ec566fb641c4ba35da9341de02e170624a2da3616819e6a1cf4f4395454cc386","value_sha256":"ec566fb641c4ba35da9341de02e170624a2da3616819e6a1cf4f4395454cc386"},{"shape":[1,22,128],"sha256":"f2a66ce5e3986b883434ba9c91e8bbefc099bdc4011aada8ed89a4cefbc4cd16","value_sha256":"f2a66ce5e3986b883434ba9c91e8bbefc099bdc4011aada8ed89a4cefbc4cd16"}],[{"shape":[1,3,10240],"sha256":"a361723352d381d4456b0488309d7fc3a670b9557f0293da8323b1b1170b9c71","value_sha256":"a361723352d381d4456b0488309d7fc3a670b9557f0293da8323b1b1170b9c71"},{"shape":[1,48,128,128],"sha256":"19dd73c3b6450f2ae672d602046f8fdc159e7dc8d178befe72fe8c95829004a7","value_sha256":"19dd73c3b6450f2ae672d602046f8fdc159e7dc8d178befe72fe8c95829004a7"}],[{"shape":[1,3,10240],"sha256":"dd9b66e88449a34f1f346b367c5a689f72e2549a24622ad1e04876978c1a49f6","value_sha256":"dd9b66e88449a34f1f346b367c5a689f72e2549a24622ad1e04876978c1a49f6"},{"shape":[1,48,128,128],"sha256":"da16afcaa1e209ffb6f08639c70e56c3d3b3fe88449fa5700d4c2374f545ffc3","value_sha256":"da16afcaa1e209ffb6f08639c70e56c3d3b3fe88449fa5700d4c2374f545ffc3"}],[{"shape":[1,3,10240],"sha256":"8309772a5f9abc9a257f4dca60d98744cae411e8cd5b59f38132bb9a90c9f160","value_sha256":"8309772a5f9abc9a257f4dca60d98744cae411e8cd5b59f38132bb9a90c9f160"},{"shape":[1,48,128,128],"sha256":"f3a26845be01884066372bfb0d5b54f1e19c3927da4a84e59ce714a68b69ceaf","value_sha256":"f3a26845be01884066372bfb0d5b54f1e19c3927da4a84e59ce714a68b69ceaf"}],[{"shape":[1,2,88,256],"sha256":"061f8fe052fe21c8505782c220e03d09ab31deb93e7987709c55cd5dc3b778bd","value_sha256":"98186c7fec2498072f93b621a7f5b14456d4a67b0548a006353d87b4e4c2e89f"},{"shape":[1,2,88,256],"sha256":"e2c9efd039839aeec320ba45c64b4e8851e941b7300ef43b3b767d7a6fc3f81a","value_sha256":"e2c9efd039839aeec320ba45c64b4e8851e941b7300ef43b3b767d7a6fc3f81a"},{"shape":[1,88,128],"sha256":"4bf22f9cc812cbd24c68193dca9e687922f11cd4f14456157745987762e1a5d3","value_sha256":"4bf22f9cc812cbd24c68193dca9e687922f11cd4f14456157745987762e1a5d3"},{"shape":[1,22,128],"sha256":"be37a2a222ed9aa0bfffc831ce64c51d50adf8da9e8f865758f0ea199abed00c","value_sha256":"be37a2a222ed9aa0bfffc831ce64c51d50adf8da9e8f865758f0ea199abed00c"}],[{"shape":[1,3,10240],"sha256":"05517c814a7757ddd5a9576f70b54d74ae147035464fab8cbb9f85effa33f350","value_sha256":"05517c814a7757ddd5a9576f70b54d74ae147035464fab8cbb9f85effa33f350"},{"shape":[1,48,128,128],"sha256":"d97ab5b464249daf10022fb42384c16568ffafde0db8a75c209f6b9d569f1733","value_sha256":"d97ab5b464249daf10022fb42384c16568ffafde0db8a75c209f6b9d569f1733"}],[{"shape":[1,3,10240],"sha256":"405fddf03a32c3d17029d69ab68db301f89652a513e9d8a55f3b6aa5b7e6942d","value_sha256":"405fddf03a32c3d17029d69ab68db301f89652a513e9d8a55f3b6aa5b7e6942d"},{"shape":[1,48,128,128],"sha256":"0f195fc269e9c513e7e43dabce7c0446f259f576f329fcaf97b64f4596e4df20","value_sha256":"0f195fc269e9c513e7e43dabce7c0446f259f576f329fcaf97b64f4596e4df20"}],[{"shape":[1,3,10240],"sha256":"72d77520b60e0279c3e51cfd0be2dae0740a4c502279b4bc4e81de5f283cdbe5","value_sha256":"72d77520b60e0279c3e51cfd0be2dae0740a4c502279b4bc4e81de5f283cdbe5"},{"shape":[1,48,128,128],"sha256":"5397183714e9f7e4b1d5b55ef9ae553c1cffa2c8bab284425a2a4e701c7ff73c","value_sha256":"5397183714e9f7e4b1d5b55ef9ae553c1cffa2c8bab284425a2a4e701c7ff73c"}],[{"shape":[1,2,88,256],"sha256":"29bf2a7e968d01f9e5944a545bdbfa18d55a4f7be1f196c7bcb7e763d31facb7","value_sha256":"29bf2a7e968d01f9e5944a545bdbfa18d55a4f7be1f196c7bcb7e763d31facb7"},{"shape":[1,2,88,256],"sha256":"d0a7d3c46f6f41ecd2a0cb57be29f2c174bf8e4ad442d448773e819a8564e335","value_sha256":"d0a7d3c46f6f41ecd2a0cb57be29f2c174bf8e4ad442d448773e819a8564e335"},{"shape":[1,88,128],"sha256":"5331b2be56a2cb81895f7fa0f0c39d1696f7e32540e2c77e9d40c1e22d23ac8d","value_sha256":"5331b2be56a2cb81895f7fa0f0c39d1696f7e32540e2c77e9d40c1e22d23ac8d"},{"shape":[1,22,128],"sha256":"9cef226bd40e273913f9cbadc580e770da1479955964bb2903aa53d6476fa840","value_sha256":"9cef226bd40e273913f9cbadc580e770da1479955964bb2903aa53d6476fa840"}],[{"shape":[1,3,10240],"sha256":"30ce86b21de9483ec50ec7eabce466e9594ea6c98c9a8beb0afb83348d05e0eb","value_sha256":"30ce86b21de9483ec50ec7eabce466e9594ea6c98c9a8beb0afb83348d05e0eb"},{"shape":[1,48,128,128],"sha256":"3b2f080d8ae3ebebfb70a4611a186fa1fedff9dbca28998066d835dc222c652d","value_sha256":"3b2f080d8ae3ebebfb70a4611a186fa1fedff9dbca28998066d835dc222c652d"}],[{"shape":[1,3,10240],"sha256":"f78baa9905f133d407c9c0b93957dd51229a13854e53e50d03c282d07866e768","value_sha256":"f78baa9905f133d407c9c0b93957dd51229a13854e53e50d03c282d07866e768"},{"shape":[1,48,128,128],"sha256":"0da307f15747719d4b52b2b072cbfd78c3597dd363a8ab5f7abc32536eccfd66","value_sha256":"0da307f15747719d4b52b2b072cbfd78c3597dd363a8ab5f7abc32536eccfd66"}],[{"shape":[1,3,10240],"sha256":"733c7b59cb4044de99c54495cacd03f04f036460cf2b51590d4fe95c7d13aa1d","value_sha256":"733c7b59cb4044de99c54495cacd03f04f036460cf2b51590d4fe95c7d13aa1d"},{"shape":[1,48,128,128],"sha256":"59f1f772b4c4a9e9b022952c286ec0df1d31e70d91890a227607e5b900a995af","value_sha256":"59f1f772b4c4a9e9b022952c286ec0df1d31e70d91890a227607e5b900a995af"}],[{"shape":[1,2,88,256],"sha256":"6387f267821783cba0e6718bb2644c736e384d8f36e0c7138b0be7606c3baf8c","value_sha256":"6387f267821783cba0e6718bb2644c736e384d8f36e0c7138b0be7606c3baf8c"},{"shape":[1,2,88,256],"sha256":"fe5b8510d94e73fb4c998cd9612064917dc7058849d0f1c67b534cf3791bc934","value_sha256":"fe5b8510d94e73fb4c998cd9612064917dc7058849d0f1c67b534cf3791bc934"},{"shape":[1,88,128],"sha256":"eb577791c6743ff0e48840da830393e4f24ab4d9938e98efcd49780758c3393a","value_sha256":"eb577791c6743ff0e48840da830393e4f24ab4d9938e98efcd49780758c3393a"},{"shape":[1,22,128],"sha256":"77b12c33a82a7d53f68da4d44d1db4c65335b7d5ec8c64f8d547f07ad7451c02","value_sha256":"77b12c33a82a7d53f68da4d44d1db4c65335b7d5ec8c64f8d547f07ad7451c02"}],[{"shape":[1,3,10240],"sha256":"e752995d1329444a5a9e67590876bc32ce30558ae9361a78bc5843c82733050f","value_sha256":"e752995d1329444a5a9e67590876bc32ce30558ae9361a78bc5843c82733050f"},{"shape":[1,48,128,128],"sha256":"df1a591a776abf17e05f8bbd32d1b811135f223c22e15addc4ed68dd01ad8493","value_sha256":"df1a591a776abf17e05f8bbd32d1b811135f223c22e15addc4ed68dd01ad8493"}],[{"shape":[1,3,10240],"sha256":"9624416c4d3593bd93d1b06785772b1f7df6643668840d532a3bee30aafa4042","value_sha256":"9624416c4d3593bd93d1b06785772b1f7df6643668840d532a3bee30aafa4042"},{"shape":[1,48,128,128],"sha256":"a7ede72d50c40571f83c46de94c660fa36d8ff812b05d0132e17571e3bbc3589","value_sha256":"a7ede72d50c40571f83c46de94c660fa36d8ff812b05d0132e17571e3bbc3589"}],[{"shape":[1,3,10240],"sha256":"f8911ad582ec57faad4441c7e0deb866a9278632d4504f788a86ce9a6d0c08ab","value_sha256":"f8911ad582ec57faad4441c7e0deb866a9278632d4504f788a86ce9a6d0c08ab"},{"shape":[1,48,128,128],"sha256":"b60e89e7467e58bfc5a0ff18bfda1c1f42a18bd37b05b1efd3d765a94aea23e5","value_sha256":"b60e89e7467e58bfc5a0ff18bfda1c1f42a18bd37b05b1efd3d765a94aea23e5"}],[{"shape":[1,2,88,256],"sha256":"161f854d9d75d6b25f61a9f5c6c8ed397ba91ad3ef7f8a134841dfdfcddfa513","value_sha256":"161f854d9d75d6b25f61a9f5c6c8ed397ba91ad3ef7f8a134841dfdfcddfa513"},{"shape":[1,2,88,256],"sha256":"9edf9661611655bad494ab069dc10755ba1adcd810278718a0461b4c6304041c","value_sha256":"9edf9661611655bad494ab069dc10755ba1adcd810278718a0461b4c6304041c"},{"shape":[1,88,128],"sha256":"810459f986074c122023ad0c1c659576deb14dbfdad7354675bac8543e53bd4e","value_sha256":"810459f986074c122023ad0c1c659576deb14dbfdad7354675bac8543e53bd4e"},{"shape":[1,22,128],"sha256":"17d21af4459d8d3afb2ce6c7f26bf482e2c8343aa425b0b2a4c8bd1a81da8899","value_sha256":"17d21af4459d8d3afb2ce6c7f26bf482e2c8343aa425b0b2a4c8bd1a81da8899"}]],"mtp":[[{"shape":[1,2,87,256],"sha256":"675223d7488e0b09910db850033d605b09afd67adab910691cc5c43096472352","value_sha256":"675223d7488e0b09910db850033d605b09afd67adab910691cc5c43096472352"},{"shape":[1,2,87,256],"sha256":"8aec54ec77cccccca2e80e320aa4a326d41962f98f8fe62e07cf818aa70281d9","value_sha256":"8aec54ec77cccccca2e80e320aa4a326d41962f98f8fe62e07cf818aa70281d9"},{"shape":[1,87,128],"sha256":"824fd057d8b840f5b8a1eb84e53210b17b75bc7a99280538c694dbb1590f346d","value_sha256":"824fd057d8b840f5b8a1eb84e53210b17b75bc7a99280538c694dbb1590f346d"},{"shape":[1,21,128],"sha256":"85deeac841e2b9fdbf303b9977b7241f56008c00eb07fe0e8a307bdd346b321b","value_sha256":"85deeac841e2b9fdbf303b9977b7241f56008c00eb07fe0e8a307bdd346b321b"}]]}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/warm.json b/docs/issue113-head-chunk/off-product/warm.json
new file mode 100644
index 0000000..8a5f2f9
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/warm.json
@@ -0,0 +1 @@
+{"case":"warm","prompt":[248045,8678,198,24342,286,4879,369,716,310,3238,13,13262,678,7047,9522,321,10419,11,6992,5774,310,279,16198,1973,24366,24150,362,13,248046,198,248045,846,198,2427,494,799,310,5600,13,248046,198,248045,74455,198,248068,198,760,1156,6587,728,310,1698,494,799,310,5600,13,1061,198,248069,271,248046,198,248045,846,198,22791,24217,13,248046,198,248045,74455,198,248068,198],"tokens":[760,1156,6587,728,310,2962,24217,13,353,8334,29014,494],"cached":58,"state_sha256":"e6cb7355aaa92a3569a365c94c55aa7a50bd731dc04f6fa320da8520d182e2f8"}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/warm/0.json b/docs/issue113-head-chunk/off-product/warm/0.json
new file mode 100644
index 0000000..4e5e56d
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/warm/0.json
@@ -0,0 +1 @@
+{"index":0,"ids":[760,1156,6587,728],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"dbab0d12f25f2be2234d58d45f86219af62deb20e22b0d73a77aa5e7b722d082"},"hidden":{"shape":[1,4,10240],"sha256":"8f9aebc38d1a3e550bed6a4663eedac7a5ed3ce61d836d1cecdfa11da67ebd0d"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/warm/1.json b/docs/issue113-head-chunk/off-product/warm/1.json
new file mode 100644
index 0000000..6ff2118
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/warm/1.json
@@ -0,0 +1 @@
+{"index":1,"ids":[310,2962,24217,494],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"d6b8e6afe3c6f5af068f240e0e69b3875311aac6929b3d5a38ab3aeaa1bf227c"},"hidden":{"shape":[1,4,10240],"sha256":"52f20217ed43d8e59ddbe037cb77f56cc3d8ef473fd1963948fff6faf8ad4212"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/warm/2.json b/docs/issue113-head-chunk/off-product/warm/2.json
new file mode 100644
index 0000000..1b67c10
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/warm/2.json
@@ -0,0 +1 @@
+{"index":2,"ids":[13,353,2582,3804],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"048e9343d4268d98455a01f7c73ee47af78bc811ef9e7680a72cae59336b9989"},"hidden":{"shape":[1,4,10240],"sha256":"8e8b07fb20baa33ad64cc76ff64403a434d520c627a182a224cf033a5a14e5d4"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/off-product/warm/3.json b/docs/issue113-head-chunk/off-product/warm/3.json
new file mode 100644
index 0000000..2c6b980
--- /dev/null
+++ b/docs/issue113-head-chunk/off-product/warm/3.json
@@ -0,0 +1 @@
+{"index":3,"ids":[8334,29014,494],"verify":true,"logits":{"shape":[1,3,248320],"sha256":"82d671c6ac7b3a0952e60ef554ca670562fd01ef25c37bd10bc27025768ea0f2"},"hidden":{"shape":[1,3,10240],"sha256":"d1bc90f284364eae8bac80a5da6ab64c9c76d6994bb09ed463a17d67e25c016e"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/plan.log b/docs/issue113-head-chunk/plan.log
new file mode 100644
index 0000000..49566a6
--- /dev/null
+++ b/docs/issue113-head-chunk/plan.log
@@ -0,0 +1,5 @@
+
+running 1 test
+test engine::metal::qwen_mtplx::verify_sdpa::issue113_plan_and_policy_match_pinned_oracle ... ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 0.00s
diff --git a/docs/issue113-head-chunk/product/cold-arrays.json b/docs/issue113-head-chunk/product/cold-arrays.json
new file mode 100644
index 0000000..7b7944a
--- /dev/null
+++ b/docs/issue113-head-chunk/product/cold-arrays.json
@@ -0,0 +1 @@
+{"trunk":[[{"shape":[1,3,10240],"sha256":"1dadce7f74c7b48ad423170b8884bf5024c3d6dd91b2f9d800afca7d32679d7b","value_sha256":"1dadce7f74c7b48ad423170b8884bf5024c3d6dd91b2f9d800afca7d32679d7b"},{"shape":[1,48,128,128],"sha256":"d7436faf9d50e278f7fa095e9c217467020a6f534a48195775c212caa0ece52d","value_sha256":"d7436faf9d50e278f7fa095e9c217467020a6f534a48195775c212caa0ece52d"}],[{"shape":[1,3,10240],"sha256":"ca1b3cf93598b62575f01aea05ff0d0f087c00fdf1b63654626ef5ec3487715a","value_sha256":"ca1b3cf93598b62575f01aea05ff0d0f087c00fdf1b63654626ef5ec3487715a"},{"shape":[1,48,128,128],"sha256":"f1364bb8165b9189279b7769e4081f213c562f4e0541acc5962cf2ae9f42a831","value_sha256":"f1364bb8165b9189279b7769e4081f213c562f4e0541acc5962cf2ae9f42a831"},{"shape":[1,9,10240],"sha256":"c9110c820fc39f8c19c770594c15e0997693015090a3c7af4b6c8ec9a31ebc4e","value_sha256":"c9110c820fc39f8c19c770594c15e0997693015090a3c7af4b6c8ec9a31ebc4e"},{"shape":[1,2],"sha256":"9febaa06152dac026dba5f2c1151c11ff6a1b2f00e92278cdfd2905c87cf2c2f","value_sha256":"9febaa06152dac026dba5f2c1151c11ff6a1b2f00e92278cdfd2905c87cf2c2f"}],[{"shape":[1,3,10240],"sha256":"320fdcf9c7ab899133952e093d1f3de613ba5581b3aaad45c4ad7b1b4e43f1f3","value_sha256":"320fdcf9c7ab899133952e093d1f3de613ba5581b3aaad45c4ad7b1b4e43f1f3"},{"shape":[1,48,128,128],"sha256":"8cae4ba20fd0e54f8da130e5f8681ece7c54672b7a6404800cded8e64a977e08","value_sha256":"8cae4ba20fd0e54f8da130e5f8681ece7c54672b7a6404800cded8e64a977e08"}],[{"shape":[1,2,58,256],"sha256":"404510256f371f0bc2e7b8af2d1f7eb2ae0322d42eb9a0ee5827a9a5c8d489c0","value_sha256":"404510256f371f0bc2e7b8af2d1f7eb2ae0322d42eb9a0ee5827a9a5c8d489c0"},{"shape":[1,2,58,256],"sha256":"f14afc7fe1c5e4c62c84121e804e14aa484db378861904d4e234ffeef10252ef","value_sha256":"f14afc7fe1c5e4c62c84121e804e14aa484db378861904d4e234ffeef10252ef"},{"shape":[1,58,128],"sha256":"ca2d0b2f09e87fab8d0a6aede7073ec53bc37b584851f0342c76244f05ca3e12","value_sha256":"ca2d0b2f09e87fab8d0a6aede7073ec53bc37b584851f0342c76244f05ca3e12"},{"shape":[1,14,128],"sha256":"a609a4f52c9b3dc0fb5d6554019786bebecdad220bf5ae6b40d4939821f12c8b","value_sha256":"a609a4f52c9b3dc0fb5d6554019786bebecdad220bf5ae6b40d4939821f12c8b"}],[{"shape":[1,3,10240],"sha256":"c14f586416a8e772fa53002514ff980ab579060981040071c3e4636fc9762070","value_sha256":"c14f586416a8e772fa53002514ff980ab579060981040071c3e4636fc9762070"},{"shape":[1,48,128,128],"sha256":"3dd403eccb148601e1ab505dee16956ad59128e2d51f1c0ad5fbe20509b6d4fc","value_sha256":"3dd403eccb148601e1ab505dee16956ad59128e2d51f1c0ad5fbe20509b6d4fc"}],[{"shape":[1,3,10240],"sha256":"d3506442712f1e9433327907d1390a1166912fac6ce36d7c339a50f9adb570fb","value_sha256":"d3506442712f1e9433327907d1390a1166912fac6ce36d7c339a50f9adb570fb"},{"shape":[1,48,128,128],"sha256":"1d5a82226d350ca721e1b3a12e5c28ae175d9116e36fccda4218efca9fe699f1","value_sha256":"1d5a82226d350ca721e1b3a12e5c28ae175d9116e36fccda4218efca9fe699f1"}],[{"shape":[1,3,10240],"sha256":"e1f5a0ab427dca8d091996febccbeebd5cbc987d7189ed1badbfe57dc14f8747","value_sha256":"e1f5a0ab427dca8d091996febccbeebd5cbc987d7189ed1badbfe57dc14f8747"},{"shape":[1,48,128,128],"sha256":"17feff27d8744d9a3b9b115b3c723b650a581c22c32cef569d827bbc24de7206","value_sha256":"17feff27d8744d9a3b9b115b3c723b650a581c22c32cef569d827bbc24de7206"}],[{"shape":[1,2,58,256],"sha256":"364736a6d66b0aa29df064c406d3f7190f05fa385ffc10b031760541dc3cc8b5","value_sha256":"364736a6d66b0aa29df064c406d3f7190f05fa385ffc10b031760541dc3cc8b5"},{"shape":[1,2,58,256],"sha256":"22369eed488919ec8c7aea527654f8c8e335ef0a995620b06e0a1ec391f91b30","value_sha256":"22369eed488919ec8c7aea527654f8c8e335ef0a995620b06e0a1ec391f91b30"},{"shape":[1,58,128],"sha256":"20dd11e3fd1a655cf5d52fe260518fd0bc896ba7abf9709d590ab891d8e75120","value_sha256":"20dd11e3fd1a655cf5d52fe260518fd0bc896ba7abf9709d590ab891d8e75120"},{"shape":[1,14,128],"sha256":"378afa222e09564605d3ccd930cc86a2c4456ca99e122c8075def851bb453568","value_sha256":"378afa222e09564605d3ccd930cc86a2c4456ca99e122c8075def851bb453568"}],[{"shape":[1,3,10240],"sha256":"3ae083450733c47e30d8564cecca1056ab20d358acd48d3c81d63eae3577980d","value_sha256":"3ae083450733c47e30d8564cecca1056ab20d358acd48d3c81d63eae3577980d"},{"shape":[1,48,128,128],"sha256":"5044b96ba3ff509a76564d5a63a68de9744bdac90c2df5e191af4f12e5478f48","value_sha256":"5044b96ba3ff509a76564d5a63a68de9744bdac90c2df5e191af4f12e5478f48"}],[{"shape":[1,3,10240],"sha256":"b827dfbf602825d83f60b7f079a9cd3a67123ae1ad08ab046bd3878b411fa6c3","value_sha256":"b827dfbf602825d83f60b7f079a9cd3a67123ae1ad08ab046bd3878b411fa6c3"},{"shape":[1,48,128,128],"sha256":"70af0a5e7ad8fa8b63c40b40cc09c225378faf36ab648ffffc5c3bd4746802c7","value_sha256":"70af0a5e7ad8fa8b63c40b40cc09c225378faf36ab648ffffc5c3bd4746802c7"}],[{"shape":[1,3,10240],"sha256":"63f127bb20b6544b831d9cb3c476f4a093024fd425376a57538755be07e517df","value_sha256":"63f127bb20b6544b831d9cb3c476f4a093024fd425376a57538755be07e517df"},{"shape":[1,48,128,128],"sha256":"1d5d23064fbf2193615d4a1f83aa1897ed9c48e4c61a217abda7fd166302fea6","value_sha256":"1d5d23064fbf2193615d4a1f83aa1897ed9c48e4c61a217abda7fd166302fea6"}],[{"shape":[1,2,58,256],"sha256":"b13ce3a6861f643b03be4bb7ed27ed736001b445252123ba1b2c4b40b5dfe4a9","value_sha256":"b13ce3a6861f643b03be4bb7ed27ed736001b445252123ba1b2c4b40b5dfe4a9"},{"shape":[1,2,58,256],"sha256":"114605ae9e08f2c065eee34f25b8f11a4805e14d8de8845ad905fa33453bc2e2","value_sha256":"114605ae9e08f2c065eee34f25b8f11a4805e14d8de8845ad905fa33453bc2e2"},{"shape":[1,58,128],"sha256":"e94d105d822b6cd44c92e25fc08ef1e33af833f9175b714f3cc805bcdf25a6d2","value_sha256":"e94d105d822b6cd44c92e25fc08ef1e33af833f9175b714f3cc805bcdf25a6d2"},{"shape":[1,14,128],"sha256":"3ebde0b0615cff228c577a6f5f886dcedcf4f60671c4a2cedcb1ee440f286824","value_sha256":"3ebde0b0615cff228c577a6f5f886dcedcf4f60671c4a2cedcb1ee440f286824"}],[{"shape":[1,3,10240],"sha256":"14fe37048d44b26273932c62ffc6e992fe7fa1adb78be368fb5926c16759c51b","value_sha256":"14fe37048d44b26273932c62ffc6e992fe7fa1adb78be368fb5926c16759c51b"},{"shape":[1,48,128,128],"sha256":"ca98f06ce0c91c1c826aa7cd86491e46dabc370ac286a968f282b871357f81a1","value_sha256":"ca98f06ce0c91c1c826aa7cd86491e46dabc370ac286a968f282b871357f81a1"}],[{"shape":[1,3,10240],"sha256":"57ce536e225a28e6386be44194455e9b225c5a5ac12762bb1c98026a3e556020","value_sha256":"57ce536e225a28e6386be44194455e9b225c5a5ac12762bb1c98026a3e556020"},{"shape":[1,48,128,128],"sha256":"870ae60d413be6a21d74ebea374d389fba081d608cf1a96b4b6d7f87f5a16f7a","value_sha256":"870ae60d413be6a21d74ebea374d389fba081d608cf1a96b4b6d7f87f5a16f7a"}],[{"shape":[1,3,10240],"sha256":"e096b86584a0e8150c81139f61472e2b3df9cc968d1be3ac02efa94f45fd5836","value_sha256":"e096b86584a0e8150c81139f61472e2b3df9cc968d1be3ac02efa94f45fd5836"},{"shape":[1,48,128,128],"sha256":"81b5d40eed643f0577e92d4638d1db83a5b9dd681bb4ff369881286d57b0f809","value_sha256":"81b5d40eed643f0577e92d4638d1db83a5b9dd681bb4ff369881286d57b0f809"}],[{"shape":[1,2,58,256],"sha256":"288f45328bac8901c5b91da6a791961eb681ab260bb2819b51316dd2348f6c4b","value_sha256":"288f45328bac8901c5b91da6a791961eb681ab260bb2819b51316dd2348f6c4b"},{"shape":[1,2,58,256],"sha256":"9fe4421eb07afacd538e0bfc117656f6d2c2aeb9143a30b0f5caac9e15b5afb1","value_sha256":"9fe4421eb07afacd538e0bfc117656f6d2c2aeb9143a30b0f5caac9e15b5afb1"},{"shape":[1,58,128],"sha256":"82b986471d2e3e61d4425a5459813c57e3c5d48228c584d88b18caef96620463","value_sha256":"82b986471d2e3e61d4425a5459813c57e3c5d48228c584d88b18caef96620463"},{"shape":[1,14,128],"sha256":"f679a8da4c496a4753ae1c55b99f4a33b4a81575cc24e5247275c36581002f1e","value_sha256":"f679a8da4c496a4753ae1c55b99f4a33b4a81575cc24e5247275c36581002f1e"}],[{"shape":[1,3,10240],"sha256":"b64dc7d77ed93c7972187edf51a9d5098c86c0a1f79b7f40745efd0d062a8a50","value_sha256":"b64dc7d77ed93c7972187edf51a9d5098c86c0a1f79b7f40745efd0d062a8a50"},{"shape":[1,48,128,128],"sha256":"39659f369f3f579226f5209b37f7ae3034adb849b4369e17fae74fd9cdc540e5","value_sha256":"39659f369f3f579226f5209b37f7ae3034adb849b4369e17fae74fd9cdc540e5"}],[{"shape":[1,3,10240],"sha256":"33480e35ac640ecebfd73fa6f97843af70ea15d4a15db0a6502ab942a4691455","value_sha256":"33480e35ac640ecebfd73fa6f97843af70ea15d4a15db0a6502ab942a4691455"},{"shape":[1,48,128,128],"sha256":"3bffda871c549738052fbc2b8ecdd90b9ca5397aeac63054f2523c64a71899c7","value_sha256":"3bffda871c549738052fbc2b8ecdd90b9ca5397aeac63054f2523c64a71899c7"}],[{"shape":[1,3,10240],"sha256":"7c0531d30334402ea9baa94147f3995a20463509b5baffa403c2d401d394af50","value_sha256":"7c0531d30334402ea9baa94147f3995a20463509b5baffa403c2d401d394af50"},{"shape":[1,48,128,128],"sha256":"aa2fa5d57ef92b8095a69a246ebb8744de4d669f488aecb7b0a0e536706d6536","value_sha256":"aa2fa5d57ef92b8095a69a246ebb8744de4d669f488aecb7b0a0e536706d6536"}],[{"shape":[1,2,58,256],"sha256":"c7c26bc11eeb8a7de0825b8d488d521596281894533a2e88821ba41b2c191384","value_sha256":"c7c26bc11eeb8a7de0825b8d488d521596281894533a2e88821ba41b2c191384"},{"shape":[1,2,58,256],"sha256":"a7e0e167e69ce63ec559ae601e69bfda453fa5c30ef237c4d917e41b7cd01ee7","value_sha256":"a7e0e167e69ce63ec559ae601e69bfda453fa5c30ef237c4d917e41b7cd01ee7"},{"shape":[1,58,128],"sha256":"c3854c5f5bd9f2cf35faa627f41cdd9c8a4583be84efd94bceeccf067d255988","value_sha256":"c3854c5f5bd9f2cf35faa627f41cdd9c8a4583be84efd94bceeccf067d255988"},{"shape":[1,14,128],"sha256":"67b217df8253d94802fcba1bdbb254046f3b616aaddc05476b1739284e64bc4e","value_sha256":"67b217df8253d94802fcba1bdbb254046f3b616aaddc05476b1739284e64bc4e"}],[{"shape":[1,3,10240],"sha256":"684e734183485072aaf5e481309bbaa51f0fae7077c5b59696a3a147b31be7f6","value_sha256":"684e734183485072aaf5e481309bbaa51f0fae7077c5b59696a3a147b31be7f6"},{"shape":[1,48,128,128],"sha256":"3b80dc277bba852e61df738a8cbf675619a58379b3811c514b962b22cef571e6","value_sha256":"3b80dc277bba852e61df738a8cbf675619a58379b3811c514b962b22cef571e6"}],[{"shape":[1,3,10240],"sha256":"06b80690ec9b6fcc02531516ebd22310f50460ba6f4ddb71d5231bfb818a8db5","value_sha256":"06b80690ec9b6fcc02531516ebd22310f50460ba6f4ddb71d5231bfb818a8db5"},{"shape":[1,48,128,128],"sha256":"0dd85a84034d13e97c7b92359a087c1acee7892084d9db34be7472ec9f4d2c3c","value_sha256":"0dd85a84034d13e97c7b92359a087c1acee7892084d9db34be7472ec9f4d2c3c"}],[{"shape":[1,3,10240],"sha256":"117127944e478f6cc3a50165d9da8e859d545c62ebd818900e2a2351aea88bc9","value_sha256":"117127944e478f6cc3a50165d9da8e859d545c62ebd818900e2a2351aea88bc9"},{"shape":[1,48,128,128],"sha256":"08a6421b4f398d2cad951fc369e4ae6e5e7879b542fa664d819e23ea8242b0f8","value_sha256":"08a6421b4f398d2cad951fc369e4ae6e5e7879b542fa664d819e23ea8242b0f8"}],[{"shape":[1,2,58,256],"sha256":"a19b0652984777afff8bdde53e59beaf55a2d9b9ec2dc79704127aa08e06ec9e","value_sha256":"a19b0652984777afff8bdde53e59beaf55a2d9b9ec2dc79704127aa08e06ec9e"},{"shape":[1,2,58,256],"sha256":"52d98c3327c445aec7c14fb5b72b50160604b45cb4ce92ec0baee9672cdab0b7","value_sha256":"52d98c3327c445aec7c14fb5b72b50160604b45cb4ce92ec0baee9672cdab0b7"},{"shape":[1,58,128],"sha256":"51e2a204e22d64ef570e249db218ed78368455bca86b645e119e22078d5a2a33","value_sha256":"51e2a204e22d64ef570e249db218ed78368455bca86b645e119e22078d5a2a33"},{"shape":[1,14,128],"sha256":"7541ad385d8d6911f952a0004c36032d9ee35f7a868f2489c99fa0189160b431","value_sha256":"7541ad385d8d6911f952a0004c36032d9ee35f7a868f2489c99fa0189160b431"}],[{"shape":[1,3,10240],"sha256":"dc48c59206d04b0bb104c8faee2a41501e1c191ed448357c59fd10ef4df40e28","value_sha256":"dc48c59206d04b0bb104c8faee2a41501e1c191ed448357c59fd10ef4df40e28"},{"shape":[1,48,128,128],"sha256":"c109d35a2888553b351cefd6fff5ba27dc8561505d0c84af012ad42cc3425750","value_sha256":"c109d35a2888553b351cefd6fff5ba27dc8561505d0c84af012ad42cc3425750"}],[{"shape":[1,3,10240],"sha256":"92eff65318109629f251528e501881aef2d2e3413753909e3ae923d3c5fba032","value_sha256":"92eff65318109629f251528e501881aef2d2e3413753909e3ae923d3c5fba032"},{"shape":[1,48,128,128],"sha256":"dacb5e779a4be3e7c3cf644d3fb43d5f56e8e30aa301a11eb7c907e4521d7dc9","value_sha256":"dacb5e779a4be3e7c3cf644d3fb43d5f56e8e30aa301a11eb7c907e4521d7dc9"}],[{"shape":[1,3,10240],"sha256":"3796f81f11ff5205d0b233bd498766ae6ac7f5514d9789aa86f0fb40cdc93ec7","value_sha256":"3796f81f11ff5205d0b233bd498766ae6ac7f5514d9789aa86f0fb40cdc93ec7"},{"shape":[1,48,128,128],"sha256":"bc195f30a5583958a258f7ac19e9bac78219d3e257651efc55429243d450ad6c","value_sha256":"bc195f30a5583958a258f7ac19e9bac78219d3e257651efc55429243d450ad6c"}],[{"shape":[1,2,58,256],"sha256":"562c3f8f50061033f43e90c82471b4ec64af4b20fed7a5baad014c999d1140c7","value_sha256":"562c3f8f50061033f43e90c82471b4ec64af4b20fed7a5baad014c999d1140c7"},{"shape":[1,2,58,256],"sha256":"cc5e63029bd2fcd067d364b575178deef166d25a8c0f94ceb7aa51f06a99ff01","value_sha256":"cc5e63029bd2fcd067d364b575178deef166d25a8c0f94ceb7aa51f06a99ff01"},{"shape":[1,58,128],"sha256":"f86eb3d80bebab6e4848e34e19c76ef05f700a97ecb90927a0e50c76c112557f","value_sha256":"f86eb3d80bebab6e4848e34e19c76ef05f700a97ecb90927a0e50c76c112557f"},{"shape":[1,14,128],"sha256":"f7dd3775ff4a43c6678bc22cf62ab2511d747ee641232c6bfdfe64e3608375f5","value_sha256":"f7dd3775ff4a43c6678bc22cf62ab2511d747ee641232c6bfdfe64e3608375f5"}],[{"shape":[1,3,10240],"sha256":"4d1c2651c5bcbd6fab54c7aaf136921f1d92f685e88daf7105b430dadc4be1f7","value_sha256":"4d1c2651c5bcbd6fab54c7aaf136921f1d92f685e88daf7105b430dadc4be1f7"},{"shape":[1,48,128,128],"sha256":"18bd3841d44f58267c1815610134cbcff2fd74439e65c69f8d719a8f028b5aaf","value_sha256":"18bd3841d44f58267c1815610134cbcff2fd74439e65c69f8d719a8f028b5aaf"}],[{"shape":[1,3,10240],"sha256":"17f9c397b052b66a06b7e390fc5d36d9750f2a61b15d80ac4323ab39ad3f61b7","value_sha256":"17f9c397b052b66a06b7e390fc5d36d9750f2a61b15d80ac4323ab39ad3f61b7"},{"shape":[1,48,128,128],"sha256":"10632e222d82fc16c55d6223bf1a03685f5e94b45cebfce23288ca05f95a320d","value_sha256":"10632e222d82fc16c55d6223bf1a03685f5e94b45cebfce23288ca05f95a320d"}],[{"shape":[1,3,10240],"sha256":"c91dacfaf8b9750e728e79b2ba5eeda1b2fcb2df1c5c4588a47a8754d6c43cfd","value_sha256":"c91dacfaf8b9750e728e79b2ba5eeda1b2fcb2df1c5c4588a47a8754d6c43cfd"},{"shape":[1,48,128,128],"sha256":"e4fa07975bd87a6179be7033df5cc9fa8a478e48493446f2c17b741d6531b7c3","value_sha256":"e4fa07975bd87a6179be7033df5cc9fa8a478e48493446f2c17b741d6531b7c3"}],[{"shape":[1,2,58,256],"sha256":"3a25ebfaf5cdaa6ef8ecaf934b34821d3125c2f3357400880f623df62f7edb32","value_sha256":"3a25ebfaf5cdaa6ef8ecaf934b34821d3125c2f3357400880f623df62f7edb32"},{"shape":[1,2,58,256],"sha256":"71e96c5ff7b9d1c7543d029580d21c9b493d4afe41f80e8c815edcc5ee843f60","value_sha256":"71e96c5ff7b9d1c7543d029580d21c9b493d4afe41f80e8c815edcc5ee843f60"},{"shape":[1,58,128],"sha256":"80f06d5c9a19a27f361e06df95a1b8588b4fb47b5706cc60b4e7aa22b24a36b5","value_sha256":"80f06d5c9a19a27f361e06df95a1b8588b4fb47b5706cc60b4e7aa22b24a36b5"},{"shape":[1,14,128],"sha256":"0783a56950cff7738adf6033219cc7d80020a6a15bcb9c287c6209a71d5e5ede","value_sha256":"0783a56950cff7738adf6033219cc7d80020a6a15bcb9c287c6209a71d5e5ede"}],[{"shape":[1,3,10240],"sha256":"e77505ac0798549d33bb8b4cd408d35378efe19b2b1f93031aa168beab8608f7","value_sha256":"e77505ac0798549d33bb8b4cd408d35378efe19b2b1f93031aa168beab8608f7"},{"shape":[1,48,128,128],"sha256":"0db1d81b109ac60eefe540f40f3a8512f0c1fabc588217b5ee829c48fa72fea6","value_sha256":"0db1d81b109ac60eefe540f40f3a8512f0c1fabc588217b5ee829c48fa72fea6"}],[{"shape":[1,3,10240],"sha256":"fbc41fefe79e329d056b825b3866d891ad095c4332e81bc44939659020bc8468","value_sha256":"fbc41fefe79e329d056b825b3866d891ad095c4332e81bc44939659020bc8468"},{"shape":[1,48,128,128],"sha256":"4ce59d057d6b0487e1e38e5561453058d6da411c5b5230959654e1386fbd8725","value_sha256":"4ce59d057d6b0487e1e38e5561453058d6da411c5b5230959654e1386fbd8725"}],[{"shape":[1,3,10240],"sha256":"806dd8e81f93b8e3ce7adbdcc190f6801f4afa9c166376bc648f068057461840","value_sha256":"806dd8e81f93b8e3ce7adbdcc190f6801f4afa9c166376bc648f068057461840"},{"shape":[1,48,128,128],"sha256":"1a1772faf0e16f7b0df4af15d64363b1e8c588b70632d73a27ee44ad3410f187","value_sha256":"1a1772faf0e16f7b0df4af15d64363b1e8c588b70632d73a27ee44ad3410f187"}],[{"shape":[1,2,58,256],"sha256":"8b92cafe208cd569b60812ff1e6fa4c3b6c6a1b17d3ddff2c565d3e216b7f90e","value_sha256":"5ab571387c94df618ea820a7387c33b3247f5fc909cda536869df5cb97866c82"},{"shape":[1,2,58,256],"sha256":"85af4b52e414bc29896e357a425e077a16d979ba31a6f32a28092a161e26ae67","value_sha256":"85af4b52e414bc29896e357a425e077a16d979ba31a6f32a28092a161e26ae67"},{"shape":[1,58,128],"sha256":"809a8dd2d47c77079cf24b2054d6bf61ffb9dc4fa3c46f94410fbbb8fdf0507b","value_sha256":"809a8dd2d47c77079cf24b2054d6bf61ffb9dc4fa3c46f94410fbbb8fdf0507b"},{"shape":[1,14,128],"sha256":"578606efad4c39fa4bb7ae2d206e8d0fd281aa828a03e3533f2630b2e2d0b0c2","value_sha256":"578606efad4c39fa4bb7ae2d206e8d0fd281aa828a03e3533f2630b2e2d0b0c2"}],[{"shape":[1,3,10240],"sha256":"e8f8c4c1ef52516a82cb71911526a4856784d378399a2e057e685d65264549bc","value_sha256":"e8f8c4c1ef52516a82cb71911526a4856784d378399a2e057e685d65264549bc"},{"shape":[1,48,128,128],"sha256":"9736e985abde7df200f1572fc57fbb0d19f8e51148b440b262aa69093d9b239b","value_sha256":"9736e985abde7df200f1572fc57fbb0d19f8e51148b440b262aa69093d9b239b"}],[{"shape":[1,3,10240],"sha256":"99e9894eabeb9f5a2e2994957bc5202124c3c3f915c0c0c2bc30bb113a87029e","value_sha256":"99e9894eabeb9f5a2e2994957bc5202124c3c3f915c0c0c2bc30bb113a87029e"},{"shape":[1,48,128,128],"sha256":"9002f290afd2f040281e61eb1725dfdc131e7e05a97f543afb7edd92fdd735cf","value_sha256":"9002f290afd2f040281e61eb1725dfdc131e7e05a97f543afb7edd92fdd735cf"}],[{"shape":[1,3,10240],"sha256":"66350ac9a1b98fa633a496512c32b10aeb0b2d5b6de99614f5ca1f2ac02ab5e9","value_sha256":"66350ac9a1b98fa633a496512c32b10aeb0b2d5b6de99614f5ca1f2ac02ab5e9"},{"shape":[1,48,128,128],"sha256":"06647aa7504328c3f6df66c438d0d932013a5323a6079e2d78b7a882bb31cd46","value_sha256":"06647aa7504328c3f6df66c438d0d932013a5323a6079e2d78b7a882bb31cd46"}],[{"shape":[1,2,58,256],"sha256":"d65a7bfce5ec5be62e70c74ca030bd672608adb04fc5301d09f28f4463c85fd9","value_sha256":"d65a7bfce5ec5be62e70c74ca030bd672608adb04fc5301d09f28f4463c85fd9"},{"shape":[1,2,58,256],"sha256":"6fd6112e2fbf7cc0d56fdbb0d6da9984094f51cd78bde0c2e7990d7615647a74","value_sha256":"6fd6112e2fbf7cc0d56fdbb0d6da9984094f51cd78bde0c2e7990d7615647a74"},{"shape":[1,58,128],"sha256":"9f4aa3d737b07bf05f949487acbc1daa997ed9af67bc9a8fe5ee82e14494be15","value_sha256":"9f4aa3d737b07bf05f949487acbc1daa997ed9af67bc9a8fe5ee82e14494be15"},{"shape":[1,14,128],"sha256":"d40c032bd2d9f15a94abd176efeaf587529c618bf31499582c30d5100d9bee84","value_sha256":"d40c032bd2d9f15a94abd176efeaf587529c618bf31499582c30d5100d9bee84"}],[{"shape":[1,3,10240],"sha256":"d585d1e41742baa05f77a176c8af06e7533af8b052f09e199e7d4c711c6b882e","value_sha256":"d585d1e41742baa05f77a176c8af06e7533af8b052f09e199e7d4c711c6b882e"},{"shape":[1,48,128,128],"sha256":"e4afd2cc06f755bb60eb4be3c7edf563324af28c11b76a0020aa2980f9ce24a9","value_sha256":"e4afd2cc06f755bb60eb4be3c7edf563324af28c11b76a0020aa2980f9ce24a9"}],[{"shape":[1,3,10240],"sha256":"9eff2fee55092fba39f4deddeaeef54d1215c72552c08809e9e428a37c157736","value_sha256":"9eff2fee55092fba39f4deddeaeef54d1215c72552c08809e9e428a37c157736"},{"shape":[1,48,128,128],"sha256":"a9ca2a3d7c4cac69bdd8bf8b6156f164379fdc6429e40ef25cfdabdb9c00da8f","value_sha256":"a9ca2a3d7c4cac69bdd8bf8b6156f164379fdc6429e40ef25cfdabdb9c00da8f"}],[{"shape":[1,3,10240],"sha256":"d44ad9f394dc877cdd0d3c205deeb4b2ebd702c7a87a620634b31751bfb266bb","value_sha256":"d44ad9f394dc877cdd0d3c205deeb4b2ebd702c7a87a620634b31751bfb266bb"},{"shape":[1,48,128,128],"sha256":"7052a5c1141dbf3e91e810e5fc1b320d9717e3c8551177bd46d348c81e45b75d","value_sha256":"7052a5c1141dbf3e91e810e5fc1b320d9717e3c8551177bd46d348c81e45b75d"}],[{"shape":[1,2,58,256],"sha256":"97dd96869abbe849a7749137a7612c18f254b50be98c819a566be3e7846b6a84","value_sha256":"97dd96869abbe849a7749137a7612c18f254b50be98c819a566be3e7846b6a84"},{"shape":[1,2,58,256],"sha256":"c69a03c7aa2273c353590a3388d683a1640985869033236bae2d91f9f81056b6","value_sha256":"c69a03c7aa2273c353590a3388d683a1640985869033236bae2d91f9f81056b6"},{"shape":[1,58,128],"sha256":"12291918ff61b275c67fcf54b8d44edcd7e2cfea20da999af2fd47a3b45ca57f","value_sha256":"12291918ff61b275c67fcf54b8d44edcd7e2cfea20da999af2fd47a3b45ca57f"},{"shape":[1,14,128],"sha256":"bb079b8921efa9d1b58f1ad2982a07eceff60ef78948d10d52c3ad3727925dfe","value_sha256":"bb079b8921efa9d1b58f1ad2982a07eceff60ef78948d10d52c3ad3727925dfe"}],[{"shape":[1,3,10240],"sha256":"098c48808d556fd5fab41b66462d88288e516d52edcd9668b6b7c162face88da","value_sha256":"098c48808d556fd5fab41b66462d88288e516d52edcd9668b6b7c162face88da"},{"shape":[1,48,128,128],"sha256":"8ce941e7d99299790eb54b59215666a5cedaaa274ca395ff19439d049fb04d29","value_sha256":"8ce941e7d99299790eb54b59215666a5cedaaa274ca395ff19439d049fb04d29"}],[{"shape":[1,3,10240],"sha256":"6b1f5933c92907d094b3cc2904e59097099106b9ebe80cae6b088d8284030515","value_sha256":"6b1f5933c92907d094b3cc2904e59097099106b9ebe80cae6b088d8284030515"},{"shape":[1,48,128,128],"sha256":"b08e7fb85772f6792089da65cc7cd3f12182a675e5e0c439b993344ca00e6584","value_sha256":"b08e7fb85772f6792089da65cc7cd3f12182a675e5e0c439b993344ca00e6584"}],[{"shape":[1,3,10240],"sha256":"d074a8d26f367000244ece7495abd1a61b7fd039370c24c0125557b8ea8ddab8","value_sha256":"d074a8d26f367000244ece7495abd1a61b7fd039370c24c0125557b8ea8ddab8"},{"shape":[1,48,128,128],"sha256":"f7487a1cae16c09cfa1f92976ff638e980b34199ae45af566aa15a6d298aba4d","value_sha256":"f7487a1cae16c09cfa1f92976ff638e980b34199ae45af566aa15a6d298aba4d"}],[{"shape":[1,2,58,256],"sha256":"4a49860f7026c853613b9f4d79351c772a93afa28122a1fc26edfb7cb9adfcf9","value_sha256":"4a49860f7026c853613b9f4d79351c772a93afa28122a1fc26edfb7cb9adfcf9"},{"shape":[1,2,58,256],"sha256":"d3069338b58ae53af6c119828dffd8410ed5d4a5a3a4946c8919ef05822492a9","value_sha256":"d3069338b58ae53af6c119828dffd8410ed5d4a5a3a4946c8919ef05822492a9"},{"shape":[1,58,128],"sha256":"d489590d97486275f7bd02a89544af17a8a5eb12f901266ccea403e3cd5d06be","value_sha256":"d489590d97486275f7bd02a89544af17a8a5eb12f901266ccea403e3cd5d06be"},{"shape":[1,14,128],"sha256":"c38e69ad2c362a2c0f67c746869a1985dea3e2fa39927b25657cc48630dcffa9","value_sha256":"c38e69ad2c362a2c0f67c746869a1985dea3e2fa39927b25657cc48630dcffa9"}]],"mtp":[[{"shape":[1,2,57,256],"sha256":"e44ef0589e8c78a540adf0f21020e47e2827c55d9d5ad5ff3d88627fdb5b47ca","value_sha256":"e44ef0589e8c78a540adf0f21020e47e2827c55d9d5ad5ff3d88627fdb5b47ca"},{"shape":[1,2,57,256],"sha256":"d0251025d7458f944665e1e721de98953c890ebad29c19e26633fac583394335","value_sha256":"d0251025d7458f944665e1e721de98953c890ebad29c19e26633fac583394335"},{"shape":[1,57,128],"sha256":"98d70f2979e98734859d65edf10d90c89faafe9144ce58faf89177b2310b6eda","value_sha256":"98d70f2979e98734859d65edf10d90c89faafe9144ce58faf89177b2310b6eda"},{"shape":[1,14,128],"sha256":"a831fe6ca5fbc0d19c2511824c95fab7a3d9ebba4a7894539deed98ec86e7532","value_sha256":"a831fe6ca5fbc0d19c2511824c95fab7a3d9ebba4a7894539deed98ec86e7532"}]]}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/cold.json b/docs/issue113-head-chunk/product/cold.json
new file mode 100644
index 0000000..5abfc66
--- /dev/null
+++ b/docs/issue113-head-chunk/product/cold.json
@@ -0,0 +1 @@
+{"case":"cold","prompt":[248045,8678,198,24342,286,4879,369,716,310,3238,13,13262,678,7047,9522,321,10419,11,6992,5774,310,279,16198,1973,24366,24150,362,13,248046,198,248045,846,198,2427,494,799,310,5600,13,248046,198,248045,74455,198,248068,198],"tokens":[760,1156,6587,728,310,1698,494,799,310,5600,13,1061],"cached":0,"state_sha256":"ed07c44dec46cebe4471ac26a136c9d8ece3d88b4abeaa7f688247a13f3b1276"}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/cold/0.json b/docs/issue113-head-chunk/product/cold/0.json
new file mode 100644
index 0000000..c91a9a6
--- /dev/null
+++ b/docs/issue113-head-chunk/product/cold/0.json
@@ -0,0 +1 @@
+{"index":0,"ids":[760,1156,369,9859],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"e3ba038c8570911a72fadeba867f1c33e5103108ef9881bfcdc008907de90839"},"hidden":{"shape":[1,4,10240],"sha256":"b3c2ef7bb9c3dfa85748d7af883210ae4393bd1b5af5397f243d632fc437552b"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/cold/1.json b/docs/issue113-head-chunk/product/cold/1.json
new file mode 100644
index 0000000..a6b6e53
--- /dev/null
+++ b/docs/issue113-head-chunk/product/cold/1.json
@@ -0,0 +1 @@
+{"index":1,"ids":[6587,728,310,1698],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"955ff038997bab8ca6a84642b1ad8cf88dade0d2075d5ea11c736c9f74a7d563"},"hidden":{"shape":[1,4,10240],"sha256":"e3a029cc190a3cbdee1e2ec34bd94181f6013a6495c56b60dcccd67130fcfbd4"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/cold/2.json b/docs/issue113-head-chunk/product/cold/2.json
new file mode 100644
index 0000000..44c7c3e
--- /dev/null
+++ b/docs/issue113-head-chunk/product/cold/2.json
@@ -0,0 +1 @@
+{"index":2,"ids":[494,799,310,5600],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"e201b5e579d7c8f58432ae495c120c66cd595241046dd9cd32d42cab72009b3b"},"hidden":{"shape":[1,4,10240],"sha256":"d56aba5df3172aeca9d26bccfe6143d8045d30e5af45f1fc6815d61634f78d2c"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/cold/3.json b/docs/issue113-head-chunk/product/cold/3.json
new file mode 100644
index 0000000..db08cb5
--- /dev/null
+++ b/docs/issue113-head-chunk/product/cold/3.json
@@ -0,0 +1 @@
+{"index":3,"ids":[13,1061],"verify":true,"logits":{"shape":[1,2,248320],"sha256":"ecf42a91034a5bd6739a3d5bc94ffdcbac8214261f5712a3196bb6b99be67860"},"hidden":{"shape":[1,2,10240],"sha256":"0296bbc47e09e0312470a1ada7ef90177009d80cef5ba79398190177bb4be7c3"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/reopened-arrays.json b/docs/issue113-head-chunk/product/reopened-arrays.json
new file mode 100644
index 0000000..e4a3f91
--- /dev/null
+++ b/docs/issue113-head-chunk/product/reopened-arrays.json
@@ -0,0 +1 @@
+{"trunk":[[{"shape":[1,3,10240],"sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84","value_sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84"},{"shape":[1,48,128,128],"sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf","value_sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf"}],[{"shape":[1,3,10240],"sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232","value_sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232"},{"shape":[1,48,128,128],"sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf","value_sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf"},{"shape":[1,9,10240],"sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08","value_sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08"},{"shape":[1,2],"sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152","value_sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152"}],[{"shape":[1,3,10240],"sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2","value_sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2"},{"shape":[1,48,128,128],"sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34","value_sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34"}],[{"shape":[1,2,88,256],"sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f","value_sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f"},{"shape":[1,2,88,256],"sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24","value_sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24"},{"shape":[1,88,128],"sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748","value_sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748"},{"shape":[1,22,128],"sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a","value_sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a"}],[{"shape":[1,3,10240],"sha256":"23af29f2397aace825e958a5faa77cb734a968ef2a7a7f058f5dc543b582c847","value_sha256":"23af29f2397aace825e958a5faa77cb734a968ef2a7a7f058f5dc543b582c847"},{"shape":[1,48,128,128],"sha256":"d4c4e8b85d76cfb913593521150b144cb2bb6a6a3aed2bfe511fad2f716616ae","value_sha256":"d4c4e8b85d76cfb913593521150b144cb2bb6a6a3aed2bfe511fad2f716616ae"}],[{"shape":[1,3,10240],"sha256":"a342824984d9f603f852b6419dbd2be196dd3c2ad66b8d272ba1c723bfc58854","value_sha256":"a342824984d9f603f852b6419dbd2be196dd3c2ad66b8d272ba1c723bfc58854"},{"shape":[1,48,128,128],"sha256":"a46384ccc56e527d39df7582277baeba2229d208cc0529ecaeafd3e53b8d6db2","value_sha256":"a46384ccc56e527d39df7582277baeba2229d208cc0529ecaeafd3e53b8d6db2"}],[{"shape":[1,3,10240],"sha256":"21a78ff13a7b64234bd2db960ccfcae99cc575c3f42b185eb8831473666547e8","value_sha256":"21a78ff13a7b64234bd2db960ccfcae99cc575c3f42b185eb8831473666547e8"},{"shape":[1,48,128,128],"sha256":"69cea0bd54b8996f97bdc9f718cb4ddba75dd4c416a1f1590a1118a4503235ee","value_sha256":"69cea0bd54b8996f97bdc9f718cb4ddba75dd4c416a1f1590a1118a4503235ee"}],[{"shape":[1,2,88,256],"sha256":"75ca9e3382094e7fb1d1fb79c1b492164178b9dc028a75a084d82cba8c719b37","value_sha256":"75ca9e3382094e7fb1d1fb79c1b492164178b9dc028a75a084d82cba8c719b37"},{"shape":[1,2,88,256],"sha256":"2ceb5beed581a848481bd02f8d09b07901c37fbda93fd92d94ba1344579c65e0","value_sha256":"2ceb5beed581a848481bd02f8d09b07901c37fbda93fd92d94ba1344579c65e0"},{"shape":[1,88,128],"sha256":"bdbd34eb86fe22271437831df743bf44efb01a7dce240a9e6b6321d0c6356cde","value_sha256":"bdbd34eb86fe22271437831df743bf44efb01a7dce240a9e6b6321d0c6356cde"},{"shape":[1,22,128],"sha256":"bd0a7f30f06069bd50a87a8981cfd1657a172fe5407b78f666899f49aa1a96fc","value_sha256":"bd0a7f30f06069bd50a87a8981cfd1657a172fe5407b78f666899f49aa1a96fc"}],[{"shape":[1,3,10240],"sha256":"ef974274168dd5c569007e254179cf0ac745e7a7b28d2fbda9f06bc58c618158","value_sha256":"ef974274168dd5c569007e254179cf0ac745e7a7b28d2fbda9f06bc58c618158"},{"shape":[1,48,128,128],"sha256":"6706d6620ead67281abd40ad7239479a760992e42f0e2b8bfd2d3841c1bde52f","value_sha256":"6706d6620ead67281abd40ad7239479a760992e42f0e2b8bfd2d3841c1bde52f"}],[{"shape":[1,3,10240],"sha256":"1c6b6c06774bcfb8190cd48a437e99c9796d88d9fa6619f461f3fc90155ac15f","value_sha256":"1c6b6c06774bcfb8190cd48a437e99c9796d88d9fa6619f461f3fc90155ac15f"},{"shape":[1,48,128,128],"sha256":"78d2709f54d5687657883f8bd9216eb981d83ea4554959108c29dc0494e85f9f","value_sha256":"78d2709f54d5687657883f8bd9216eb981d83ea4554959108c29dc0494e85f9f"}],[{"shape":[1,3,10240],"sha256":"57977ed3fb0c1ed17d8b10bd2f525f28d63aad65eb434c513b490c9eb940352f","value_sha256":"57977ed3fb0c1ed17d8b10bd2f525f28d63aad65eb434c513b490c9eb940352f"},{"shape":[1,48,128,128],"sha256":"c33c4474957879b91be8e0ca721ce66ba6a6eb533028aa3cf9509a23620203ff","value_sha256":"c33c4474957879b91be8e0ca721ce66ba6a6eb533028aa3cf9509a23620203ff"}],[{"shape":[1,2,88,256],"sha256":"e19a00773b642b8e4e9309d6cb70cf123211fcd86ba6999301a2809115edb78f","value_sha256":"e19a00773b642b8e4e9309d6cb70cf123211fcd86ba6999301a2809115edb78f"},{"shape":[1,2,88,256],"sha256":"aa68365ecfd52961b728f8192765e1aa834cbe56ecddff87cc007d3b98cc5dac","value_sha256":"aa68365ecfd52961b728f8192765e1aa834cbe56ecddff87cc007d3b98cc5dac"},{"shape":[1,88,128],"sha256":"0ef422450f947767c661a851484b2c255722dee05525638e2faa342cc681e300","value_sha256":"0ef422450f947767c661a851484b2c255722dee05525638e2faa342cc681e300"},{"shape":[1,22,128],"sha256":"1ed1799c8e3d0f0656911bfdf233f591d3a2532856fa4701dc43032087ef1ab2","value_sha256":"1ed1799c8e3d0f0656911bfdf233f591d3a2532856fa4701dc43032087ef1ab2"}],[{"shape":[1,3,10240],"sha256":"33a2b31ca65d7f70d132117054cd457e87466b4939920dd29263564ea20d5c0c","value_sha256":"33a2b31ca65d7f70d132117054cd457e87466b4939920dd29263564ea20d5c0c"},{"shape":[1,48,128,128],"sha256":"8ba516367289ba29ea548682ca846fa0764bd3948ee856be8e5d19a06c86bea4","value_sha256":"8ba516367289ba29ea548682ca846fa0764bd3948ee856be8e5d19a06c86bea4"}],[{"shape":[1,3,10240],"sha256":"6e6b859184153d7b80f1e84a6e4224d521dae73c5bc9815076c037497fd9678a","value_sha256":"6e6b859184153d7b80f1e84a6e4224d521dae73c5bc9815076c037497fd9678a"},{"shape":[1,48,128,128],"sha256":"e57bcb1b6ccc4bb49d6c8c792fd854114696365377ca09e386aae9663fe62111","value_sha256":"e57bcb1b6ccc4bb49d6c8c792fd854114696365377ca09e386aae9663fe62111"}],[{"shape":[1,3,10240],"sha256":"aff052554b4f8095bac69952dc9637b28b28ccc03bf27ac26bb98da8be635769","value_sha256":"aff052554b4f8095bac69952dc9637b28b28ccc03bf27ac26bb98da8be635769"},{"shape":[1,48,128,128],"sha256":"2a2975148dfa647ca7af6ffaa3d6b82a6ee4e46f1b3d08e8d941c6073b8dc708","value_sha256":"2a2975148dfa647ca7af6ffaa3d6b82a6ee4e46f1b3d08e8d941c6073b8dc708"}],[{"shape":[1,2,88,256],"sha256":"c36614a9b99f84ef13a7550abd83a81bc1fe4b2c619418f7eb78ba5f630af56b","value_sha256":"c36614a9b99f84ef13a7550abd83a81bc1fe4b2c619418f7eb78ba5f630af56b"},{"shape":[1,2,88,256],"sha256":"c00d7aa398b8e627be8c98584509c04e2d668cbe85cff3c7032e46cb0148903c","value_sha256":"c00d7aa398b8e627be8c98584509c04e2d668cbe85cff3c7032e46cb0148903c"},{"shape":[1,88,128],"sha256":"156d95137a26b9ebb84b2ccaeea64e145d45f5557950c86da0d19e776fc1bc5f","value_sha256":"156d95137a26b9ebb84b2ccaeea64e145d45f5557950c86da0d19e776fc1bc5f"},{"shape":[1,22,128],"sha256":"945c939ef933319f802fde62d6db22728c35738821d21647ac64c3ee30ed65b9","value_sha256":"945c939ef933319f802fde62d6db22728c35738821d21647ac64c3ee30ed65b9"}],[{"shape":[1,3,10240],"sha256":"457d091ab375ff69cd61041c116a903d6a7ecac75a12b3f77d92411b05dd19cb","value_sha256":"457d091ab375ff69cd61041c116a903d6a7ecac75a12b3f77d92411b05dd19cb"},{"shape":[1,48,128,128],"sha256":"6d43c9ebfc0ad35176429ffb7379a80f0d1d4e44830655a1a66408230abe90bf","value_sha256":"6d43c9ebfc0ad35176429ffb7379a80f0d1d4e44830655a1a66408230abe90bf"}],[{"shape":[1,3,10240],"sha256":"8edba564162e6152c3cada686e749f37f4ba50a999b1c1cb93a41fc13132e191","value_sha256":"8edba564162e6152c3cada686e749f37f4ba50a999b1c1cb93a41fc13132e191"},{"shape":[1,48,128,128],"sha256":"cc1e836d7be25016bdd272aea2674735026363b02a9e49b088ca54c9e419ecaf","value_sha256":"cc1e836d7be25016bdd272aea2674735026363b02a9e49b088ca54c9e419ecaf"}],[{"shape":[1,3,10240],"sha256":"8ca7077aef12bb5981f3dcbf42f45d8d961ad65248f74bca91f7c271a3c73b9c","value_sha256":"8ca7077aef12bb5981f3dcbf42f45d8d961ad65248f74bca91f7c271a3c73b9c"},{"shape":[1,48,128,128],"sha256":"0efcb17ce919b684b3779274c474c8ffc89e36ccd5a90aa4763f1fa311865163","value_sha256":"0efcb17ce919b684b3779274c474c8ffc89e36ccd5a90aa4763f1fa311865163"}],[{"shape":[1,2,88,256],"sha256":"4eb822ebd43a6b56547196de8c04b93f3b5b20c3b063ba82b05b6b2fc00a1e08","value_sha256":"4eb822ebd43a6b56547196de8c04b93f3b5b20c3b063ba82b05b6b2fc00a1e08"},{"shape":[1,2,88,256],"sha256":"6f6037b65cf72e99ae95df7d662906b7236d6648595e6c6021d588b06c901d50","value_sha256":"6f6037b65cf72e99ae95df7d662906b7236d6648595e6c6021d588b06c901d50"},{"shape":[1,88,128],"sha256":"b13b714ace3ec8a73168fff81166f267a30bb3b3dca199ebd6d9f297e22d3cbb","value_sha256":"b13b714ace3ec8a73168fff81166f267a30bb3b3dca199ebd6d9f297e22d3cbb"},{"shape":[1,22,128],"sha256":"06e5ff478478ee1a710fb892424172717fd0881fd9e39d9757635ebfef6a17af","value_sha256":"06e5ff478478ee1a710fb892424172717fd0881fd9e39d9757635ebfef6a17af"}],[{"shape":[1,3,10240],"sha256":"99ea167ef3b9cdb4b9c105348e814fd3d1a0e2022c0aeab181f7f38340f3b432","value_sha256":"99ea167ef3b9cdb4b9c105348e814fd3d1a0e2022c0aeab181f7f38340f3b432"},{"shape":[1,48,128,128],"sha256":"11ef876b8c15cc91454277e92cdabfbfe91c3a35d1c1a2e8aeec0cc224113b9e","value_sha256":"11ef876b8c15cc91454277e92cdabfbfe91c3a35d1c1a2e8aeec0cc224113b9e"}],[{"shape":[1,3,10240],"sha256":"baf0923981974ff892e115a69049618f8e325b8beff769bec83f6c803a59f623","value_sha256":"baf0923981974ff892e115a69049618f8e325b8beff769bec83f6c803a59f623"},{"shape":[1,48,128,128],"sha256":"cd9b63ead383d25a184cab17d944e3a9b91923a221322a7c97a36be563a526cd","value_sha256":"cd9b63ead383d25a184cab17d944e3a9b91923a221322a7c97a36be563a526cd"}],[{"shape":[1,3,10240],"sha256":"a454bb59c8aac167b6826584fa0ae561da1ac2e3819f1aa046a516876fc6c506","value_sha256":"a454bb59c8aac167b6826584fa0ae561da1ac2e3819f1aa046a516876fc6c506"},{"shape":[1,48,128,128],"sha256":"b8d2047192b894f097eeeb0e943d8e732fba9481cc20c4f24779c4c8a5d94091","value_sha256":"b8d2047192b894f097eeeb0e943d8e732fba9481cc20c4f24779c4c8a5d94091"}],[{"shape":[1,2,88,256],"sha256":"fcb08c9786c93de86d3c6404cb9e674521cfa7ac41199df8d3da886d32d76754","value_sha256":"fcb08c9786c93de86d3c6404cb9e674521cfa7ac41199df8d3da886d32d76754"},{"shape":[1,2,88,256],"sha256":"2869ad71bb6f0aec650fa614482789563782f48a3280030fc9b7138dc48e2ee3","value_sha256":"2869ad71bb6f0aec650fa614482789563782f48a3280030fc9b7138dc48e2ee3"},{"shape":[1,88,128],"sha256":"a301869179deb4c07e4c125b29957dc6f635c08e4318f223d78b7a5afeaa38a8","value_sha256":"a301869179deb4c07e4c125b29957dc6f635c08e4318f223d78b7a5afeaa38a8"},{"shape":[1,22,128],"sha256":"bf77e99e57d4c548e7c6be3375f17821a116837cf9bd09cebc5dc7ca00e9e26c","value_sha256":"bf77e99e57d4c548e7c6be3375f17821a116837cf9bd09cebc5dc7ca00e9e26c"}],[{"shape":[1,3,10240],"sha256":"5a141665706b652733a7458367775637fb0e5593f988e5ed5932a61bbdee5528","value_sha256":"5a141665706b652733a7458367775637fb0e5593f988e5ed5932a61bbdee5528"},{"shape":[1,48,128,128],"sha256":"e1bb2841a6b57c9720ea50396036633559a4e9501bc3ddf5a67836f12ba5b62c","value_sha256":"e1bb2841a6b57c9720ea50396036633559a4e9501bc3ddf5a67836f12ba5b62c"}],[{"shape":[1,3,10240],"sha256":"67c1f4bd6bb14f381283c71cbdd55966af1784202fb75c18c4ec8536fe1f0e65","value_sha256":"67c1f4bd6bb14f381283c71cbdd55966af1784202fb75c18c4ec8536fe1f0e65"},{"shape":[1,48,128,128],"sha256":"847a97f31bccf45c8de961089477aeb2d7af44e2f9029a5785afc6ca5a9027d5","value_sha256":"847a97f31bccf45c8de961089477aeb2d7af44e2f9029a5785afc6ca5a9027d5"}],[{"shape":[1,3,10240],"sha256":"d7d0157be88198115d2ea8936388ee7c9ab2730e10654a046c2d7fb971981bc2","value_sha256":"d7d0157be88198115d2ea8936388ee7c9ab2730e10654a046c2d7fb971981bc2"},{"shape":[1,48,128,128],"sha256":"3f4812e18648465cfc0e10525bd16bc32a32ad74803986f9e1ce6076b8786be2","value_sha256":"3f4812e18648465cfc0e10525bd16bc32a32ad74803986f9e1ce6076b8786be2"}],[{"shape":[1,2,88,256],"sha256":"fb8b1ccb372fda0602f0c5838d51ab46ce2cb3aecf5673413e206af914b9ee85","value_sha256":"fb8b1ccb372fda0602f0c5838d51ab46ce2cb3aecf5673413e206af914b9ee85"},{"shape":[1,2,88,256],"sha256":"a6f93d1c84f1d71753d1a7ba4cd4590289e52aa9981711853ce636074555184d","value_sha256":"a6f93d1c84f1d71753d1a7ba4cd4590289e52aa9981711853ce636074555184d"},{"shape":[1,88,128],"sha256":"7f4806907e909a6b9a5cb17ca721c07fe77ac3aa365a6940d86d86d5a5bc8a52","value_sha256":"7f4806907e909a6b9a5cb17ca721c07fe77ac3aa365a6940d86d86d5a5bc8a52"},{"shape":[1,22,128],"sha256":"a260f656750162410ea2dfec7b5ae0ee403ae29da7a57d905fd93a823a345f9e","value_sha256":"a260f656750162410ea2dfec7b5ae0ee403ae29da7a57d905fd93a823a345f9e"}],[{"shape":[1,3,10240],"sha256":"ffd995c891c8983d6cf2bba9895627884913d0532e8ee561bc69987a2e6644cc","value_sha256":"ffd995c891c8983d6cf2bba9895627884913d0532e8ee561bc69987a2e6644cc"},{"shape":[1,48,128,128],"sha256":"ebb1e44092a7f4843b194d50054c18229128540d8869b528666b2c0826af72ff","value_sha256":"ebb1e44092a7f4843b194d50054c18229128540d8869b528666b2c0826af72ff"}],[{"shape":[1,3,10240],"sha256":"268cf105ff4cc226cf4c40a46eef26650fdd4dea1e398c81f4fd456813092813","value_sha256":"268cf105ff4cc226cf4c40a46eef26650fdd4dea1e398c81f4fd456813092813"},{"shape":[1,48,128,128],"sha256":"01b30ea217235762534adb10875be7811dc4a031d841142a8e71edf7a02f6202","value_sha256":"01b30ea217235762534adb10875be7811dc4a031d841142a8e71edf7a02f6202"}],[{"shape":[1,3,10240],"sha256":"965ed9a6a79f88c2c7a610e4ff49a3face0cbb1c13814135cf206e1c0f96e326","value_sha256":"965ed9a6a79f88c2c7a610e4ff49a3face0cbb1c13814135cf206e1c0f96e326"},{"shape":[1,48,128,128],"sha256":"01c25d008bbd248eac36e70f0dbbb72597105b56b58fafb8cc348011ad3aeb12","value_sha256":"01c25d008bbd248eac36e70f0dbbb72597105b56b58fafb8cc348011ad3aeb12"}],[{"shape":[1,2,88,256],"sha256":"88cda706be1e27ef1251b1fedf30cfa201a305502a09134e3282736e45c9f8aa","value_sha256":"88cda706be1e27ef1251b1fedf30cfa201a305502a09134e3282736e45c9f8aa"},{"shape":[1,2,88,256],"sha256":"78bb9fea678d59bf723d846866bc46b062af3188b3a5dfb5dae0a996e8e4f719","value_sha256":"78bb9fea678d59bf723d846866bc46b062af3188b3a5dfb5dae0a996e8e4f719"},{"shape":[1,88,128],"sha256":"39430166fbe23327a5d078ba372eb18a3fadc82253886d7ad186bdf95c722228","value_sha256":"39430166fbe23327a5d078ba372eb18a3fadc82253886d7ad186bdf95c722228"},{"shape":[1,22,128],"sha256":"28089aa680484b36139d5afee6b15c0b0acfd3ae528f36d9b6e6b18e65db5bb6","value_sha256":"28089aa680484b36139d5afee6b15c0b0acfd3ae528f36d9b6e6b18e65db5bb6"}],[{"shape":[1,3,10240],"sha256":"db1c3cbab49e324e65452b31ba9656bc95ca70dd2bf32f1af1dde7a002d59d29","value_sha256":"db1c3cbab49e324e65452b31ba9656bc95ca70dd2bf32f1af1dde7a002d59d29"},{"shape":[1,48,128,128],"sha256":"f809b0171e94c5cec2ca972b733d6c21ef487195867669c9acc3700696331f3e","value_sha256":"f809b0171e94c5cec2ca972b733d6c21ef487195867669c9acc3700696331f3e"}],[{"shape":[1,3,10240],"sha256":"61dcb190d48b9b056f3d9e6b3f6da8ca1327615b339ccd7900d56dac40c3528f","value_sha256":"61dcb190d48b9b056f3d9e6b3f6da8ca1327615b339ccd7900d56dac40c3528f"},{"shape":[1,48,128,128],"sha256":"33b6e4f3dbb38a75049c7e0c8748ddfa229b7a97c8d5ea8bfffa1810b3ff1608","value_sha256":"33b6e4f3dbb38a75049c7e0c8748ddfa229b7a97c8d5ea8bfffa1810b3ff1608"}],[{"shape":[1,3,10240],"sha256":"8fb2f7b4fcb9b795caa3fe87ba09210b68f95cd1f8e4ee598fa2d8ef0a1cddef","value_sha256":"8fb2f7b4fcb9b795caa3fe87ba09210b68f95cd1f8e4ee598fa2d8ef0a1cddef"},{"shape":[1,48,128,128],"sha256":"cfb8c0a67de5e3c2a08c49ab89a95cea396c4b5530dece4bb6dc1e292f704281","value_sha256":"cfb8c0a67de5e3c2a08c49ab89a95cea396c4b5530dece4bb6dc1e292f704281"}],[{"shape":[1,2,88,256],"sha256":"5a0da5529eeb692ff189e85986ed3bbafd96a99444f8d975b1349975c52d6946","value_sha256":"23c735b3da2aa189a08c35de556f3d638ca42fb360f62c85ee2018f85c2fffa7"},{"shape":[1,2,88,256],"sha256":"4aa114f8361c5bde1fc959817ce2b5d891edf02eb7a1ee1000c97e6faca78b95","value_sha256":"4aa114f8361c5bde1fc959817ce2b5d891edf02eb7a1ee1000c97e6faca78b95"},{"shape":[1,88,128],"sha256":"aa44ae0f9c64d7b81a5a2d03b58de1f19bd11a948aa61a6c193a4b6e5469970b","value_sha256":"aa44ae0f9c64d7b81a5a2d03b58de1f19bd11a948aa61a6c193a4b6e5469970b"},{"shape":[1,22,128],"sha256":"a0995be864606aa29529e0d797f35c31d0d3de9ee623d76fa45f5ce4b87dd4f4","value_sha256":"a0995be864606aa29529e0d797f35c31d0d3de9ee623d76fa45f5ce4b87dd4f4"}],[{"shape":[1,3,10240],"sha256":"e4ff3d6eba578ef7c52c05e27a1efd0d1a35ab643b64c82c7044c90327e2d010","value_sha256":"e4ff3d6eba578ef7c52c05e27a1efd0d1a35ab643b64c82c7044c90327e2d010"},{"shape":[1,48,128,128],"sha256":"7de0be2d7c867a38a323272081641806101cc5d14c8f37a65dca7b86979b8af0","value_sha256":"7de0be2d7c867a38a323272081641806101cc5d14c8f37a65dca7b86979b8af0"}],[{"shape":[1,3,10240],"sha256":"b7e6512706b5582f5f1a2d3409b926552694eb986a7344ec2ab2011fe3ea0e6e","value_sha256":"b7e6512706b5582f5f1a2d3409b926552694eb986a7344ec2ab2011fe3ea0e6e"},{"shape":[1,48,128,128],"sha256":"d46cf1f72475f437db15506d1c159ca2ea7580ef240fca13817995104ac6938b","value_sha256":"d46cf1f72475f437db15506d1c159ca2ea7580ef240fca13817995104ac6938b"}],[{"shape":[1,3,10240],"sha256":"df17d1305bbcaf474ae22908d0897d871424db73ab4de5e72ab5a550e7666257","value_sha256":"df17d1305bbcaf474ae22908d0897d871424db73ab4de5e72ab5a550e7666257"},{"shape":[1,48,128,128],"sha256":"0fd02e86361404ca4fb55f6d756f14787e0f37ad03e9648ee4d5130c5697dc55","value_sha256":"0fd02e86361404ca4fb55f6d756f14787e0f37ad03e9648ee4d5130c5697dc55"}],[{"shape":[1,2,88,256],"sha256":"8966f22a4de9fd3ed9035e602c6b5730d6916daf1d1041411e6945afa4858440","value_sha256":"8966f22a4de9fd3ed9035e602c6b5730d6916daf1d1041411e6945afa4858440"},{"shape":[1,2,88,256],"sha256":"8ea24411ee21fd6c5e50c6ce88e25ed9a7f4433852114d538c402d54531bd7a2","value_sha256":"8ea24411ee21fd6c5e50c6ce88e25ed9a7f4433852114d538c402d54531bd7a2"},{"shape":[1,88,128],"sha256":"7d2134a084d27f8072754a9fec03facd14ffae3d38f45f61ad1ac2a24d7b8175","value_sha256":"7d2134a084d27f8072754a9fec03facd14ffae3d38f45f61ad1ac2a24d7b8175"},{"shape":[1,22,128],"sha256":"25e429a491b003408e95940f5f037dd9abde7bd38c748d06fe9e7d7a0cd14f8c","value_sha256":"25e429a491b003408e95940f5f037dd9abde7bd38c748d06fe9e7d7a0cd14f8c"}],[{"shape":[1,3,10240],"sha256":"fe0011e0b328f04c60869ace5e1e16df6739793dcd51a1309df42dc5f639cf86","value_sha256":"fe0011e0b328f04c60869ace5e1e16df6739793dcd51a1309df42dc5f639cf86"},{"shape":[1,48,128,128],"sha256":"03108befe82b6db08114de55f1efbb9cf0b00ffd3c2701e0cfc334d7e8b45d2f","value_sha256":"03108befe82b6db08114de55f1efbb9cf0b00ffd3c2701e0cfc334d7e8b45d2f"}],[{"shape":[1,3,10240],"sha256":"70d9e8f0766035db9a6df27d69b345fa60db120d631ec7aa5817d754be7be781","value_sha256":"70d9e8f0766035db9a6df27d69b345fa60db120d631ec7aa5817d754be7be781"},{"shape":[1,48,128,128],"sha256":"3ae75e227d68f85d37cf6de4ef43f6cd7d610b7b1f298d2ab41e65166892ef69","value_sha256":"3ae75e227d68f85d37cf6de4ef43f6cd7d610b7b1f298d2ab41e65166892ef69"}],[{"shape":[1,3,10240],"sha256":"4281b65d24625eae42daed8caff45e34d07270ca8722bca5aac660202256c32f","value_sha256":"4281b65d24625eae42daed8caff45e34d07270ca8722bca5aac660202256c32f"},{"shape":[1,48,128,128],"sha256":"2b26c641c9eacee9c66760be14478123068d4ae74a59b235b62c3c6d5565bbe0","value_sha256":"2b26c641c9eacee9c66760be14478123068d4ae74a59b235b62c3c6d5565bbe0"}],[{"shape":[1,2,88,256],"sha256":"0156ed0676481c1e9534a55b2779c80b26b1b4262122afd276991672f4d13dba","value_sha256":"0156ed0676481c1e9534a55b2779c80b26b1b4262122afd276991672f4d13dba"},{"shape":[1,2,88,256],"sha256":"eeee6171061647ed1677ebe59bb49b2ca15e0dffee5d936bcfe97a77a05207d0","value_sha256":"eeee6171061647ed1677ebe59bb49b2ca15e0dffee5d936bcfe97a77a05207d0"},{"shape":[1,88,128],"sha256":"a9fa7284631037665443e03494b1bb4fe32f25791f308903fe950667b76907f4","value_sha256":"a9fa7284631037665443e03494b1bb4fe32f25791f308903fe950667b76907f4"},{"shape":[1,22,128],"sha256":"dbc72d1d83f28442a5a80c547c187eacc1a7f2c7f494dff16fe15074c4e5a9be","value_sha256":"dbc72d1d83f28442a5a80c547c187eacc1a7f2c7f494dff16fe15074c4e5a9be"}],[{"shape":[1,3,10240],"sha256":"7093d030a67627399bfb064699482f914d23366eefee4265b235d8d85a230978","value_sha256":"7093d030a67627399bfb064699482f914d23366eefee4265b235d8d85a230978"},{"shape":[1,48,128,128],"sha256":"bebb33ca1fae20f6cd9b83af7272740020323f1bbe3fe62c4978c6e2448cb660","value_sha256":"bebb33ca1fae20f6cd9b83af7272740020323f1bbe3fe62c4978c6e2448cb660"}],[{"shape":[1,3,10240],"sha256":"c03f0185a17ccf082894797d1716f5cd14f93f960607e5ddf94a40ccc1105038","value_sha256":"c03f0185a17ccf082894797d1716f5cd14f93f960607e5ddf94a40ccc1105038"},{"shape":[1,48,128,128],"sha256":"d40a25e169416b62c7c24693846f6d0e49ae32191a0b25b27d556de72d0ba41b","value_sha256":"d40a25e169416b62c7c24693846f6d0e49ae32191a0b25b27d556de72d0ba41b"}],[{"shape":[1,3,10240],"sha256":"f503b18b3b47f5e21499c5e9510d39b86e89a05631ac0b28beaf1548c895a27e","value_sha256":"f503b18b3b47f5e21499c5e9510d39b86e89a05631ac0b28beaf1548c895a27e"},{"shape":[1,48,128,128],"sha256":"e37360995b834ed990fb3a63b83684efc045b2035dd7de06f9d75ab9eda91914","value_sha256":"e37360995b834ed990fb3a63b83684efc045b2035dd7de06f9d75ab9eda91914"}],[{"shape":[1,2,88,256],"sha256":"716c40276b1c9e46ba8f7e76fd920ed25f506706aff59be263123ab1cfe61aae","value_sha256":"716c40276b1c9e46ba8f7e76fd920ed25f506706aff59be263123ab1cfe61aae"},{"shape":[1,2,88,256],"sha256":"6242a9c8793306892addc2ae9762c417cdfb350ccbc93aa4c90cb179cbf930f8","value_sha256":"6242a9c8793306892addc2ae9762c417cdfb350ccbc93aa4c90cb179cbf930f8"},{"shape":[1,88,128],"sha256":"83443016576a524ac3bc23bb0e7625f2659c0ff35d55d9cc34f6c541027d085b","value_sha256":"83443016576a524ac3bc23bb0e7625f2659c0ff35d55d9cc34f6c541027d085b"},{"shape":[1,22,128],"sha256":"53bd6c3f3da8bc5c19e1d6160ca9dec71925b79be161ded2088f7a7e75b4de34","value_sha256":"53bd6c3f3da8bc5c19e1d6160ca9dec71925b79be161ded2088f7a7e75b4de34"}]],"mtp":[[{"shape":[1,2,87,256],"sha256":"e9aade6cf8cedfff8b01a5836c4bb5164ee3dc4c276b24e1ded346bdf4d8d959","value_sha256":"e9aade6cf8cedfff8b01a5836c4bb5164ee3dc4c276b24e1ded346bdf4d8d959"},{"shape":[1,2,87,256],"sha256":"77e78cf6c7b1f08ec2876307e5166221fec26152df863daf44ccaffdb21fa079","value_sha256":"77e78cf6c7b1f08ec2876307e5166221fec26152df863daf44ccaffdb21fa079"},{"shape":[1,87,128],"sha256":"1f1b3729be3d37dd1d1ae5b3cb28e228f49918da266f73af3bb2ae0b1ab00cff","value_sha256":"1f1b3729be3d37dd1d1ae5b3cb28e228f49918da266f73af3bb2ae0b1ab00cff"},{"shape":[1,21,128],"sha256":"3fce4dd0cf1e1f69b65ef85f94813066b85ce04efa10903f394c90133ee9055e","value_sha256":"3fce4dd0cf1e1f69b65ef85f94813066b85ce04efa10903f394c90133ee9055e"}]]}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/reopened.json b/docs/issue113-head-chunk/product/reopened.json
new file mode 100644
index 0000000..f7ba3b5
--- /dev/null
+++ b/docs/issue113-head-chunk/product/reopened.json
@@ -0,0 +1 @@
+{"case":"reopened","prompt":[248045,8678,198,24342,286,4879,369,716,310,3238,13,13262,678,7047,9522,321,10419,11,6992,5774,310,279,16198,1973,24366,24150,362,13,248046,198,248045,846,198,2427,494,799,310,5600,13,248046,198,248045,74455,198,248068,198,760,1156,6587,728,310,1698,494,799,310,5600,13,1061,198,248069,271,248046,198,248045,846,198,22791,24217,13,248046,198,248045,74455,198,248068,198],"tokens":[760,1156,6587,728,310,2962,24217,13,353,8334,29014,494],"cached":58,"state_sha256":"257714fbbf6f6a92b5562f554fae93c36abaff2e72bac7ad61a0d874e14e8234"}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/reopened/0.json b/docs/issue113-head-chunk/product/reopened/0.json
new file mode 100644
index 0000000..71c57f2
--- /dev/null
+++ b/docs/issue113-head-chunk/product/reopened/0.json
@@ -0,0 +1 @@
+{"index":0,"ids":[760,1156,6587,728],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"0d002e5ab00ee76250e5deca14358353c777fd1f1804937070ca229f74cdc9f6"},"hidden":{"shape":[1,4,10240],"sha256":"cb6bc68f9d37d0ffb544b2c635144d8f91a715c5172fa31a32be7aab84d5941b"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/reopened/1.json b/docs/issue113-head-chunk/product/reopened/1.json
new file mode 100644
index 0000000..87ebb2e
--- /dev/null
+++ b/docs/issue113-head-chunk/product/reopened/1.json
@@ -0,0 +1 @@
+{"index":1,"ids":[310,2962,24217,494],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"e052a03d4908e670c534c18453447dcb7190bf180dd281ffd3ace950675174e1"},"hidden":{"shape":[1,4,10240],"sha256":"2caa9c5b1d863080a4ec81e55103e9b3429261933f13cac971dfc09a8a1c130f"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/reopened/2.json b/docs/issue113-head-chunk/product/reopened/2.json
new file mode 100644
index 0000000..5fbd30f
--- /dev/null
+++ b/docs/issue113-head-chunk/product/reopened/2.json
@@ -0,0 +1 @@
+{"index":2,"ids":[13,353,2582,3804],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"f64c3236f08fa43f85ee491cc1121d5f584dfd1fbcd58b1bf9ef69b441813036"},"hidden":{"shape":[1,4,10240],"sha256":"eba14099db5411740f69183e22dbe7d50aef3e5cfde093319eaa39008fbd11c8"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/reopened/3.json b/docs/issue113-head-chunk/product/reopened/3.json
new file mode 100644
index 0000000..833f0a3
--- /dev/null
+++ b/docs/issue113-head-chunk/product/reopened/3.json
@@ -0,0 +1 @@
+{"index":3,"ids":[8334,29014,494],"verify":true,"logits":{"shape":[1,3,248320],"sha256":"6821599090afcd8e042f955477276a52843d0087c6074a1a77a43b9768cd2c0c"},"hidden":{"shape":[1,3,10240],"sha256":"e358aa99aa0738c30f102ad646ebef5244b7740ca7ccaa3804469e3dc8ac217c"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/warm-arrays.json b/docs/issue113-head-chunk/product/warm-arrays.json
new file mode 100644
index 0000000..e4a3f91
--- /dev/null
+++ b/docs/issue113-head-chunk/product/warm-arrays.json
@@ -0,0 +1 @@
+{"trunk":[[{"shape":[1,3,10240],"sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84","value_sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84"},{"shape":[1,48,128,128],"sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf","value_sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf"}],[{"shape":[1,3,10240],"sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232","value_sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232"},{"shape":[1,48,128,128],"sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf","value_sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf"},{"shape":[1,9,10240],"sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08","value_sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08"},{"shape":[1,2],"sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152","value_sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152"}],[{"shape":[1,3,10240],"sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2","value_sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2"},{"shape":[1,48,128,128],"sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34","value_sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34"}],[{"shape":[1,2,88,256],"sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f","value_sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f"},{"shape":[1,2,88,256],"sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24","value_sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24"},{"shape":[1,88,128],"sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748","value_sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748"},{"shape":[1,22,128],"sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a","value_sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a"}],[{"shape":[1,3,10240],"sha256":"23af29f2397aace825e958a5faa77cb734a968ef2a7a7f058f5dc543b582c847","value_sha256":"23af29f2397aace825e958a5faa77cb734a968ef2a7a7f058f5dc543b582c847"},{"shape":[1,48,128,128],"sha256":"d4c4e8b85d76cfb913593521150b144cb2bb6a6a3aed2bfe511fad2f716616ae","value_sha256":"d4c4e8b85d76cfb913593521150b144cb2bb6a6a3aed2bfe511fad2f716616ae"}],[{"shape":[1,3,10240],"sha256":"a342824984d9f603f852b6419dbd2be196dd3c2ad66b8d272ba1c723bfc58854","value_sha256":"a342824984d9f603f852b6419dbd2be196dd3c2ad66b8d272ba1c723bfc58854"},{"shape":[1,48,128,128],"sha256":"a46384ccc56e527d39df7582277baeba2229d208cc0529ecaeafd3e53b8d6db2","value_sha256":"a46384ccc56e527d39df7582277baeba2229d208cc0529ecaeafd3e53b8d6db2"}],[{"shape":[1,3,10240],"sha256":"21a78ff13a7b64234bd2db960ccfcae99cc575c3f42b185eb8831473666547e8","value_sha256":"21a78ff13a7b64234bd2db960ccfcae99cc575c3f42b185eb8831473666547e8"},{"shape":[1,48,128,128],"sha256":"69cea0bd54b8996f97bdc9f718cb4ddba75dd4c416a1f1590a1118a4503235ee","value_sha256":"69cea0bd54b8996f97bdc9f718cb4ddba75dd4c416a1f1590a1118a4503235ee"}],[{"shape":[1,2,88,256],"sha256":"75ca9e3382094e7fb1d1fb79c1b492164178b9dc028a75a084d82cba8c719b37","value_sha256":"75ca9e3382094e7fb1d1fb79c1b492164178b9dc028a75a084d82cba8c719b37"},{"shape":[1,2,88,256],"sha256":"2ceb5beed581a848481bd02f8d09b07901c37fbda93fd92d94ba1344579c65e0","value_sha256":"2ceb5beed581a848481bd02f8d09b07901c37fbda93fd92d94ba1344579c65e0"},{"shape":[1,88,128],"sha256":"bdbd34eb86fe22271437831df743bf44efb01a7dce240a9e6b6321d0c6356cde","value_sha256":"bdbd34eb86fe22271437831df743bf44efb01a7dce240a9e6b6321d0c6356cde"},{"shape":[1,22,128],"sha256":"bd0a7f30f06069bd50a87a8981cfd1657a172fe5407b78f666899f49aa1a96fc","value_sha256":"bd0a7f30f06069bd50a87a8981cfd1657a172fe5407b78f666899f49aa1a96fc"}],[{"shape":[1,3,10240],"sha256":"ef974274168dd5c569007e254179cf0ac745e7a7b28d2fbda9f06bc58c618158","value_sha256":"ef974274168dd5c569007e254179cf0ac745e7a7b28d2fbda9f06bc58c618158"},{"shape":[1,48,128,128],"sha256":"6706d6620ead67281abd40ad7239479a760992e42f0e2b8bfd2d3841c1bde52f","value_sha256":"6706d6620ead67281abd40ad7239479a760992e42f0e2b8bfd2d3841c1bde52f"}],[{"shape":[1,3,10240],"sha256":"1c6b6c06774bcfb8190cd48a437e99c9796d88d9fa6619f461f3fc90155ac15f","value_sha256":"1c6b6c06774bcfb8190cd48a437e99c9796d88d9fa6619f461f3fc90155ac15f"},{"shape":[1,48,128,128],"sha256":"78d2709f54d5687657883f8bd9216eb981d83ea4554959108c29dc0494e85f9f","value_sha256":"78d2709f54d5687657883f8bd9216eb981d83ea4554959108c29dc0494e85f9f"}],[{"shape":[1,3,10240],"sha256":"57977ed3fb0c1ed17d8b10bd2f525f28d63aad65eb434c513b490c9eb940352f","value_sha256":"57977ed3fb0c1ed17d8b10bd2f525f28d63aad65eb434c513b490c9eb940352f"},{"shape":[1,48,128,128],"sha256":"c33c4474957879b91be8e0ca721ce66ba6a6eb533028aa3cf9509a23620203ff","value_sha256":"c33c4474957879b91be8e0ca721ce66ba6a6eb533028aa3cf9509a23620203ff"}],[{"shape":[1,2,88,256],"sha256":"e19a00773b642b8e4e9309d6cb70cf123211fcd86ba6999301a2809115edb78f","value_sha256":"e19a00773b642b8e4e9309d6cb70cf123211fcd86ba6999301a2809115edb78f"},{"shape":[1,2,88,256],"sha256":"aa68365ecfd52961b728f8192765e1aa834cbe56ecddff87cc007d3b98cc5dac","value_sha256":"aa68365ecfd52961b728f8192765e1aa834cbe56ecddff87cc007d3b98cc5dac"},{"shape":[1,88,128],"sha256":"0ef422450f947767c661a851484b2c255722dee05525638e2faa342cc681e300","value_sha256":"0ef422450f947767c661a851484b2c255722dee05525638e2faa342cc681e300"},{"shape":[1,22,128],"sha256":"1ed1799c8e3d0f0656911bfdf233f591d3a2532856fa4701dc43032087ef1ab2","value_sha256":"1ed1799c8e3d0f0656911bfdf233f591d3a2532856fa4701dc43032087ef1ab2"}],[{"shape":[1,3,10240],"sha256":"33a2b31ca65d7f70d132117054cd457e87466b4939920dd29263564ea20d5c0c","value_sha256":"33a2b31ca65d7f70d132117054cd457e87466b4939920dd29263564ea20d5c0c"},{"shape":[1,48,128,128],"sha256":"8ba516367289ba29ea548682ca846fa0764bd3948ee856be8e5d19a06c86bea4","value_sha256":"8ba516367289ba29ea548682ca846fa0764bd3948ee856be8e5d19a06c86bea4"}],[{"shape":[1,3,10240],"sha256":"6e6b859184153d7b80f1e84a6e4224d521dae73c5bc9815076c037497fd9678a","value_sha256":"6e6b859184153d7b80f1e84a6e4224d521dae73c5bc9815076c037497fd9678a"},{"shape":[1,48,128,128],"sha256":"e57bcb1b6ccc4bb49d6c8c792fd854114696365377ca09e386aae9663fe62111","value_sha256":"e57bcb1b6ccc4bb49d6c8c792fd854114696365377ca09e386aae9663fe62111"}],[{"shape":[1,3,10240],"sha256":"aff052554b4f8095bac69952dc9637b28b28ccc03bf27ac26bb98da8be635769","value_sha256":"aff052554b4f8095bac69952dc9637b28b28ccc03bf27ac26bb98da8be635769"},{"shape":[1,48,128,128],"sha256":"2a2975148dfa647ca7af6ffaa3d6b82a6ee4e46f1b3d08e8d941c6073b8dc708","value_sha256":"2a2975148dfa647ca7af6ffaa3d6b82a6ee4e46f1b3d08e8d941c6073b8dc708"}],[{"shape":[1,2,88,256],"sha256":"c36614a9b99f84ef13a7550abd83a81bc1fe4b2c619418f7eb78ba5f630af56b","value_sha256":"c36614a9b99f84ef13a7550abd83a81bc1fe4b2c619418f7eb78ba5f630af56b"},{"shape":[1,2,88,256],"sha256":"c00d7aa398b8e627be8c98584509c04e2d668cbe85cff3c7032e46cb0148903c","value_sha256":"c00d7aa398b8e627be8c98584509c04e2d668cbe85cff3c7032e46cb0148903c"},{"shape":[1,88,128],"sha256":"156d95137a26b9ebb84b2ccaeea64e145d45f5557950c86da0d19e776fc1bc5f","value_sha256":"156d95137a26b9ebb84b2ccaeea64e145d45f5557950c86da0d19e776fc1bc5f"},{"shape":[1,22,128],"sha256":"945c939ef933319f802fde62d6db22728c35738821d21647ac64c3ee30ed65b9","value_sha256":"945c939ef933319f802fde62d6db22728c35738821d21647ac64c3ee30ed65b9"}],[{"shape":[1,3,10240],"sha256":"457d091ab375ff69cd61041c116a903d6a7ecac75a12b3f77d92411b05dd19cb","value_sha256":"457d091ab375ff69cd61041c116a903d6a7ecac75a12b3f77d92411b05dd19cb"},{"shape":[1,48,128,128],"sha256":"6d43c9ebfc0ad35176429ffb7379a80f0d1d4e44830655a1a66408230abe90bf","value_sha256":"6d43c9ebfc0ad35176429ffb7379a80f0d1d4e44830655a1a66408230abe90bf"}],[{"shape":[1,3,10240],"sha256":"8edba564162e6152c3cada686e749f37f4ba50a999b1c1cb93a41fc13132e191","value_sha256":"8edba564162e6152c3cada686e749f37f4ba50a999b1c1cb93a41fc13132e191"},{"shape":[1,48,128,128],"sha256":"cc1e836d7be25016bdd272aea2674735026363b02a9e49b088ca54c9e419ecaf","value_sha256":"cc1e836d7be25016bdd272aea2674735026363b02a9e49b088ca54c9e419ecaf"}],[{"shape":[1,3,10240],"sha256":"8ca7077aef12bb5981f3dcbf42f45d8d961ad65248f74bca91f7c271a3c73b9c","value_sha256":"8ca7077aef12bb5981f3dcbf42f45d8d961ad65248f74bca91f7c271a3c73b9c"},{"shape":[1,48,128,128],"sha256":"0efcb17ce919b684b3779274c474c8ffc89e36ccd5a90aa4763f1fa311865163","value_sha256":"0efcb17ce919b684b3779274c474c8ffc89e36ccd5a90aa4763f1fa311865163"}],[{"shape":[1,2,88,256],"sha256":"4eb822ebd43a6b56547196de8c04b93f3b5b20c3b063ba82b05b6b2fc00a1e08","value_sha256":"4eb822ebd43a6b56547196de8c04b93f3b5b20c3b063ba82b05b6b2fc00a1e08"},{"shape":[1,2,88,256],"sha256":"6f6037b65cf72e99ae95df7d662906b7236d6648595e6c6021d588b06c901d50","value_sha256":"6f6037b65cf72e99ae95df7d662906b7236d6648595e6c6021d588b06c901d50"},{"shape":[1,88,128],"sha256":"b13b714ace3ec8a73168fff81166f267a30bb3b3dca199ebd6d9f297e22d3cbb","value_sha256":"b13b714ace3ec8a73168fff81166f267a30bb3b3dca199ebd6d9f297e22d3cbb"},{"shape":[1,22,128],"sha256":"06e5ff478478ee1a710fb892424172717fd0881fd9e39d9757635ebfef6a17af","value_sha256":"06e5ff478478ee1a710fb892424172717fd0881fd9e39d9757635ebfef6a17af"}],[{"shape":[1,3,10240],"sha256":"99ea167ef3b9cdb4b9c105348e814fd3d1a0e2022c0aeab181f7f38340f3b432","value_sha256":"99ea167ef3b9cdb4b9c105348e814fd3d1a0e2022c0aeab181f7f38340f3b432"},{"shape":[1,48,128,128],"sha256":"11ef876b8c15cc91454277e92cdabfbfe91c3a35d1c1a2e8aeec0cc224113b9e","value_sha256":"11ef876b8c15cc91454277e92cdabfbfe91c3a35d1c1a2e8aeec0cc224113b9e"}],[{"shape":[1,3,10240],"sha256":"baf0923981974ff892e115a69049618f8e325b8beff769bec83f6c803a59f623","value_sha256":"baf0923981974ff892e115a69049618f8e325b8beff769bec83f6c803a59f623"},{"shape":[1,48,128,128],"sha256":"cd9b63ead383d25a184cab17d944e3a9b91923a221322a7c97a36be563a526cd","value_sha256":"cd9b63ead383d25a184cab17d944e3a9b91923a221322a7c97a36be563a526cd"}],[{"shape":[1,3,10240],"sha256":"a454bb59c8aac167b6826584fa0ae561da1ac2e3819f1aa046a516876fc6c506","value_sha256":"a454bb59c8aac167b6826584fa0ae561da1ac2e3819f1aa046a516876fc6c506"},{"shape":[1,48,128,128],"sha256":"b8d2047192b894f097eeeb0e943d8e732fba9481cc20c4f24779c4c8a5d94091","value_sha256":"b8d2047192b894f097eeeb0e943d8e732fba9481cc20c4f24779c4c8a5d94091"}],[{"shape":[1,2,88,256],"sha256":"fcb08c9786c93de86d3c6404cb9e674521cfa7ac41199df8d3da886d32d76754","value_sha256":"fcb08c9786c93de86d3c6404cb9e674521cfa7ac41199df8d3da886d32d76754"},{"shape":[1,2,88,256],"sha256":"2869ad71bb6f0aec650fa614482789563782f48a3280030fc9b7138dc48e2ee3","value_sha256":"2869ad71bb6f0aec650fa614482789563782f48a3280030fc9b7138dc48e2ee3"},{"shape":[1,88,128],"sha256":"a301869179deb4c07e4c125b29957dc6f635c08e4318f223d78b7a5afeaa38a8","value_sha256":"a301869179deb4c07e4c125b29957dc6f635c08e4318f223d78b7a5afeaa38a8"},{"shape":[1,22,128],"sha256":"bf77e99e57d4c548e7c6be3375f17821a116837cf9bd09cebc5dc7ca00e9e26c","value_sha256":"bf77e99e57d4c548e7c6be3375f17821a116837cf9bd09cebc5dc7ca00e9e26c"}],[{"shape":[1,3,10240],"sha256":"5a141665706b652733a7458367775637fb0e5593f988e5ed5932a61bbdee5528","value_sha256":"5a141665706b652733a7458367775637fb0e5593f988e5ed5932a61bbdee5528"},{"shape":[1,48,128,128],"sha256":"e1bb2841a6b57c9720ea50396036633559a4e9501bc3ddf5a67836f12ba5b62c","value_sha256":"e1bb2841a6b57c9720ea50396036633559a4e9501bc3ddf5a67836f12ba5b62c"}],[{"shape":[1,3,10240],"sha256":"67c1f4bd6bb14f381283c71cbdd55966af1784202fb75c18c4ec8536fe1f0e65","value_sha256":"67c1f4bd6bb14f381283c71cbdd55966af1784202fb75c18c4ec8536fe1f0e65"},{"shape":[1,48,128,128],"sha256":"847a97f31bccf45c8de961089477aeb2d7af44e2f9029a5785afc6ca5a9027d5","value_sha256":"847a97f31bccf45c8de961089477aeb2d7af44e2f9029a5785afc6ca5a9027d5"}],[{"shape":[1,3,10240],"sha256":"d7d0157be88198115d2ea8936388ee7c9ab2730e10654a046c2d7fb971981bc2","value_sha256":"d7d0157be88198115d2ea8936388ee7c9ab2730e10654a046c2d7fb971981bc2"},{"shape":[1,48,128,128],"sha256":"3f4812e18648465cfc0e10525bd16bc32a32ad74803986f9e1ce6076b8786be2","value_sha256":"3f4812e18648465cfc0e10525bd16bc32a32ad74803986f9e1ce6076b8786be2"}],[{"shape":[1,2,88,256],"sha256":"fb8b1ccb372fda0602f0c5838d51ab46ce2cb3aecf5673413e206af914b9ee85","value_sha256":"fb8b1ccb372fda0602f0c5838d51ab46ce2cb3aecf5673413e206af914b9ee85"},{"shape":[1,2,88,256],"sha256":"a6f93d1c84f1d71753d1a7ba4cd4590289e52aa9981711853ce636074555184d","value_sha256":"a6f93d1c84f1d71753d1a7ba4cd4590289e52aa9981711853ce636074555184d"},{"shape":[1,88,128],"sha256":"7f4806907e909a6b9a5cb17ca721c07fe77ac3aa365a6940d86d86d5a5bc8a52","value_sha256":"7f4806907e909a6b9a5cb17ca721c07fe77ac3aa365a6940d86d86d5a5bc8a52"},{"shape":[1,22,128],"sha256":"a260f656750162410ea2dfec7b5ae0ee403ae29da7a57d905fd93a823a345f9e","value_sha256":"a260f656750162410ea2dfec7b5ae0ee403ae29da7a57d905fd93a823a345f9e"}],[{"shape":[1,3,10240],"sha256":"ffd995c891c8983d6cf2bba9895627884913d0532e8ee561bc69987a2e6644cc","value_sha256":"ffd995c891c8983d6cf2bba9895627884913d0532e8ee561bc69987a2e6644cc"},{"shape":[1,48,128,128],"sha256":"ebb1e44092a7f4843b194d50054c18229128540d8869b528666b2c0826af72ff","value_sha256":"ebb1e44092a7f4843b194d50054c18229128540d8869b528666b2c0826af72ff"}],[{"shape":[1,3,10240],"sha256":"268cf105ff4cc226cf4c40a46eef26650fdd4dea1e398c81f4fd456813092813","value_sha256":"268cf105ff4cc226cf4c40a46eef26650fdd4dea1e398c81f4fd456813092813"},{"shape":[1,48,128,128],"sha256":"01b30ea217235762534adb10875be7811dc4a031d841142a8e71edf7a02f6202","value_sha256":"01b30ea217235762534adb10875be7811dc4a031d841142a8e71edf7a02f6202"}],[{"shape":[1,3,10240],"sha256":"965ed9a6a79f88c2c7a610e4ff49a3face0cbb1c13814135cf206e1c0f96e326","value_sha256":"965ed9a6a79f88c2c7a610e4ff49a3face0cbb1c13814135cf206e1c0f96e326"},{"shape":[1,48,128,128],"sha256":"01c25d008bbd248eac36e70f0dbbb72597105b56b58fafb8cc348011ad3aeb12","value_sha256":"01c25d008bbd248eac36e70f0dbbb72597105b56b58fafb8cc348011ad3aeb12"}],[{"shape":[1,2,88,256],"sha256":"88cda706be1e27ef1251b1fedf30cfa201a305502a09134e3282736e45c9f8aa","value_sha256":"88cda706be1e27ef1251b1fedf30cfa201a305502a09134e3282736e45c9f8aa"},{"shape":[1,2,88,256],"sha256":"78bb9fea678d59bf723d846866bc46b062af3188b3a5dfb5dae0a996e8e4f719","value_sha256":"78bb9fea678d59bf723d846866bc46b062af3188b3a5dfb5dae0a996e8e4f719"},{"shape":[1,88,128],"sha256":"39430166fbe23327a5d078ba372eb18a3fadc82253886d7ad186bdf95c722228","value_sha256":"39430166fbe23327a5d078ba372eb18a3fadc82253886d7ad186bdf95c722228"},{"shape":[1,22,128],"sha256":"28089aa680484b36139d5afee6b15c0b0acfd3ae528f36d9b6e6b18e65db5bb6","value_sha256":"28089aa680484b36139d5afee6b15c0b0acfd3ae528f36d9b6e6b18e65db5bb6"}],[{"shape":[1,3,10240],"sha256":"db1c3cbab49e324e65452b31ba9656bc95ca70dd2bf32f1af1dde7a002d59d29","value_sha256":"db1c3cbab49e324e65452b31ba9656bc95ca70dd2bf32f1af1dde7a002d59d29"},{"shape":[1,48,128,128],"sha256":"f809b0171e94c5cec2ca972b733d6c21ef487195867669c9acc3700696331f3e","value_sha256":"f809b0171e94c5cec2ca972b733d6c21ef487195867669c9acc3700696331f3e"}],[{"shape":[1,3,10240],"sha256":"61dcb190d48b9b056f3d9e6b3f6da8ca1327615b339ccd7900d56dac40c3528f","value_sha256":"61dcb190d48b9b056f3d9e6b3f6da8ca1327615b339ccd7900d56dac40c3528f"},{"shape":[1,48,128,128],"sha256":"33b6e4f3dbb38a75049c7e0c8748ddfa229b7a97c8d5ea8bfffa1810b3ff1608","value_sha256":"33b6e4f3dbb38a75049c7e0c8748ddfa229b7a97c8d5ea8bfffa1810b3ff1608"}],[{"shape":[1,3,10240],"sha256":"8fb2f7b4fcb9b795caa3fe87ba09210b68f95cd1f8e4ee598fa2d8ef0a1cddef","value_sha256":"8fb2f7b4fcb9b795caa3fe87ba09210b68f95cd1f8e4ee598fa2d8ef0a1cddef"},{"shape":[1,48,128,128],"sha256":"cfb8c0a67de5e3c2a08c49ab89a95cea396c4b5530dece4bb6dc1e292f704281","value_sha256":"cfb8c0a67de5e3c2a08c49ab89a95cea396c4b5530dece4bb6dc1e292f704281"}],[{"shape":[1,2,88,256],"sha256":"5a0da5529eeb692ff189e85986ed3bbafd96a99444f8d975b1349975c52d6946","value_sha256":"23c735b3da2aa189a08c35de556f3d638ca42fb360f62c85ee2018f85c2fffa7"},{"shape":[1,2,88,256],"sha256":"4aa114f8361c5bde1fc959817ce2b5d891edf02eb7a1ee1000c97e6faca78b95","value_sha256":"4aa114f8361c5bde1fc959817ce2b5d891edf02eb7a1ee1000c97e6faca78b95"},{"shape":[1,88,128],"sha256":"aa44ae0f9c64d7b81a5a2d03b58de1f19bd11a948aa61a6c193a4b6e5469970b","value_sha256":"aa44ae0f9c64d7b81a5a2d03b58de1f19bd11a948aa61a6c193a4b6e5469970b"},{"shape":[1,22,128],"sha256":"a0995be864606aa29529e0d797f35c31d0d3de9ee623d76fa45f5ce4b87dd4f4","value_sha256":"a0995be864606aa29529e0d797f35c31d0d3de9ee623d76fa45f5ce4b87dd4f4"}],[{"shape":[1,3,10240],"sha256":"e4ff3d6eba578ef7c52c05e27a1efd0d1a35ab643b64c82c7044c90327e2d010","value_sha256":"e4ff3d6eba578ef7c52c05e27a1efd0d1a35ab643b64c82c7044c90327e2d010"},{"shape":[1,48,128,128],"sha256":"7de0be2d7c867a38a323272081641806101cc5d14c8f37a65dca7b86979b8af0","value_sha256":"7de0be2d7c867a38a323272081641806101cc5d14c8f37a65dca7b86979b8af0"}],[{"shape":[1,3,10240],"sha256":"b7e6512706b5582f5f1a2d3409b926552694eb986a7344ec2ab2011fe3ea0e6e","value_sha256":"b7e6512706b5582f5f1a2d3409b926552694eb986a7344ec2ab2011fe3ea0e6e"},{"shape":[1,48,128,128],"sha256":"d46cf1f72475f437db15506d1c159ca2ea7580ef240fca13817995104ac6938b","value_sha256":"d46cf1f72475f437db15506d1c159ca2ea7580ef240fca13817995104ac6938b"}],[{"shape":[1,3,10240],"sha256":"df17d1305bbcaf474ae22908d0897d871424db73ab4de5e72ab5a550e7666257","value_sha256":"df17d1305bbcaf474ae22908d0897d871424db73ab4de5e72ab5a550e7666257"},{"shape":[1,48,128,128],"sha256":"0fd02e86361404ca4fb55f6d756f14787e0f37ad03e9648ee4d5130c5697dc55","value_sha256":"0fd02e86361404ca4fb55f6d756f14787e0f37ad03e9648ee4d5130c5697dc55"}],[{"shape":[1,2,88,256],"sha256":"8966f22a4de9fd3ed9035e602c6b5730d6916daf1d1041411e6945afa4858440","value_sha256":"8966f22a4de9fd3ed9035e602c6b5730d6916daf1d1041411e6945afa4858440"},{"shape":[1,2,88,256],"sha256":"8ea24411ee21fd6c5e50c6ce88e25ed9a7f4433852114d538c402d54531bd7a2","value_sha256":"8ea24411ee21fd6c5e50c6ce88e25ed9a7f4433852114d538c402d54531bd7a2"},{"shape":[1,88,128],"sha256":"7d2134a084d27f8072754a9fec03facd14ffae3d38f45f61ad1ac2a24d7b8175","value_sha256":"7d2134a084d27f8072754a9fec03facd14ffae3d38f45f61ad1ac2a24d7b8175"},{"shape":[1,22,128],"sha256":"25e429a491b003408e95940f5f037dd9abde7bd38c748d06fe9e7d7a0cd14f8c","value_sha256":"25e429a491b003408e95940f5f037dd9abde7bd38c748d06fe9e7d7a0cd14f8c"}],[{"shape":[1,3,10240],"sha256":"fe0011e0b328f04c60869ace5e1e16df6739793dcd51a1309df42dc5f639cf86","value_sha256":"fe0011e0b328f04c60869ace5e1e16df6739793dcd51a1309df42dc5f639cf86"},{"shape":[1,48,128,128],"sha256":"03108befe82b6db08114de55f1efbb9cf0b00ffd3c2701e0cfc334d7e8b45d2f","value_sha256":"03108befe82b6db08114de55f1efbb9cf0b00ffd3c2701e0cfc334d7e8b45d2f"}],[{"shape":[1,3,10240],"sha256":"70d9e8f0766035db9a6df27d69b345fa60db120d631ec7aa5817d754be7be781","value_sha256":"70d9e8f0766035db9a6df27d69b345fa60db120d631ec7aa5817d754be7be781"},{"shape":[1,48,128,128],"sha256":"3ae75e227d68f85d37cf6de4ef43f6cd7d610b7b1f298d2ab41e65166892ef69","value_sha256":"3ae75e227d68f85d37cf6de4ef43f6cd7d610b7b1f298d2ab41e65166892ef69"}],[{"shape":[1,3,10240],"sha256":"4281b65d24625eae42daed8caff45e34d07270ca8722bca5aac660202256c32f","value_sha256":"4281b65d24625eae42daed8caff45e34d07270ca8722bca5aac660202256c32f"},{"shape":[1,48,128,128],"sha256":"2b26c641c9eacee9c66760be14478123068d4ae74a59b235b62c3c6d5565bbe0","value_sha256":"2b26c641c9eacee9c66760be14478123068d4ae74a59b235b62c3c6d5565bbe0"}],[{"shape":[1,2,88,256],"sha256":"0156ed0676481c1e9534a55b2779c80b26b1b4262122afd276991672f4d13dba","value_sha256":"0156ed0676481c1e9534a55b2779c80b26b1b4262122afd276991672f4d13dba"},{"shape":[1,2,88,256],"sha256":"eeee6171061647ed1677ebe59bb49b2ca15e0dffee5d936bcfe97a77a05207d0","value_sha256":"eeee6171061647ed1677ebe59bb49b2ca15e0dffee5d936bcfe97a77a05207d0"},{"shape":[1,88,128],"sha256":"a9fa7284631037665443e03494b1bb4fe32f25791f308903fe950667b76907f4","value_sha256":"a9fa7284631037665443e03494b1bb4fe32f25791f308903fe950667b76907f4"},{"shape":[1,22,128],"sha256":"dbc72d1d83f28442a5a80c547c187eacc1a7f2c7f494dff16fe15074c4e5a9be","value_sha256":"dbc72d1d83f28442a5a80c547c187eacc1a7f2c7f494dff16fe15074c4e5a9be"}],[{"shape":[1,3,10240],"sha256":"7093d030a67627399bfb064699482f914d23366eefee4265b235d8d85a230978","value_sha256":"7093d030a67627399bfb064699482f914d23366eefee4265b235d8d85a230978"},{"shape":[1,48,128,128],"sha256":"bebb33ca1fae20f6cd9b83af7272740020323f1bbe3fe62c4978c6e2448cb660","value_sha256":"bebb33ca1fae20f6cd9b83af7272740020323f1bbe3fe62c4978c6e2448cb660"}],[{"shape":[1,3,10240],"sha256":"c03f0185a17ccf082894797d1716f5cd14f93f960607e5ddf94a40ccc1105038","value_sha256":"c03f0185a17ccf082894797d1716f5cd14f93f960607e5ddf94a40ccc1105038"},{"shape":[1,48,128,128],"sha256":"d40a25e169416b62c7c24693846f6d0e49ae32191a0b25b27d556de72d0ba41b","value_sha256":"d40a25e169416b62c7c24693846f6d0e49ae32191a0b25b27d556de72d0ba41b"}],[{"shape":[1,3,10240],"sha256":"f503b18b3b47f5e21499c5e9510d39b86e89a05631ac0b28beaf1548c895a27e","value_sha256":"f503b18b3b47f5e21499c5e9510d39b86e89a05631ac0b28beaf1548c895a27e"},{"shape":[1,48,128,128],"sha256":"e37360995b834ed990fb3a63b83684efc045b2035dd7de06f9d75ab9eda91914","value_sha256":"e37360995b834ed990fb3a63b83684efc045b2035dd7de06f9d75ab9eda91914"}],[{"shape":[1,2,88,256],"sha256":"716c40276b1c9e46ba8f7e76fd920ed25f506706aff59be263123ab1cfe61aae","value_sha256":"716c40276b1c9e46ba8f7e76fd920ed25f506706aff59be263123ab1cfe61aae"},{"shape":[1,2,88,256],"sha256":"6242a9c8793306892addc2ae9762c417cdfb350ccbc93aa4c90cb179cbf930f8","value_sha256":"6242a9c8793306892addc2ae9762c417cdfb350ccbc93aa4c90cb179cbf930f8"},{"shape":[1,88,128],"sha256":"83443016576a524ac3bc23bb0e7625f2659c0ff35d55d9cc34f6c541027d085b","value_sha256":"83443016576a524ac3bc23bb0e7625f2659c0ff35d55d9cc34f6c541027d085b"},{"shape":[1,22,128],"sha256":"53bd6c3f3da8bc5c19e1d6160ca9dec71925b79be161ded2088f7a7e75b4de34","value_sha256":"53bd6c3f3da8bc5c19e1d6160ca9dec71925b79be161ded2088f7a7e75b4de34"}]],"mtp":[[{"shape":[1,2,87,256],"sha256":"e9aade6cf8cedfff8b01a5836c4bb5164ee3dc4c276b24e1ded346bdf4d8d959","value_sha256":"e9aade6cf8cedfff8b01a5836c4bb5164ee3dc4c276b24e1ded346bdf4d8d959"},{"shape":[1,2,87,256],"sha256":"77e78cf6c7b1f08ec2876307e5166221fec26152df863daf44ccaffdb21fa079","value_sha256":"77e78cf6c7b1f08ec2876307e5166221fec26152df863daf44ccaffdb21fa079"},{"shape":[1,87,128],"sha256":"1f1b3729be3d37dd1d1ae5b3cb28e228f49918da266f73af3bb2ae0b1ab00cff","value_sha256":"1f1b3729be3d37dd1d1ae5b3cb28e228f49918da266f73af3bb2ae0b1ab00cff"},{"shape":[1,21,128],"sha256":"3fce4dd0cf1e1f69b65ef85f94813066b85ce04efa10903f394c90133ee9055e","value_sha256":"3fce4dd0cf1e1f69b65ef85f94813066b85ce04efa10903f394c90133ee9055e"}]]}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/warm.json b/docs/issue113-head-chunk/product/warm.json
new file mode 100644
index 0000000..02d64ad
--- /dev/null
+++ b/docs/issue113-head-chunk/product/warm.json
@@ -0,0 +1 @@
+{"case":"warm","prompt":[248045,8678,198,24342,286,4879,369,716,310,3238,13,13262,678,7047,9522,321,10419,11,6992,5774,310,279,16198,1973,24366,24150,362,13,248046,198,248045,846,198,2427,494,799,310,5600,13,248046,198,248045,74455,198,248068,198,760,1156,6587,728,310,1698,494,799,310,5600,13,1061,198,248069,271,248046,198,248045,846,198,22791,24217,13,248046,198,248045,74455,198,248068,198],"tokens":[760,1156,6587,728,310,2962,24217,13,353,8334,29014,494],"cached":58,"state_sha256":"257714fbbf6f6a92b5562f554fae93c36abaff2e72bac7ad61a0d874e14e8234"}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/warm/0.json b/docs/issue113-head-chunk/product/warm/0.json
new file mode 100644
index 0000000..71c57f2
--- /dev/null
+++ b/docs/issue113-head-chunk/product/warm/0.json
@@ -0,0 +1 @@
+{"index":0,"ids":[760,1156,6587,728],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"0d002e5ab00ee76250e5deca14358353c777fd1f1804937070ca229f74cdc9f6"},"hidden":{"shape":[1,4,10240],"sha256":"cb6bc68f9d37d0ffb544b2c635144d8f91a715c5172fa31a32be7aab84d5941b"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/warm/1.json b/docs/issue113-head-chunk/product/warm/1.json
new file mode 100644
index 0000000..87ebb2e
--- /dev/null
+++ b/docs/issue113-head-chunk/product/warm/1.json
@@ -0,0 +1 @@
+{"index":1,"ids":[310,2962,24217,494],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"e052a03d4908e670c534c18453447dcb7190bf180dd281ffd3ace950675174e1"},"hidden":{"shape":[1,4,10240],"sha256":"2caa9c5b1d863080a4ec81e55103e9b3429261933f13cac971dfc09a8a1c130f"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/warm/2.json b/docs/issue113-head-chunk/product/warm/2.json
new file mode 100644
index 0000000..5fbd30f
--- /dev/null
+++ b/docs/issue113-head-chunk/product/warm/2.json
@@ -0,0 +1 @@
+{"index":2,"ids":[13,353,2582,3804],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"f64c3236f08fa43f85ee491cc1121d5f584dfd1fbcd58b1bf9ef69b441813036"},"hidden":{"shape":[1,4,10240],"sha256":"eba14099db5411740f69183e22dbe7d50aef3e5cfde093319eaa39008fbd11c8"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/product/warm/3.json b/docs/issue113-head-chunk/product/warm/3.json
new file mode 100644
index 0000000..833f0a3
--- /dev/null
+++ b/docs/issue113-head-chunk/product/warm/3.json
@@ -0,0 +1 @@
+{"index":3,"ids":[8334,29014,494],"verify":true,"logits":{"shape":[1,3,248320],"sha256":"6821599090afcd8e042f955477276a52843d0087c6074a1a77a43b9768cd2c0c"},"hidden":{"shape":[1,3,10240],"sha256":"e358aa99aa0738c30f102ad646ebef5244b7740ca7ccaa3804469e3dc8ac217c"}}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/provenance.json b/docs/issue113-head-chunk/provenance.json
new file mode 100644
index 0000000..c3efbb6
--- /dev/null
+++ b/docs/issue113-head-chunk/provenance.json
@@ -0,0 +1,114 @@
+{
+  "baseline": "7b894fbfeed81d2944b9cf479e60638ef04987c7",
+  "oracle": "9882703f3105363ddc37eca9f97aa09a1d387112",
+  "mlx": "0.32.2",
+  "old_checkpoint": {
+    "sha256": "0e090c9a590b1c7ac1de088db4a4a4637205267e4ff170dec53bd82f0153895c",
+    "identity": {
+      "model": [
+        241,
+        186,
+        133,
+        5,
+        200,
+        173,
+        242,
+        119,
+        171,
+        239,
+        119,
+        94,
+        213,
+        34,
+        200,
+        231,
+        175,
+        107,
+        18,
+        34,
+        60,
+        60,
+        31,
+        178,
+        144,
+        116,
+        167,
+        229,
+        84,
+        1,
+        48,
+        163
+      ],
+      "context": 8192,
+      "vocabulary": 248320,
+      "mtp": true
+    },
+    "tokens": [
+      248045,
+      8678,
+      198,
+      24342,
+      286,
+      4879,
+      369,
+      716,
+      310,
+      3238,
+      13,
+      13262,
+      678,
+      7047,
+      9522,
+      321,
+      10419,
+      11,
+      6992,
+      5774,
+      310,
+      279,
+      16198,
+      1973,
+      24366,
+      24150,
+      362,
+      13,
+      248046,
+      198,
+      248045,
+      846,
+      198,
+      2427,
+      494,
+      799,
+      310,
+      5600,
+      13,
+      248046,
+      198,
+      248045,
+      74455,
+      198,
+      248068,
+      198,
+      760,
+      1156,
+      6587,
+      728,
+      310,
+      1698,
+      494,
+      799,
+      310,
+      5600,
+      13,
+      1061
+    ],
+    "verify_gate": true
+  },
+  "sha256": {
+    "tests/fixtures/mtplx-head-chunk.json": "57ff569dafb00b9defe1f8e7d184b90943dca1eaac21677984d8f22434c01fa2",
+    "/private/tmp/ds4-issue113-final-test": "f15440399c642860ea4ad1fb0e7811f2ed799b0410f823606d21812003051436",
+    "/private/tmp/ds4-issue113-product/new.bin": "f0e249f2792c4242ba630eeacca6e4443045f27bd7d4dc7d575d9a480c86ce2a",
+    "/private/tmp/ds4-issue113-off-product/new.bin": "907471e846658d6c13a744dc89e7332d79ec3dc7a4de55ca0c22333495b12b58"
+  }
+}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/qsa-module-0.log b/docs/issue113-head-chunk/qsa-module-0.log
new file mode 100644
index 0000000..b14a255
--- /dev/null
+++ b/docs/issue113-head-chunk/qsa-module-0.log
@@ -0,0 +1,9 @@
+
+running 1 test
+test engine::metal::qwen_mtplx::qsa_attention::mtplx_qsa_attention_module_matches_reference ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
+ds4: Metal 4 tensor API enabled for Tensor kernels
+ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
+ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 0.94s
diff --git a/docs/issue113-head-chunk/qsa-module-1.log b/docs/issue113-head-chunk/qsa-module-1.log
new file mode 100644
index 0000000..4d5e1b3
--- /dev/null
+++ b/docs/issue113-head-chunk/qsa-module-1.log
@@ -0,0 +1,9 @@
+
+running 1 test
+test engine::metal::qwen_mtplx::qsa_attention::mtplx_qsa_attention_module_matches_reference ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
+ds4: Metal 4 tensor API enabled for Tensor kernels
+ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
+ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 548 filtered out; finished in 0.93s
diff --git a/docs/issue113-head-chunk/receipts.json b/docs/issue113-head-chunk/receipts.json
new file mode 100644
index 0000000..0faf967
--- /dev/null
+++ b/docs/issue113-head-chunk/receipts.json
@@ -0,0 +1,3181 @@
+{
+  "fixtures": [
+    {
+      "case": "head_chunk",
+      "index": 0,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 1,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 2,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 3,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 4,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 5,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 6,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 7,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 10,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 11,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 12,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 13,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 14,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 15,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 16,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 17,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 18,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 19,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 22,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 23,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 24,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 25,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 26,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 27,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 28,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 29,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 30,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 34,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 35,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 36,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 37,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 38,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 39,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 40,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 41,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 42,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 46,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 47,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 48,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 49,
+      "exact": true,
+      "fused": 6,
+      "subcalls": 6
+    },
+    {
+      "case": "head_chunk",
+      "index": 50,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 51,
+      "exact": true,
+      "fused": 6,
+      "subcalls": 6
+    },
+    {
+      "case": "head_chunk",
+      "index": 52,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 53,
+      "exact": true,
+      "fused": 6,
+      "subcalls": 6
+    },
+    {
+      "case": "head_chunk",
+      "index": 54,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 57,
+      "exact": true,
+      "fused": 6,
+      "subcalls": 6
+    },
+    {
+      "case": "head_chunk",
+      "index": 58,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 59,
+      "exact": true,
+      "fused": 6,
+      "subcalls": 6
+    },
+    {
+      "case": "head_chunk",
+      "index": 60,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 61,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 8
+    },
+    {
+      "case": "head_chunk",
+      "index": 62,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 63,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 8
+    },
+    {
+      "case": "head_chunk",
+      "index": 64,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 65,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 8
+    },
+    {
+      "case": "head_chunk",
+      "index": 66,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 70,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 71,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 8
+    },
+    {
+      "case": "head_chunk",
+      "index": 72,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 73,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 12
+    },
+    {
+      "case": "head_chunk",
+      "index": 74,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 75,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 12
+    },
+    {
+      "case": "head_chunk",
+      "index": 76,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 77,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 12
+    },
+    {
+      "case": "head_chunk",
+      "index": 78,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 82,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 83,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 12
+    },
+    {
+      "case": "head_chunk",
+      "index": 84,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 85,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 24
+    },
+    {
+      "case": "head_chunk",
+      "index": 86,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 87,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 24
+    },
+    {
+      "case": "head_chunk",
+      "index": 88,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 89,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 24
+    },
+    {
+      "case": "head_chunk",
+      "index": 90,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 94,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 95,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 24
+    },
+    {
+      "case": "head_chunk",
+      "index": 96,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 97,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 98,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 99,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 100,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 101,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 102,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 103,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 106,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 107,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 108,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 109,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 110,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 111,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 112,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 113,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 114,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 118,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 119,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 120,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 121,
+      "exact": true,
+      "fused": 2,
+      "subcalls": 2
+    },
+    {
+      "case": "head_chunk",
+      "index": 122,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 123,
+      "exact": true,
+      "fused": 2,
+      "subcalls": 2
+    },
+    {
+      "case": "head_chunk",
+      "index": 124,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 125,
+      "exact": true,
+      "fused": 2,
+      "subcalls": 2
+    },
+    {
+      "case": "head_chunk",
+      "index": 126,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 130,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 131,
+      "exact": true,
+      "fused": 2,
+      "subcalls": 2
+    },
+    {
+      "case": "head_chunk",
+      "index": 132,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 133,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 134,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 135,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 136,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 137,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 138,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 142,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 143,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 144,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 145,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 146,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 147,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 148,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 149,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 150,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 151,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 154,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 155,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 156,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 157,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 158,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 159,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 160,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 161,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 162,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 163,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 166,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 167,
+      "exact": true,
+      "fused": 1,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 168,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 169,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 170,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 171,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 172,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 173,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 174,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 178,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 179,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 180,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 181,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 182,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 183,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 184,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 185,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 186,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 190,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 191,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 192,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 193,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 194,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 195,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 196,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 197,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    },
+    {
+      "case": "head_chunk",
+      "index": 198,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 202,
+      "exact": true,
+      "fused": 0,
+      "subcalls": 1
+    },
+    {
+      "case": "head_chunk",
+      "index": 203,
+      "exact": true,
+      "fused": 4,
+      "subcalls": 4
+    }
+  ],
+  "product": [
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 0,
+        "ids": [
+          760,
+          1156,
+          369,
+          9859
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "e3ba038c8570911a72fadeba867f1c33e5103108ef9881bfcdc008907de90839"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "b3c2ef7bb9c3dfa85748d7af883210ae4393bd1b5af5397f243d632fc437552b"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 1,
+        "ids": [
+          6587,
+          728,
+          310,
+          1698
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "955ff038997bab8ca6a84642b1ad8cf88dade0d2075d5ea11c736c9f74a7d563"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "e3a029cc190a3cbdee1e2ec34bd94181f6013a6495c56b60dcccd67130fcfbd4"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 2,
+        "ids": [
+          494,
+          799,
+          310,
+          5600
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "e201b5e579d7c8f58432ae495c120c66cd595241046dd9cd32d42cab72009b3b"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "d56aba5df3172aeca9d26bccfe6143d8045d30e5af45f1fc6815d61634f78d2c"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 3,
+        "ids": [
+          13,
+          1061
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            2,
+            248320
+          ],
+          "sha256": "ecf42a91034a5bd6739a3d5bc94ffdcbac8214261f5712a3196bb6b99be67860"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            2,
+            10240
+          ],
+          "sha256": "0296bbc47e09e0312470a1ada7ef90177009d80cef5ba79398190177bb4be7c3"
+        }
+      }
+    },
+    {
+      "case": "cold",
+      "prompt": [
+        248045,
+        8678,
+        198,
+        24342,
+        286,
+        4879,
+        369,
+        716,
+        310,
+        3238,
+        13,
+        13262,
+        678,
+        7047,
+        9522,
+        321,
+        10419,
+        11,
+        6992,
+        5774,
+        310,
+        279,
+        16198,
+        1973,
+        24366,
+        24150,
+        362,
+        13,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        2427,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198
+      ],
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        1698,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        1061
+      ],
+      "cached": 0,
+      "state_sha256": "ed07c44dec46cebe4471ac26a136c9d8ece3d88b4abeaa7f688247a13f3b1276"
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 0,
+        "ids": [
+          760,
+          1156,
+          6587,
+          728
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "0d002e5ab00ee76250e5deca14358353c777fd1f1804937070ca229f74cdc9f6"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "cb6bc68f9d37d0ffb544b2c635144d8f91a715c5172fa31a32be7aab84d5941b"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 1,
+        "ids": [
+          310,
+          2962,
+          24217,
+          494
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "e052a03d4908e670c534c18453447dcb7190bf180dd281ffd3ace950675174e1"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "2caa9c5b1d863080a4ec81e55103e9b3429261933f13cac971dfc09a8a1c130f"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 2,
+        "ids": [
+          13,
+          353,
+          2582,
+          3804
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "f64c3236f08fa43f85ee491cc1121d5f584dfd1fbcd58b1bf9ef69b441813036"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "eba14099db5411740f69183e22dbe7d50aef3e5cfde093319eaa39008fbd11c8"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 3,
+        "ids": [
+          8334,
+          29014,
+          494
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            3,
+            248320
+          ],
+          "sha256": "6821599090afcd8e042f955477276a52843d0087c6074a1a77a43b9768cd2c0c"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            3,
+            10240
+          ],
+          "sha256": "e358aa99aa0738c30f102ad646ebef5244b7740ca7ccaa3804469e3dc8ac217c"
+        }
+      }
+    },
+    {
+      "case": "warm",
+      "prompt": [
+        248045,
+        8678,
+        198,
+        24342,
+        286,
+        4879,
+        369,
+        716,
+        310,
+        3238,
+        13,
+        13262,
+        678,
+        7047,
+        9522,
+        321,
+        10419,
+        11,
+        6992,
+        5774,
+        310,
+        279,
+        16198,
+        1973,
+        24366,
+        24150,
+        362,
+        13,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        2427,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198,
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        1698,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        1061,
+        198,
+        248069,
+        271,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        22791,
+        24217,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198
+      ],
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        2962,
+        24217,
+        13,
+        353,
+        8334,
+        29014,
+        494
+      ],
+      "cached": 58,
+      "state_sha256": "257714fbbf6f6a92b5562f554fae93c36abaff2e72bac7ad61a0d874e14e8234"
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 0,
+        "ids": [
+          760,
+          1156,
+          6587,
+          728
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "0d002e5ab00ee76250e5deca14358353c777fd1f1804937070ca229f74cdc9f6"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "cb6bc68f9d37d0ffb544b2c635144d8f91a715c5172fa31a32be7aab84d5941b"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 1,
+        "ids": [
+          310,
+          2962,
+          24217,
+          494
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "e052a03d4908e670c534c18453447dcb7190bf180dd281ffd3ace950675174e1"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "2caa9c5b1d863080a4ec81e55103e9b3429261933f13cac971dfc09a8a1c130f"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 2,
+        "ids": [
+          13,
+          353,
+          2582,
+          3804
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "f64c3236f08fa43f85ee491cc1121d5f584dfd1fbcd58b1bf9ef69b441813036"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "eba14099db5411740f69183e22dbe7d50aef3e5cfde093319eaa39008fbd11c8"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 3,
+        "ids": [
+          8334,
+          29014,
+          494
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            3,
+            248320
+          ],
+          "sha256": "6821599090afcd8e042f955477276a52843d0087c6074a1a77a43b9768cd2c0c"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            3,
+            10240
+          ],
+          "sha256": "e358aa99aa0738c30f102ad646ebef5244b7740ca7ccaa3804469e3dc8ac217c"
+        }
+      }
+    },
+    {
+      "case": "reopened",
+      "prompt": [
+        248045,
+        8678,
+        198,
+        24342,
+        286,
+        4879,
+        369,
+        716,
+        310,
+        3238,
+        13,
+        13262,
+        678,
+        7047,
+        9522,
+        321,
+        10419,
+        11,
+        6992,
+        5774,
+        310,
+        279,
+        16198,
+        1973,
+        24366,
+        24150,
+        362,
+        13,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        2427,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198,
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        1698,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        1061,
+        198,
+        248069,
+        271,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        22791,
+        24217,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198
+      ],
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        2962,
+        24217,
+        13,
+        353,
+        8334,
+        29014,
+        494
+      ],
+      "cached": 58,
+      "state_sha256": "257714fbbf6f6a92b5562f554fae93c36abaff2e72bac7ad61a0d874e14e8234"
+    },
+    {
+      "case": "checkpoint",
+      "actual_old_rejected": true,
+      "warm_reopened_state_exact": true
+    }
+  ],
+  "off-product": [
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 0,
+        "ids": [
+          760,
+          1156,
+          369,
+          9859
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "16ff543d6e28cdbe7dbd56891e061ddd4b04291a9ca25bdf1f4437b4e3c5528d"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "b9cc3aa8c913e508793b7486f77a10e7acd4698436060b1b6bf8abfbd59edee6"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 1,
+        "ids": [
+          6587,
+          728,
+          310,
+          1698
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "24e58b3561aac59d52bf55183bc9b911432481e9613a5b98f1442945bfd6903a"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "eddb5de99ea61034c3f0eb46ef1a515cae10c92333d069afce68cc4a5e5f9e3e"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 2,
+        "ids": [
+          494,
+          799,
+          310,
+          5600
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "f5f2c33a36fce4088f222ec7448389865dd74ea4c017e4e2e49f157693a33d64"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "330d68ed62e77a12e38fe7677ffd2b9548f655c7d33e6e1607e431d1b8957e86"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 3,
+        "ids": [
+          13,
+          1061
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            2,
+            248320
+          ],
+          "sha256": "f44498bbf6e6e9d8d909c0e4db24b5ef9bda5866b4a6a14e188865432538643c"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            2,
+            10240
+          ],
+          "sha256": "dec8b787a562878d091c854235120732e98f5781751b6d6a428e5cd02228deb0"
+        }
+      }
+    },
+    {
+      "case": "cold",
+      "prompt": [
+        248045,
+        8678,
+        198,
+        24342,
+        286,
+        4879,
+        369,
+        716,
+        310,
+        3238,
+        13,
+        13262,
+        678,
+        7047,
+        9522,
+        321,
+        10419,
+        11,
+        6992,
+        5774,
+        310,
+        279,
+        16198,
+        1973,
+        24366,
+        24150,
+        362,
+        13,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        2427,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198
+      ],
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        1698,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        1061
+      ],
+      "cached": 0,
+      "state_sha256": "eb61c20e4a89bb55c0284d86587966af49d6c601e5421b50eff0762ecf6842b1"
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 0,
+        "ids": [
+          760,
+          1156,
+          6587,
+          728
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "dbab0d12f25f2be2234d58d45f86219af62deb20e22b0d73a77aa5e7b722d082"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "8f9aebc38d1a3e550bed6a4663eedac7a5ed3ce61d836d1cecdfa11da67ebd0d"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 1,
+        "ids": [
+          310,
+          2962,
+          24217,
+          494
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "d6b8e6afe3c6f5af068f240e0e69b3875311aac6929b3d5a38ab3aeaa1bf227c"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "52f20217ed43d8e59ddbe037cb77f56cc3d8ef473fd1963948fff6faf8ad4212"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 2,
+        "ids": [
+          13,
+          353,
+          2582,
+          3804
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "048e9343d4268d98455a01f7c73ee47af78bc811ef9e7680a72cae59336b9989"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "8e8b07fb20baa33ad64cc76ff64403a434d520c627a182a224cf033a5a14e5d4"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 3,
+        "ids": [
+          8334,
+          29014,
+          494
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            3,
+            248320
+          ],
+          "sha256": "82d671c6ac7b3a0952e60ef554ca670562fd01ef25c37bd10bc27025768ea0f2"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            3,
+            10240
+          ],
+          "sha256": "d1bc90f284364eae8bac80a5da6ab64c9c76d6994bb09ed463a17d67e25c016e"
+        }
+      }
+    },
+    {
+      "case": "warm",
+      "prompt": [
+        248045,
+        8678,
+        198,
+        24342,
+        286,
+        4879,
+        369,
+        716,
+        310,
+        3238,
+        13,
+        13262,
+        678,
+        7047,
+        9522,
+        321,
+        10419,
+        11,
+        6992,
+        5774,
+        310,
+        279,
+        16198,
+        1973,
+        24366,
+        24150,
+        362,
+        13,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        2427,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198,
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        1698,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        1061,
+        198,
+        248069,
+        271,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        22791,
+        24217,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198
+      ],
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        2962,
+        24217,
+        13,
+        353,
+        8334,
+        29014,
+        494
+      ],
+      "cached": 58,
+      "state_sha256": "e6cb7355aaa92a3569a365c94c55aa7a50bd731dc04f6fa320da8520d182e2f8"
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 0,
+        "ids": [
+          760,
+          1156,
+          6587,
+          728
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "dbab0d12f25f2be2234d58d45f86219af62deb20e22b0d73a77aa5e7b722d082"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "8f9aebc38d1a3e550bed6a4663eedac7a5ed3ce61d836d1cecdfa11da67ebd0d"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 1,
+        "ids": [
+          310,
+          2962,
+          24217,
+          494
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "d6b8e6afe3c6f5af068f240e0e69b3875311aac6929b3d5a38ab3aeaa1bf227c"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "52f20217ed43d8e59ddbe037cb77f56cc3d8ef473fd1963948fff6faf8ad4212"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 2,
+        "ids": [
+          13,
+          353,
+          2582,
+          3804
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            4,
+            248320
+          ],
+          "sha256": "048e9343d4268d98455a01f7c73ee47af78bc811ef9e7680a72cae59336b9989"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            4,
+            10240
+          ],
+          "sha256": "8e8b07fb20baa33ad64cc76ff64403a434d520c627a182a224cf033a5a14e5d4"
+        }
+      }
+    },
+    {
+      "case": "verify_forward",
+      "record": {
+        "index": 3,
+        "ids": [
+          8334,
+          29014,
+          494
+        ],
+        "verify": true,
+        "logits": {
+          "shape": [
+            1,
+            3,
+            248320
+          ],
+          "sha256": "82d671c6ac7b3a0952e60ef554ca670562fd01ef25c37bd10bc27025768ea0f2"
+        },
+        "hidden": {
+          "shape": [
+            1,
+            3,
+            10240
+          ],
+          "sha256": "d1bc90f284364eae8bac80a5da6ab64c9c76d6994bb09ed463a17d67e25c016e"
+        }
+      }
+    },
+    {
+      "case": "reopened",
+      "prompt": [
+        248045,
+        8678,
+        198,
+        24342,
+        286,
+        4879,
+        369,
+        716,
+        310,
+        3238,
+        13,
+        13262,
+        678,
+        7047,
+        9522,
+        321,
+        10419,
+        11,
+        6992,
+        5774,
+        310,
+        279,
+        16198,
+        1973,
+        24366,
+        24150,
+        362,
+        13,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        2427,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198,
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        1698,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        1061,
+        198,
+        248069,
+        271,
+        248046,
+        198,
+        248045,
+        846,
+        198,
+        22791,
+        24217,
+        13,
+        248046,
+        198,
+        248045,
+        74455,
+        198,
+        248068,
+        198
+      ],
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        2962,
+        24217,
+        13,
+        353,
+        8334,
+        29014,
+        494
+      ],
+      "cached": 58,
+      "state_sha256": "e6cb7355aaa92a3569a365c94c55aa7a50bd731dc04f6fa320da8520d182e2f8"
+    },
+    {
+      "case": "checkpoint",
+      "actual_old_rejected": false,
+      "warm_reopened_state_exact": true
+    }
+  ],
+  "model-oracle": [
+    {
+      "case": "environment",
+      "overrides": {
+        "MTPLX_CONTEXT_COPY": "0",
+        "MTPLX_SKIP_VERIFY_SNAPSHOT": "0",
+        "MTPLX_FAMILY_CAPTURE_COMMIT": "1",
+        "MTPLX_FUSED_HC_V3": "1",
+        "MTPLX_FUSED_GDN_INPROJ": "1",
+        "MTPLX_FUSED_GATE_UP": "1",
+        "MTPLX_FUSED_GDN_CONVNORM": "1",
+        "MTPLX_FUSED_GDN_STEP": "1",
+        "MTPLX_FUSED_CONVNORM_VERIFY": "1",
+        "MTPLX_QSA_GATHER": "1",
+        "MTPLX_COMPILED_GDN": "1",
+        "MTPLX_QWEN4EXP_COMPILE": "1",
+        "MTPLX_QWEN4_BATCHED_TARGET_DISTRIBUTIONS": "1",
+        "MTPLX_QWEN4_FIXED_M4_VERIFY": "1",
+        "MTPLX_QWEN4_COMPILED_MTP_PREPARE": "1",
+        "MTPLX_QWEN4_RELAXED_DRAFT_TIES": "1",
+        "MTPLX_QWEN4_OPDIET": "1",
+        "MTPLX_QWEN4_PLE_PREFILL_LOOKAHEAD": "1",
+        "MTPLX_QWEN4_PLE_FIRST_GATHER_EARLY": "1",
+        "MTPLX_SESSION_BANK_SHED_BOUNDARIES": "1",
+        "MTPLX_SESSION_BANK_PROTECTED_TERMINAL": "1",
+        "MTPLX_NGRAM_PREWARM": "auto",
+        "MTPLX_QWEN4_VERIFY_GLUE": "1",
+        "MTPLX_QWEN4_VERIFY_GLUE_ITEMS": "qsa_rope,qsa_rope_idx",
+        "MTPLX_QSA_M4_FUSED_KV_GATHER": "1",
+        "MTPLX_QSA_GATHER_MAX_ROWS": "32",
+        "MTPLX_BATCH_TARGET_ARRAYS": "1",
+        "MTPLX_LAZY_TARGET_DISTRIBUTIONS": "0",
+        "MTPLX_COMPILED_VERIFY": "1",
+        "MTPLX_NAX_VERIFY": "0",
+        "MTPLX_QWEN4_PREFILL_WIDE_CHUNK": "4096",
+        "MTPLX_QSA_PREFILL_WIDE_MIN_CONTEXT": "16384",
+        "MTPLX_SUSTAINED_PREFILL": "1",
+        "MTPLX_PREFILL_CHUNK_SIZE": "2048"
+      }
+    },
+    {
+      "case": "prefill_compare",
+      "phase": "cold",
+      "part": "logits",
+      "shape": [
+        1,
+        1,
+        248320
+      ],
+      "exact": true
+    },
+    {
+      "case": "prefill_compare",
+      "phase": "cold",
+      "part": "hidden",
+      "shape": [
+        1,
+        1,
+        10240
+      ],
+      "exact": true
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        760
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "cold",
+      "index": 0,
+      "ids": [
+        760,
+        1156,
+        369,
+        9859
+      ],
+      "logits_exact": true,
+      "logits_sha256": "e3ba038c8570911a72fadeba867f1c33e5103108ef9881bfcdc008907de90839",
+      "hidden_exact": true,
+      "hidden_sha256": "b3c2ef7bb9c3dfa85748d7af883210ae4393bd1b5af5397f243d632fc437552b"
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        1156
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        6587
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "cold",
+      "index": 1,
+      "ids": [
+        6587,
+        728,
+        310,
+        1698
+      ],
+      "logits_exact": true,
+      "logits_sha256": "955ff038997bab8ca6a84642b1ad8cf88dade0d2075d5ea11c736c9f74a7d563",
+      "hidden_exact": true,
+      "hidden_sha256": "e3a029cc190a3cbdee1e2ec34bd94181f6013a6495c56b60dcccd67130fcfbd4"
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        728,
+        310,
+        1698
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        494
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "cold",
+      "index": 2,
+      "ids": [
+        494,
+        799,
+        310,
+        5600
+      ],
+      "logits_exact": true,
+      "logits_sha256": "e201b5e579d7c8f58432ae495c120c66cd595241046dd9cd32d42cab72009b3b",
+      "hidden_exact": true,
+      "hidden_sha256": "d56aba5df3172aeca9d26bccfe6143d8045d30e5af45f1fc6815d61634f78d2c"
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        799,
+        310,
+        5600
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        13
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "cold",
+      "index": 3,
+      "ids": [
+        13,
+        1061
+      ],
+      "logits_exact": true,
+      "logits_sha256": "ecf42a91034a5bd6739a3d5bc94ffdcbac8214261f5712a3196bb6b99be67860",
+      "hidden_exact": true,
+      "hidden_sha256": "0296bbc47e09e0312470a1ada7ef90177009d80cef5ba79398190177bb4be7c3"
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        1061
+      ]
+    },
+    {
+      "case": "cache_arrays",
+      "phase": "cold",
+      "values_exact": true,
+      "zero_sign_differences": [],
+      "key35_bit_coordinates": []
+    },
+    {
+      "case": "turn",
+      "phase": "cold",
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        1698,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        1061
+      ],
+      "cached": 0,
+      "verify_calls": 4,
+      "final_exact": true
+    },
+    {
+      "case": "prefill_compare",
+      "phase": "warm",
+      "part": "logits",
+      "shape": [
+        1,
+        1,
+        248320
+      ],
+      "exact": true
+    },
+    {
+      "case": "prefill_compare",
+      "phase": "warm",
+      "part": "hidden",
+      "shape": [
+        1,
+        1,
+        10240
+      ],
+      "exact": true
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        760
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "warm",
+      "index": 0,
+      "ids": [
+        760,
+        1156,
+        6587,
+        728
+      ],
+      "logits_exact": true,
+      "logits_sha256": "0d002e5ab00ee76250e5deca14358353c777fd1f1804937070ca229f74cdc9f6",
+      "hidden_exact": true,
+      "hidden_sha256": "cb6bc68f9d37d0ffb544b2c635144d8f91a715c5172fa31a32be7aab84d5941b"
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        1156,
+        6587,
+        728
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        310
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "warm",
+      "index": 1,
+      "ids": [
+        310,
+        2962,
+        24217,
+        494
+      ],
+      "logits_exact": true,
+      "logits_sha256": "e052a03d4908e670c534c18453447dcb7190bf180dd281ffd3ace950675174e1",
+      "hidden_exact": true,
+      "hidden_sha256": "2caa9c5b1d863080a4ec81e55103e9b3429261933f13cac971dfc09a8a1c130f"
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        2962,
+        24217
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        13
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "warm",
+      "index": 2,
+      "ids": [
+        13,
+        353,
+        2582,
+        3804
+      ],
+      "logits_exact": true,
+      "logits_sha256": "f64c3236f08fa43f85ee491cc1121d5f584dfd1fbcd58b1bf9ef69b441813036",
+      "hidden_exact": true,
+      "hidden_sha256": "eba14099db5411740f69183e22dbe7d50aef3e5cfde093319eaa39008fbd11c8"
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        353
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        8334
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "warm",
+      "index": 3,
+      "ids": [
+        8334,
+        29014,
+        494
+      ],
+      "logits_exact": true,
+      "logits_sha256": "6821599090afcd8e042f955477276a52843d0087c6074a1a77a43b9768cd2c0c",
+      "hidden_exact": true,
+      "hidden_sha256": "e358aa99aa0738c30f102ad646ebef5244b7740ca7ccaa3804469e3dc8ac217c"
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        29014,
+        494
+      ]
+    },
+    {
+      "case": "cache_arrays",
+      "phase": "warm",
+      "values_exact": true,
+      "zero_sign_differences": [],
+      "key35_bit_coordinates": []
+    },
+    {
+      "case": "turn",
+      "phase": "warm",
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        2962,
+        24217,
+        13,
+        353,
+        8334,
+        29014,
+        494
+      ],
+      "cached": 58,
+      "verify_calls": 4,
+      "final_exact": true
+    }
+  ],
+  "off-oracle": [
+    {
+      "case": "environment",
+      "overrides": {
+        "MTPLX_CONTEXT_COPY": "0",
+        "MTPLX_QWEN4_VERIFY_SDPA_HEAD_CHUNK": "0",
+        "MTPLX_SKIP_VERIFY_SNAPSHOT": "0",
+        "MTPLX_FAMILY_CAPTURE_COMMIT": "1",
+        "MTPLX_FUSED_HC_V3": "1",
+        "MTPLX_FUSED_GDN_INPROJ": "1",
+        "MTPLX_FUSED_GATE_UP": "1",
+        "MTPLX_FUSED_GDN_CONVNORM": "1",
+        "MTPLX_FUSED_GDN_STEP": "1",
+        "MTPLX_FUSED_CONVNORM_VERIFY": "1",
+        "MTPLX_QSA_GATHER": "1",
+        "MTPLX_COMPILED_GDN": "1",
+        "MTPLX_QWEN4EXP_COMPILE": "1",
+        "MTPLX_QWEN4_BATCHED_TARGET_DISTRIBUTIONS": "1",
+        "MTPLX_QWEN4_FIXED_M4_VERIFY": "1",
+        "MTPLX_QWEN4_COMPILED_MTP_PREPARE": "1",
+        "MTPLX_QWEN4_RELAXED_DRAFT_TIES": "1",
+        "MTPLX_QWEN4_OPDIET": "1",
+        "MTPLX_QWEN4_PLE_PREFILL_LOOKAHEAD": "1",
+        "MTPLX_QWEN4_PLE_FIRST_GATHER_EARLY": "1",
+        "MTPLX_SESSION_BANK_SHED_BOUNDARIES": "1",
+        "MTPLX_SESSION_BANK_PROTECTED_TERMINAL": "1",
+        "MTPLX_NGRAM_PREWARM": "auto",
+        "MTPLX_QWEN4_VERIFY_GLUE": "1",
+        "MTPLX_QWEN4_VERIFY_GLUE_ITEMS": "qsa_rope,qsa_rope_idx",
+        "MTPLX_QSA_M4_FUSED_KV_GATHER": "1",
+        "MTPLX_QSA_GATHER_MAX_ROWS": "32",
+        "MTPLX_BATCH_TARGET_ARRAYS": "1",
+        "MTPLX_LAZY_TARGET_DISTRIBUTIONS": "0",
+        "MTPLX_COMPILED_VERIFY": "1",
+        "MTPLX_NAX_VERIFY": "0",
+        "MTPLX_QWEN4_PREFILL_WIDE_CHUNK": "4096",
+        "MTPLX_QSA_PREFILL_WIDE_MIN_CONTEXT": "16384",
+        "MTPLX_SUSTAINED_PREFILL": "1",
+        "MTPLX_PREFILL_CHUNK_SIZE": "2048"
+      }
+    },
+    {
+      "case": "prefill_compare",
+      "phase": "cold",
+      "part": "logits",
+      "shape": [
+        1,
+        1,
+        248320
+      ],
+      "exact": true
+    },
+    {
+      "case": "prefill_compare",
+      "phase": "cold",
+      "part": "hidden",
+      "shape": [
+        1,
+        1,
+        10240
+      ],
+      "exact": true
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        760
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "cold",
+      "index": 0,
+      "ids": [
+        760,
+        1156,
+        369,
+        9859
+      ],
+      "logits_exact": true,
+      "logits_sha256": "16ff543d6e28cdbe7dbd56891e061ddd4b04291a9ca25bdf1f4437b4e3c5528d",
+      "hidden_exact": true,
+      "hidden_sha256": "b9cc3aa8c913e508793b7486f77a10e7acd4698436060b1b6bf8abfbd59edee6"
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        1156
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        6587
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "cold",
+      "index": 1,
+      "ids": [
+        6587,
+        728,
+        310,
+        1698
+      ],
+      "logits_exact": true,
+      "logits_sha256": "24e58b3561aac59d52bf55183bc9b911432481e9613a5b98f1442945bfd6903a",
+      "hidden_exact": true,
+      "hidden_sha256": "eddb5de99ea61034c3f0eb46ef1a515cae10c92333d069afce68cc4a5e5f9e3e"
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        728,
+        310,
+        1698
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        494
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "cold",
+      "index": 2,
+      "ids": [
+        494,
+        799,
+        310,
+        5600
+      ],
+      "logits_exact": true,
+      "logits_sha256": "f5f2c33a36fce4088f222ec7448389865dd74ea4c017e4e2e49f157693a33d64",
+      "hidden_exact": true,
+      "hidden_sha256": "330d68ed62e77a12e38fe7677ffd2b9548f655c7d33e6e1607e431d1b8957e86"
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        799,
+        310,
+        5600
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        13
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "cold",
+      "index": 3,
+      "ids": [
+        13,
+        1061
+      ],
+      "logits_exact": true,
+      "logits_sha256": "f44498bbf6e6e9d8d909c0e4db24b5ef9bda5866b4a6a14e188865432538643c",
+      "hidden_exact": true,
+      "hidden_sha256": "dec8b787a562878d091c854235120732e98f5781751b6d6a428e5cd02228deb0"
+    },
+    {
+      "case": "tokens",
+      "phase": "cold",
+      "ids": [
+        1061
+      ]
+    },
+    {
+      "case": "cache_arrays",
+      "phase": "cold",
+      "values_exact": true,
+      "zero_sign_differences": [],
+      "key35_bit_coordinates": []
+    },
+    {
+      "case": "turn",
+      "phase": "cold",
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        1698,
+        494,
+        799,
+        310,
+        5600,
+        13,
+        1061
+      ],
+      "cached": 0,
+      "verify_calls": 4,
+      "final_exact": true
+    },
+    {
+      "case": "prefill_compare",
+      "phase": "warm",
+      "part": "logits",
+      "shape": [
+        1,
+        1,
+        248320
+      ],
+      "exact": true
+    },
+    {
+      "case": "prefill_compare",
+      "phase": "warm",
+      "part": "hidden",
+      "shape": [
+        1,
+        1,
+        10240
+      ],
+      "exact": true
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        760
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "warm",
+      "index": 0,
+      "ids": [
+        760,
+        1156,
+        6587,
+        728
+      ],
+      "logits_exact": true,
+      "logits_sha256": "dbab0d12f25f2be2234d58d45f86219af62deb20e22b0d73a77aa5e7b722d082",
+      "hidden_exact": true,
+      "hidden_sha256": "8f9aebc38d1a3e550bed6a4663eedac7a5ed3ce61d836d1cecdfa11da67ebd0d"
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        1156,
+        6587,
+        728
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        310
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "warm",
+      "index": 1,
+      "ids": [
+        310,
+        2962,
+        24217,
+        494
+      ],
+      "logits_exact": true,
+      "logits_sha256": "d6b8e6afe3c6f5af068f240e0e69b3875311aac6929b3d5a38ab3aeaa1bf227c",
+      "hidden_exact": true,
+      "hidden_sha256": "52f20217ed43d8e59ddbe037cb77f56cc3d8ef473fd1963948fff6faf8ad4212"
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        2962,
+        24217
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        13
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "warm",
+      "index": 2,
+      "ids": [
+        13,
+        353,
+        2582,
+        3804
+      ],
+      "logits_exact": true,
+      "logits_sha256": "048e9343d4268d98455a01f7c73ee47af78bc811ef9e7680a72cae59336b9989",
+      "hidden_exact": true,
+      "hidden_sha256": "8e8b07fb20baa33ad64cc76ff64403a434d520c627a182a224cf033a5a14e5d4"
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        353
+      ]
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        8334
+      ]
+    },
+    {
+      "case": "verify_forward",
+      "phase": "warm",
+      "index": 3,
+      "ids": [
+        8334,
+        29014,
+        494
+      ],
+      "logits_exact": true,
+      "logits_sha256": "82d671c6ac7b3a0952e60ef554ca670562fd01ef25c37bd10bc27025768ea0f2",
+      "hidden_exact": true,
+      "hidden_sha256": "d1bc90f284364eae8bac80a5da6ab64c9c76d6994bb09ed463a17d67e25c016e"
+    },
+    {
+      "case": "tokens",
+      "phase": "warm",
+      "ids": [
+        29014,
+        494
+      ]
+    },
+    {
+      "case": "cache_arrays",
+      "phase": "warm",
+      "values_exact": true,
+      "zero_sign_differences": [],
+      "key35_bit_coordinates": []
+    },
+    {
+      "case": "turn",
+      "phase": "warm",
+      "tokens": [
+        760,
+        1156,
+        6587,
+        728,
+        310,
+        2962,
+        24217,
+        13,
+        353,
+        8334,
+        29014,
+        494
+      ],
+      "cached": 58,
+      "verify_calls": 4,
+      "final_exact": true
+    }
+  ],
+  "gates": [
+    {
+      "case": "gate",
+      "dtype": "bf16",
+      "phase": "prefill",
+      "values": 65280,
+      "exact": true
+    },
+    {
+      "case": "inject",
+      "phase": "prefill",
+      "values": 65280,
+      "exact": true
+    },
+    {
+      "case": "gate",
+      "dtype": "bf16",
+      "phase": "ar_decode",
+      "values": 65280,
+      "exact": true
+    },
+    {
+      "case": "inject",
+      "phase": "ar_decode",
+      "values": 65280,
+      "exact": true
+    },
+    {
+      "case": "gate",
+      "dtype": "bf16",
+      "phase": "decode_verify",
+      "values": 65280,
+      "exact": true
+    },
+    {
+      "case": "inject",
+      "phase": "decode_verify",
+      "values": 65280,
+      "exact": true
+    },
+    {
+      "case": "gate",
+      "dtype": "f16",
+      "phase": "prefill",
+      "values": 63488,
+      "exact": true
+    },
+    {
+      "case": "gate",
+      "dtype": "f16",
+      "phase": "ar_decode",
+      "values": 63488,
+      "exact": true
+    },
+    {
+      "case": "gate",
+      "dtype": "f16",
+      "phase": "decode_verify",
+      "values": 63488,
+      "exact": true
+    },
+    {
+      "case": "gate",
+      "dtype": "f32",
+      "phase": "prefill",
+      "values": 65536,
+      "exact": true
+    },
+    {
+      "case": "gate",
+      "dtype": "f32",
+      "phase": "ar_decode",
+      "values": 65536,
+      "exact": true
+    },
+    {
+      "case": "gate",
+      "dtype": "f32",
+      "phase": "decode_verify",
+      "values": 65536,
+      "exact": true
+    }
+  ]
+}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/resource-summaries.json b/docs/issue113-head-chunk/resource-summaries.json
new file mode 100644
index 0000000..5608c75
--- /dev/null
+++ b/docs/issue113-head-chunk/resource-summaries.json
@@ -0,0 +1,210 @@
+{
+  "fixtures": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 1393,
+    "peak": {
+      "cpu_time_ns": 97251291,
+      "resident_bytes": 70975488,
+      "physical_bytes": 177374120,
+      "peak_physical_bytes": 178209680,
+      "disk_read_bytes": 3022848,
+      "disk_write_bytes": 32768
+    },
+    "error": null
+  },
+  "checkpoint": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 547,
+    "peak": {
+      "cpu_time_ns": 42551583,
+      "resident_bytes": 60342272,
+      "physical_bytes": 29934312,
+      "peak_physical_bytes": 29934312,
+      "disk_read_bytes": 11739136,
+      "disk_write_bytes": 49152
+    },
+    "error": null
+  },
+  "product": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 11862,
+    "peak": {
+      "cpu_time_ns": 13477662708,
+      "resident_bytes": 25462865920,
+      "physical_bytes": 77020489800,
+      "peak_physical_bytes": 77020489800,
+      "disk_read_bytes": 73493147648,
+      "disk_write_bytes": 286756864
+    },
+    "error": null
+  },
+  "off-product": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 11658,
+    "peak": {
+      "cpu_time_ns": 15682090666,
+      "resident_bytes": 25093898240,
+      "physical_bytes": 76906686368,
+      "peak_physical_bytes": 76906686368,
+      "disk_read_bytes": 73721593856,
+      "disk_write_bytes": 286756864
+    },
+    "error": null
+  },
+  "model-oracle": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 15366,
+    "peak": {
+      "cpu_time_ns": 17251837750,
+      "resident_bytes": 25390841856,
+      "physical_bytes": 76987541504,
+      "peak_physical_bytes": 76988311456,
+      "disk_read_bytes": 105532772352,
+      "disk_write_bytes": 18446744073709305856
+    },
+    "error": null
+  },
+  "off-oracle": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 14548,
+    "peak": {
+      "cpu_time_ns": 16934928666,
+      "resident_bytes": 24856903680,
+      "physical_bytes": 76988049384,
+      "peak_physical_bytes": 76988835696,
+      "disk_read_bytes": 103881760768,
+      "disk_write_bytes": 32665600
+    },
+    "error": null
+  },
+  "mtplx_qsa_eager_array_graph_matches_reference": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 1150,
+    "peak": {
+      "cpu_time_ns": 838671541,
+      "resident_bytes": 114753536,
+      "physical_bytes": 281641800,
+      "peak_physical_bytes": 289866592,
+      "disk_read_bytes": 29503488,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "mtplx_qsa_dense_mask_graph_matches_reference": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 839,
+    "peak": {
+      "cpu_time_ns": 633162250,
+      "resident_bytes": 194674688,
+      "physical_bytes": 353747832,
+      "peak_physical_bytes": 364217208,
+      "disk_read_bytes": 458752,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "mtplx_qsa_prefill_gather_graph_matches_reference": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 525,
+    "peak": {
+      "cpu_time_ns": 332477666,
+      "resident_bytes": 79740928,
+      "physical_bytes": 256574184,
+      "peak_physical_bytes": 262914792,
+      "disk_read_bytes": 245760,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "mtplx_qsa_rows_gather_graph_matches_reference": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 310,
+    "peak": {
+      "cpu_time_ns": 164053875,
+      "resident_bytes": 84410368,
+      "physical_bytes": 301417192,
+      "peak_physical_bytes": 326026008,
+      "disk_read_bytes": 0,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "mtplx_sdpa_fallback_graph_matches_reference": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 841,
+    "peak": {
+      "cpu_time_ns": 643992875,
+      "resident_bytes": 195117056,
+      "physical_bytes": 501580664,
+      "peak_physical_bytes": 504791928,
+      "disk_read_bytes": 376832,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "qsa-module-0": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 1038,
+    "peak": {
+      "cpu_time_ns": 689164750,
+      "resident_bytes": 544718848,
+      "physical_bytes": 1604994344,
+      "peak_physical_bytes": 1607386432,
+      "disk_read_bytes": 4894720,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "qsa-module-1": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 1048,
+    "peak": {
+      "cpu_time_ns": 712567875,
+      "resident_bytes": 545734656,
+      "physical_bytes": 1605911824,
+      "peak_physical_bytes": 1605944592,
+      "disk_read_bytes": 114688,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "gates": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 312,
+    "peak": {
+      "cpu_time_ns": 54212208,
+      "resident_bytes": 65257472,
+      "physical_bytes": 153076432,
+      "peak_physical_bytes": 153404112,
+      "disk_read_bytes": 5132288,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "deepseek": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 9789,
+    "peak": {
+      "cpu_time_ns": 2929129750,
+      "resident_bytes": 130400256,
+      "physical_bytes": 308790112,
+      "peak_physical_bytes": 319767392,
+      "disk_read_bytes": 86721708032,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  },
+  "glm": {
+    "event": "test_resource_summary",
+    "elapsed_ms": 11991,
+    "peak": {
+      "cpu_time_ns": 3695045041,
+      "resident_bytes": 905674752,
+      "physical_bytes": 1337132544,
+      "peak_physical_bytes": 1488602768,
+      "disk_read_bytes": 96508018688,
+      "disk_write_bytes": 0
+    },
+    "error": null
+  }
+}
\ No newline at end of file
diff --git a/docs/issue113-head-chunk/tests.log b/docs/issue113-head-chunk/tests.log
new file mode 100644
index 0000000..19f04a0
--- /dev/null
+++ b/docs/issue113-head-chunk/tests.log
@@ -0,0 +1,592 @@
+   Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
+    Finished `test` profile [unoptimized + debuginfo] target(s) in 14.50s
+warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
+note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
+     Running unittests src/main.rs (target/debug/deps/ds4_server-5679c54d8848b708)
+
+running 549 tests
+test agent::tests::anchored_edits_require_unique_head_and_tail ... ok
+test a2ui::tests::streaming_parser_only_accepts_complete_a2ui_jsonl_lines ... ok
+test agent::tests::ai_risk_response_requires_structured_risk_and_reason ... ok
+test agent::tests::ai_shell_risk_assessment_receives_the_dev_brain_root ... ok
+test a2ui::tests::embedded_catalog_and_validator_cover_the_same_components ... ok
+test agent::tests::dev_brain_paths_are_trusted_by_shell_risk_assessment ... ok
+test a2ui::tests::basic_formatters_follow_catalog_examples ... ok
+test a2ui::tests::dismissal_boundaries_replay_history_and_a_fresh_active_epoch ... ok
+test a2ui::tests::catalog_accepts_pie_donut_and_heatmap_charts ... ok
+test a2ui::tests::catalog_validation_rejects_invalid_variants_and_function_arguments ... ok
+test a2ui::tests::latest_created_surface_is_the_active_surface ... ok
+test a2ui::tests::lifecycle_updates_incrementally_and_json_pointer_upserts ... ok
+test a2ui::tests::catalog_validation_and_actions_use_current_local_data ... ok
+test agent::tests::ralph_loop_stops_on_cap_blocker_failure_and_cancellation ... ok
+test agent::tests::present_svg_parsers_recover_exact_source_for_both_model_transports ... ok
+test a2ui::tests::v1_action_and_function_round_trips_follow_current_schema ... ok
+test a2ui::tests::all_basics_compose_into_one_surface ... ok
+test agent::tests::ralph_report_semantics_and_handoff_bound_are_enforced ... ok
+test agent::tests::risky_shell_commands_require_one_time_approval ... ok
+test agent::tests::model_adapters_preserve_primitives_and_report_recoverable_syntax_errors ... ok
+test agent::tests::scripted_ralph_backend_gets_fresh_prompts_and_only_validated_handoffs ... ok
+test agent::tests::generated_schemas_match_the_executable_tool_contracts ... ok
+test agent::tests::project_boundary_rejects_parent_paths ... ok
+test agent::tests::repeated_call_advisories_use_canonical_arguments_and_reset_per_user_turn ... ok
+test a2ui::export::tests::static_svg_renderer_covers_the_complete_catalog ... ok
+test agent::tests::stored_tool_cards_restore_ai_approval_reasons_by_call ... ok
+test agent::tests::ralph_report_schemas_are_child_only_and_parse_for_both_models ... ok
+test agent::tests::present_svg_executes_without_side_effects_and_maps_only_completed_cards ... ok
+test agent::tests::tool_output_keeps_terminal_lines_but_parameter_summaries_do_not ... ok
+test agent::web::tests::browser_helpers_preserve_ds4_wire_values ... ok
+test agent::web::tests::headless_browser_extracts_a_local_page ... ignored, requires an installed Chrome browser
+test agent::tests::glob_confinement_skips_symlinks_and_honors_cancellation ... ok
+test agent::tests::validation_is_precise_and_prevents_side_effects_until_corrected ... ok
+test app::generation::tests::attached_image_marker_is_model_only ... ok
+test agent::tests::present_svg_validation_enforces_the_static_vector_boundary ... ok
+test agent::tests::prompts_and_parsers_expose_the_reference_tool_set ... ok
+test agent::tests::successful_structured_file_tools_report_project_paths ... ok
+test agent::tests::glob_finds_all_regular_file_names_deterministically_without_reading_them ... ok
+test agent::tests::standard_agent_skills_are_discovered_prompted_and_read_only ... ok
+test agent::tests::ralph_round_contexts_are_unique_and_child_jobs_are_cleaned_up ... ok
+test app::generation::tests::plain_chat_uses_the_same_unmodified_prompt_in_ui_and_headless ... ok
+test agent::tests::shell_commands_skip_user_startup_files ... ok
+test app::generation::tests::queued_guidance_is_one_reference_style_user_turn ... ok
+test app::generation::tests::title_context_starts_at_the_first_user_and_ignores_system_rows ... ok
+test app::generation::tests::turn_summary_accumulates_model_continuations ... ok
+test agent::tests::shell_environment_is_loaded_once_from_login_and_interactive_startup_files ... ok
+test agent::tests::standard_file_tools_can_maintain_only_managed_dev_brain_pages ... ok
+test app::git::tests::unified_diff_parser_aligns_replacements_for_split_view ... ok
+test app::git::tests::unstage_works_before_the_first_commit ... ok
+test app::git::tests::configured_git_diff_options_change_the_rendered_patch ... ok
+test app::preferences::tests::runtime_config_updates_after_lock_poisoning ... ok
+test app::git::tests::libgit2_status_merges_staged_and_worktree_changes ... ok
+test app::projects::tests::discarding_a_session_removes_every_checkpoint_stage ... ok
+test app::projects::tests::draft_titles_follow_the_stored_session_count ... ok
+test agent::tests::denial_and_stop_cancel_commands_awaiting_approval ... ok
+test app::projects::tests::session_states_are_exclusive_and_ordered ... ok
+test app::projects::tests::sweeping_removes_checkpoints_of_sessions_that_no_longer_exist ... ok
+test app::projects::tests::git_state_lists_and_switches_local_branches ... ok
+test app::tests::background_activity_drives_polling_indicators_and_quit_confirmation ... ok
+test app::git::tests::origin_fetch_pull_and_push_use_the_current_branch ... ok
+test agent::tests::shell_environment_probe_stops_blocked_startup_files ... ok
+test app::tests::chat_html_export_embeds_loaded_a2ui_images_as_data_urls ... ok
+test agent::tests::oversized_shell_output_is_bounded_and_retained_until_cleanup ... ok
+test app::tests::chat_only_follows_updates_while_near_the_tail ... ok
+test app::tests::ds4_gib_and_streaming_cache_inputs_are_typed ... ok
+test app::tests::one_shot_titles_are_reduced_to_a_single_sidebar_line ... ok
+test app::tests::owned_runtime_paths_stay_in_application_support ... ok
+test app::tests::preferences_shortcut_and_dspark_support_are_explicit ... ok
+test app::tests::source_does_not_depend_on_a_sibling_ds4_checkout ... ok
+test app::tests::stored_session_always_selects_its_model ... ok
+test app::tests::tab_scrolls_a_field_back_into_the_preferences_viewport ... ok
+test app::view::a2ui::tests::renderer_input_helpers_cover_dates_numbers_choices_and_weights ... ok
+test app::view::chat::tests::chat_divider_groups_token_counts_by_thousands ... ok
+test app::view::chat::tests::composer_uses_shift_enter_for_line_breaks ... ok
+test app::view::git::tests::split_diff_width_covers_the_longest_side_and_tabs ... ok
+test app::tests::chat_export_preserves_completed_svg_and_keeps_failures_generic ... ok
+test app::view::tests::action_buttons_share_geometry ... ok
+test app::view::tests::context_pie_wedge_follows_the_fill_grade ... ok
+test app::view::tests::download_measurements_are_readable ... ok
+test app::view::tests::raised_surfaces_stay_dark ... ok
+test compaction::tests::a_new_summary_replaces_the_previous_rebuild_summary ... ok
+test compaction::tests::hard_trigger_and_retry_reserve_match_the_reference ... ok
+test compaction::tests::soft_trigger_matches_reference_thresholds ... ok
+test compaction::tests::summary_budget_keeps_answer_room_and_rejects_exhausted_contexts ... ok
+test compaction::tests::summary_control_markup_is_private ... ok
+test compaction::tests::tail_prefers_the_first_user_boundary_after_target ... ok
+test config::tests::defaults_write_an_empty_file_and_reload_unchanged ... ok
+test config::tests::deprecated_flash_config_migrates_and_persists_only_0731 ... ok
+test config::tests::explicit_0731_dspark_opt_out_survives_the_default ... ok
+test app::preferences::tests::selectors_rehydrate_their_generation_and_model_profiles ... ok
+test config::tests::invalid_files_are_reported_instead_of_reset ... ok
+test config::tests::model_profiles_expose_only_their_upstream_thinking_modes ... ok
+test config::tests::legacy_flat_model_settings_are_dropped_but_the_prompt_is_kept ... ok
+test config::tests::only_changed_values_are_written_and_read_back ... ok
+test config::tests::qwen_selection_and_native_defaults_survive_restart ... ok
+test app::generation::tests::model_history_uses_persisted_content_without_changing_the_transcript ... ok
+test app::preferences::tests::multiline_system_prompt_round_trips_through_the_editor ... ok
+test app::generation::tests::legacy_generation_rows_collapse_into_their_user_turn ... ok
+test app::generation::tests::assistant_markdown_keeps_llm_tables ... ok
+test app::generation::tests::forced_compaction_requires_new_visible_chat ... ok
+test app::tests::assistant_stream_keeps_reasoning_separate_from_the_answer ... ok
+test app::tests::chat_export_formats_visible_turns_reasoning_and_bash_tools ... ok
+test app::generation::tests::complete_tool_call_in_reasoning_gets_one_correction ... ok
+test app::view::stats::tests::current_session_summary_uses_persisted_chat_facts ... ok
+test app::generation::tests::extension_context_deduplicates_session_start_and_rearms_after_compaction ... ok
+test app::generation::tests::last_compaction_selects_its_tail_without_hiding_history ... ok
+test app::tests::chat_html_export_is_safe_standalone_and_renders_embedded_surfaces ... ok
+test app::generation::tests::a2ui_auto_switch_waits_for_a_renderable_surface ... ok
+test dev_brain::tests::path_confinement_and_unrelated_notes_are_preserved ... ok
+test database::tests::context_work_reactivates_an_archived_session ... ok
+test dsml::tests::parses_reference_and_legacy_dsml_syntaxes ... ok
+test dsml::tests::recovers_complete_invokes_before_an_incomplete_one ... ok
+test dsml::tests::recovers_complete_invokes_without_the_outer_closing_tag ... ok
+test engine::ds4_sampling::tests::heap_fallback_preserves_rng_and_signed_zero_ties ... ok
+test engine::gguf::tests::accepts_exact_mxfp4_block_layout ... ok
+test engine::gguf::tests::keeps_unknown_tensor_kinds_rejected ... ok
+test engine::gguf::tests::maps_metadata_and_tensor_payload_without_copying_weights ... ok
+test engine::gguf::tests::mxfp4_scalar_dot_matches_independent_values ... ok
+test engine::gguf::tests::rejects_mxfp4_dimension_overflow ... ok
+test engine::gguf::tests::rejects_mxfp4_rows_that_are_not_block_aligned ... ok
+test engine::gguf::tests::rejects_truncated_mxfp4_payload ... ok
+test database::tests::a2ui_dismissal_persists_a_fresh_surface_boundary ... ok
+test database::tests::sessions_are_ordered_by_recent_use_within_sidebar_groups ... ok
+test database::tests::inline_svg_tool_call_survives_session_switch_compaction_and_reopen ... ok
+test engine::metal::glm::tests::dsa_indexer_schedule_matches_the_reference ... ok
+test engine::metal::glm::tests::glm53_m5_text_performance_gate ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm53_m5_vision_performance_and_oracle_gate ... ignored, requires GLM 5.3 Q2, its vision sidecar, a fixed image, and Apple Metal
+test engine::metal::glm::tests::glm53_reference_logits_replay_separates_sampling_from_execution ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm53_reference_prompt_tokens_match_shared_runtime ... ignored, installed GLM and DS4SERVER_GLM_REFERENCE_LOG from the standalone reference
+test engine::metal::glm::tests::glm53_two_row_verifier_restores_recurrent_state ... ignored, installed GLM 5.3 and Metal; run under test-supervisor
+test engine::metal::glm::tests::glm_byte_budget_reserves_prefill_before_dynamic_experts ... ok
+test engine::metal::glm::tests::glm_mtp_preserves_target_tokens_and_drafts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm_prefill_schedule_matches_ds4_boundaries ... ok
+test engine::metal::glm::tests::glm_resident_decode_flushes_periodically_without_a_final_or_ssd_flush ... ok
+test engine::metal::glm::tests::repeated_glm_prompt_rewinds_one_token_for_logits ... ok
+test engine::metal::glm::tests::resident_and_streamed_glm_match_ds4_decode_oracles ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
+test engine::metal::glm::tests::streamed_glm_uses_ds4_indexed_prefill_for_long_prompts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
+test engine::metal::gpu::qwen_mtplx_allocation_tests::backend_allocation_policy_is_explicit_inside_and_outside_qwen_scope ... ignored, requires Apple Metal; allocation only, no GPU dispatch or model
+test engine::metal::gpu::qwen_mtplx_allocation_tests::dispatch_retains_allocation_without_changing_data_donation ... ignored, requires Apple Metal, no model weights
+test engine::metal::gpu::qwen_mtplx_allocation_tests::empty_storage_preserves_data_identity_views_and_zero_allocator_usage ... ignored, requires Apple Metal context; empty descriptors, no GPU allocation
+test engine::metal::gpu::qwen_mtplx_allocation_tests::mtplx_tensor_cpu_access_releases_native_storage ... ignored, requires Apple Metal; CPU buffer access only, no GPU dispatch or model
+test engine::metal::markov::tests::installed_markov_worker_dispatch ... ignored, requires installed 0731 DSpark support; CPU dispatch diagnostic, not parity acceptance
+test engine::metal::markov::tests::markov_workers_reuse_threads_and_buffers_with_ordered_ties ... ok
+test engine::metal::power_throttle_keeps_ds4_ewma_and_full_power_bypass ... ok
+test dev_brain::tests::skill_index_migration_does_not_overwrite_an_unrelated_root_note ... ok
+test engine::metal::qwen::tests::affine_split_k_matches_reference_shapes_and_scratch_bound ... ok
+test engine::metal::qwen::tests::dense_attention_crossover_is_phase_specific ... ok
+test engine::metal::qwen::tests::qsa_selection_and_capacity_cover_the_full_native_window ... ok
+test engine::metal::qwen::tests::qwen_affine_qmv_and_qmm_match_mlx_reference ... ignored, requires the pinned Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_affine_qmv_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_core_boundary_and_checkpoint_are_stable ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_fused_gdn_input_rows_match_separate_projections ... ignored, requires the pinned Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_gdn_conv_norm_rows_matches_mtplx ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_gdn_delta_rows_matches_mtplx ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_lazy_target_probabilities_match_batched_sampling ... ignored, requires the pinned Qwen metadata and Apple Metal; no model inference
+test engine::metal::qwen::tests::qwen_m5_performance_gate ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_metal_primitives_match_reference_vectors ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_mtp_matches_target_and_restores_its_cache ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_ple_gather_copies_only_the_requested_row ... ok
+test engine::kvstore::tests::opening_reclaims_checkpoints_the_index_cannot_reach ... ok
+test engine::metal::qwen::tests::qwen_qsa_dense_mask_accepts_unsorted_blocks ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_full_context_preallocation_succeeds ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_logical_depth_oracles_cover_prefill_decode_and_state ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_real_layer_executes_the_first_sparse_token ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_selection_and_attention_remain_bounded_at_native_depths ... ignored, requires Apple Metal and allocates the full QSA score window
+test engine::metal::qwen::tests::qwen_reference_layer_trace ... ignored, diagnostic trace against the pinned MLX-LM reference
+test engine::metal::qwen::tests::qwen_reference_sdpa_trace ... ignored, diagnostic comparison with captured MTPLX SDPA inputs
+test engine::metal::qwen::tests::qwen_reference_stateful_trace ... ignored, diagnostic trace against the pinned MLX-LM reference
+test engine::metal::qwen::tests::qwen_reference_verify_trace ... ignored, diagnostic trace against the pinned MTPLX verifier
+test engine::metal::qwen::tests::qwen_sorted_routed_prefill_matches_grouped_output ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_ple_hash_contract_matches_golden_boundaries ... ok
+test engine::metal::qwen::tests::qwen_stage_counters_preserve_async_batches ... ignored, requires pinned Qwen embeddings and Apple Metal; stage profiler opt-in
+test engine::metal::qwen::tests::qwen_target_submission_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_verify_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen_mtplx::allocator::mtplx_allocations_preserve_logical_bounds_with_reference_rounding ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_heap_limits_and_gpu_lifetime ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_replays_reference_allocations ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::ar::mtplx_ar_driver_matches_reference ... ignored, requires Apple Metal; unchanged MTPLX generate_ar orchestration oracle
+test engine::metal::qwen_mtplx::array::mtplx_array_detach_clears_every_siblings_inputs_and_primitive ... ok
+test engine::kvstore::tests::budget_evicts_the_least_valuable_unprotected_entry ... ok
+test engine::metal::qwen_mtplx::array::mtplx_array_layout_preserves_dtype_shape_stride_and_materialization_flags ... ok
+test engine::metal::qwen_mtplx::array::mtplx_array_materialization_keeps_geometry_and_rejects_short_storage ... ignored, requires Apple Metal; typed materialized descriptors without model weights
+test engine::metal::qwen_mtplx::array::mtplx_dtype_promotion_matches_all_reference_pairs ... ok
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_dense_mask_graph_matches_reference ... ignored, requires Apple Metal; original MTPLX block mask including sentinel and tails
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_eager_array_graph_matches_reference ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_prefill_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX tiled prefill gather
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_rows_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX QSA rows-gather attention
+test engine::metal::qwen_mtplx::attention::mtplx_sdpa_fallback_graph_matches_reference ... ignored, requires Apple M5 Max; actual Qwen SDPA fallback reference
+test engine::metal::qwen_mtplx::binary::mtplx_graph_binary_preserves_reference_branches ... ignored, requires Apple Metal; binary graph broadcast/layout and donation branches
+test engine::metal::qwen_mtplx::binding::mtplx_installed_module_binding_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS and SSD policy; only small norms and four table rows evaluated
+test engine::metal::qwen_mtplx::cache::mtplx_kv_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX KVCache growth, trim and state receipts
+test engine::metal::qwen_mtplx::cache::mtplx_qsa_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX positional cache, mirror and held state aliases
+test engine::kvstore::tests::longest_compatible_prefix_wins_and_hits_survive_reopen ... ok
+test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_scalar_bits_and_custom_kernel_identity ... ignored, requires Metal buffers and external supervisor; scalar ownership only, no GPU dispatch
+test database::tests::session_model_migration_preserves_data_and_accepts_catalog ... ok
+test engine::metal::qwen_mtplx::allocator::mtplx_allocator_cache_matches_live_reference_trace ... ok
+test engine::metal::qwen_mtplx::context_copy::prompt_copy_match_and_backoff_preserve_reference_policy ... ok
+test engine::metal::qwen_mtplx::decoder::mtplx_decoder_connected_attention_and_moe_advance_caches ... ignored, requires Apple Metal and external test-supervisor; complete decoder/cache smoke
+test engine::metal::qwen_mtplx::dense::mtplx_dense_attention_batches_match_reference ... ignored, requires Apple M5 Max; actual BF16 batched matmul reference
+test engine::metal::qwen_mtplx::dense::mtplx_dense_float_graph_matches_reference ... ignored, requires Apple M5 Max; floating Matmul factory and original dispatch
+test engine::metal::qwen_mtplx::dense::mtplx_qsa_score_array_graph_matches_reference ... ignored, requires Apple M5 Max; canonical QSA F32 graph with the original TF32 setting
+test engine::metal::qwen_mtplx::compiled::mtplx_replay_replaces_descriptors_without_rebuilding_primitives ... ok
+test engine::metal::qwen_mtplx::encoder::mtplx_empty_bindings_preserve_nil_dependencies_without_dummy_storage ... ignored, requires Apple Metal; empty original-kernel grids and nil bindings
+test engine::metal::qwen_mtplx::decoder::mtplx_decoder_residual_preserves_graph_order ... ok
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_commits_after_primitives_and_balances_completion ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_scope_routes_operators_and_allocator ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_shared_events_and_error_completion_order ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_untracked_dependencies_across_encoders_and_commands ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_drains_failure_preserves_cause_and_retries_only_failed_nodes ... ignored, requires Apple Metal; supervised evaluator failure and retry
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_releases_intermediates_and_preserves_descriptor_donation ... ignored, requires Apple Metal; graph ownership and last-consumer release
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_retains_tracer_topology_without_input_donation ... ignored, requires Apple Metal; evaluator tracer donation and retained topology
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_runs_cpu_gpu_siblings_and_async_continuations ... ignored, requires Apple Metal; tiny supervised CPU/GPU evaluator graph
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_barrier_epochs_and_commit_thresholds ... ok
+test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_tracks_stream_fences_per_leaf_and_cross_device_union ... ok
+test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_bfs_width_and_sibling_choice ... ok
+test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_scheduled_detach_and_tracer_errors ... ok
+test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ar_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_AR_REFERENCE required
+test engine::metal::qwen_mtplx::execution::mtplx_full_installed_mtp_prefill_matches_reference ... ignored, full installed MTP model; external guard and DS4_QWEN38_MTP_PREFILL_REFERENCE required
+test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ping_forward_matches_reference ... ignored, full installed model; requires external memory guard and DS4_QWEN38_FORWARD_REFERENCE
+test engine::metal::qwen_mtplx::execution::qwen_power_throttle_drains_before_sleep_and_bypasses_at_full_power ... ignored, requires Apple Metal; run under test-supervisor
+test engine::metal::qwen_mtplx::fused::mtplx_compiled_constant_spelling ... ok
+test engine::metal::qwen_mtplx::fused::mtplx_compiled_fusion_gpu_matches_reference ... ignored, requires Metal and external supervisor; generated fusion vs pinned SiLU plus multi-output/scalar dispatch
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_forward_matches_model_reference ... ignored, requires Apple Metal; actual MTPLX GDN forward and cache receipts
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_prefill_and_verify_match_model_reference ... ignored, requires Apple Metal; complete GDN Conv1d prefill and fused verify graphs
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_staged_primitives_match_reference ... ignored, requires Apple Metal; 80 actual delta-update receipts and six conv-normalization kernels
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_step_preserves_views_and_siblings ... ignored, requires Apple Metal; GDN custom primitive strided copies and sibling roots
+test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_event_snapshots_preserve_values_streams_and_shared_errors ... ignored, requires Apple Metal events, no model weights
+test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fence_coherence_preserves_empty_and_partial_word_spans ... ignored, requires Apple Metal3, model-free empty coherent dispatch
+test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fences_transfer_cpu_gpu_gpu_cpu_in_both_modes ... ignored, requires Apple Metal3, tiny model-free coherent fence kernels
+test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_forward_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX GatedResidual reference receipts
+test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_pack_and_reads_match_reference ... ignored, requires Apple Metal; original hyper primitives and runtime pack
+test engine::metal::qwen_mtplx::indexing::mtplx_graph_indexing_preserves_layouts_and_offsets ... ignored, requires Apple Metal; Select layout branches, GatherAxis views and empty arguments
+test engine::metal::qwen_mtplx::indexing::mtplx_scatter_axis_graph_matches_reference ... ignored, requires Apple Metal; pinned Bool ScatterAxis layout and broadcast paths
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_numeric_receipts ... ignored, #103 diagnostic; Metal required, eager versus compiled numeric operands
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_sampling_receipts ... ignored, #103 diagnostic; Metal required, pins currently broken sampling law and invalid rows
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_sorted_gather_receipts ... ignored, #103 diagnostic; M5 and pinned runtime shader required; initialized sentinels
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_tokenizer_receipts ... ignored, #103 diagnostic; installed tokenizer and latest oracle JSONL; no GPU
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_vision_position_receipts ... ignored, #103 diagnostic; small active attention module on Metal; mixed position boundary
+test engine::metal::qwen_mtplx::issue104::issue104_installed_ar_stream_and_state ... ignored, full pinned Bare-Speed model; external memory guard required
+test engine::metal::qwen_mtplx::issue105::issue105_batched_greedy_rejects_each_invalid_row ... ignored, Metal required; run serially under test-supervisor
+test engine::metal::qwen_mtplx::issue106::issue106_complete_state_and_checkpoint_recovery ... ignored, installed model and actual pre-fix checkpoint; external memory guard required
+test engine::metal::qwen_mtplx::issue106::issue106_upstream_exit_matrix ... ignored, installed model, scripted logits on real caches; external memory guard required
+test engine::metal::qwen_mtplx::issue108::issue108_attention_chunk_positions ... ignored, Metal and pinned MTPLX chunk fixtures; external supervisor required
+test engine::metal::qwen_mtplx::issue109::issue109_gate_phase_and_dtype_laws ... ignored, Metal and exhaustive pinned gate fixtures; external supervisor required
+test engine::metal::qwen_mtplx::issue110::issue110_actual_projection_rows_match_oracle ... ignored, installed layer weights and pinned oracle receipts; external memory supervisor required
+test engine::metal::qwen_mtplx::issue110::issue110_guard_allocation_refusal_releases_graph_and_retries ... ignored, Metal allocation refusal and retry; external supervisor required
+test engine::metal::qwen_mtplx::issue110::issue110_large_cold_and_warm_prefill ... ignored, full installed model and large partial chunks; serial memory supervisor required
+test engine::metal::qwen_mtplx::issue112_bench::issue112_hidden_prefill ... ignored, full installed model hidden-state prefill; serial supervisor required
+test engine::metal::qwen_mtplx::issue112_bench::issue112_paired_prefill ... ignored, full installed model; serial memory supervisor and explicit output directory
+test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_dequantize_matches_reference ... ignored, requires Apple Metal; original table dequantize dtype/group/bit/layout receipts
+test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_quantize_matches_reference ... ignored, requires Apple Metal; original affine quantize dtype/group/bit/layout receipts
+test engine::metal::qwen_mtplx::linear::mtplx_graph_gather_qmm_matches_reference ... ignored, requires Apple Metal; original GatherQMM array layouts and expert routing
+test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_batched_identity ... ignored, requires Apple Metal and external supervisor; strided/broadcast batched affine dispatch
+test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_matches_reference ... ignored, requires Apple M5 Max Metal; 78 pinned affine fixtures through lazy graph
+test engine::metal::qwen_mtplx::linear::sorted_gather_guard_matches_pinned_call_predicates ... ok
+test engine::metal::qwen_mtplx::load::mtplx_safetensors_load_laziness_boundaries_and_lifetime ... ignored, requires Apple Metal; bounded CPU Load and dependent GPU graph
+test engine::metal::qwen_mtplx::mlp::mtplx_graph_shared_mlp_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX shared MLP reference receipts
+test engine::metal::qwen_mtplx::mlp::mtplx_graph_split_and_compiled_silu_layouts ... ignored, requires Apple Metal; Split sibling ownership and compiled SiLU layouts
+test engine::metal::qwen_mtplx::model::mtplx_model_compile_policy_and_run_boundaries ... ok
+test engine::metal::qwen_mtplx::moe::mtplx_graph_sparse_moe_matches_reference ... ignored, requires Apple M5 Max; full MoE Array graph, no model weights; use test-supervisor
+test engine::metal::qwen_mtplx::moe_sorted::issue112_all_kernel_variants ... ignored, Metal synthetic all supported row-map variants; external supervisor
+test engine::metal::qwen_mtplx::moe_sorted::issue112_canary_failures ... ignored, Metal canary rejection and sticky fallback; external supervisor
+test engine::metal::qwen_mtplx::moe_sorted::issue112_full_model_state ... ignored, full installed model; validates all cold/warm state receipts against the pre-port oracle
+test engine::metal::qwen_mtplx::moe_sorted::issue112_installed_gate_up ... ignored, installed projection weights; external supervisor required
+test engine::metal::qwen_mtplx::moe_sorted::row_map_applicability ... ok
+test database::tests::migrations_and_crud_work ... ok
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_embedding_matches_reference ... ignored, requires Apple Metal; full resident NGram graph and lazy decode history
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_gpu_rows_and_history_match_reference ... ignored, requires Apple Metal; original lazy NGram GPU row graph, no table
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_resident_matches_reference ... ignored, requires Apple Metal; original NGramTable resident gather/dequantize receipts
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_table_priority ... ignored, requires Apple Metal; NGram table routing and missing-sidecar behavior
+test engine::metal::qwen_mtplx::ngram_sidecar::mtplx_ngram_sidecar_matches_reference ... ignored, requires Apple Metal; actual file-backed MTPLX sidecar/LRU/dequantization receipts
+test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_allclose_branches ... ignored, requires Apple Metal; stage verification special values and original reduction branches
+test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_matches_reference ... ignored, requires Apple Metal; original complete stage lifecycle and file-backed tables
+test engine::metal::qwen_mtplx::normalization::mtplx_graph_norm_modules_match_reference ... ignored, requires Apple Metal; connected Qwen normalization graph golden fixtures
+test engine::metal::qwen_mtplx::normalization::mtplx_graph_rms_norm_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph RMSNorm donation, copy, broadcast and looped kernels
+test engine::metal::qwen_mtplx::normalization::mtplx_logsumexp_graph_matches_reference ... ignored, requires Apple Metal; original sampler LogSumExp branches
+test engine::metal::qwen_mtplx::normalization::mtplx_softmax_graph_storage_matches_reference ... ignored, requires Apple Metal; original Softmax type and storage branches
+test engine::metal::qwen_mtplx::ops::mtplx_concatenate_preserves_general_general_copy_semantics ... ignored, requires Apple Metal; Concatenate promotion, offsets, strided writers and validation
+test engine::metal::qwen_mtplx::ops::mtplx_owned_operations_build_and_execute_qsa_layout_graph ... ignored, requires Apple Metal; self-contained lazy QSA-layout recipe, no callback map
+test engine::metal::qwen_mtplx::ops::mtplx_slice_and_contiguous_preserve_cache_window_semantics ... ignored, requires Apple Metal; normalized Slice and Contiguous cache-window graph
+test engine::metal::qwen_mtplx::owner::issue105_product::issue105_product_ar_cleanup_and_http_retry ... ignored, installed Qwen model; serial test-supervisor with memory guard required
+test engine::metal::qwen_mtplx::owner::issue105_product::issue105_product_mtp_cleanup_and_http_retry ... ignored, installed Qwen MTP model; serial test-supervisor with memory guard required
+test engine::metal::qwen_mtplx::owner::issue106_product::issue106_product_discards_old_checkpoint_and_recovers_cold ... ignored, installed model and actual pre-fix checkpoint; external memory guard required
+test engine::metal::qwen_mtplx::owner::issue107_product::issue107_vision_product ... ignored, installed model plus vision tower; serial external memory supervisor required
+test engine::metal::qwen_mtplx::owner::issue109_product::issue109_verify_rows_and_warm_checkpoint ... ignored, installed model; serial external memory supervisor required
+test engine::metal::qwen_mtplx::fused::mtplx_compile_fusion_preserves_parent_and_output_boundaries ... ok
+test engine::metal::qwen_mtplx::owner::issue109_product::issue113_default_verify_and_checkpoint ... ignored, installed model and actual pre-head-chunk checkpoint; serial external memory supervisor
+test engine::metal::qwen_mtplx::owner::issue111_product::issue111_token_prefix_cache ... ignored, installed model; serial external memory supervisor required
+test engine::metal::qwen_mtplx::owner::qwen_product_generator_chat_matches_reference ... ignored, full installed model through product Generator; external supervisor and chat reference required
+test engine::metal::qwen_mtplx::owner::qwen_vision_chat_checkpoint_preserves_image_identity ... ignored, requires installed Qwen, Metal, and DS4_QWEN_VISION_IMAGE; run under test-supervisor
+test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_connected_matches_reference ... ignored, requires Apple M5 Max; complete original PLELayer with real lookup
+test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_projection_and_cache_match_reference ... ignored, requires Apple M5 Max; actual MTPLX PLELayer after embedding lookup
+test engine::metal::qwen_mtplx::policy::qwen_server_policy_matches_original_functions ... ignored, requires DS4_QWEN38_POLICY_REFERENCE from pinned MTPLX; CPU-only
+test engine::metal::qwen_mtplx::owner::qwen_mtp_progress_publishes_without_turn_completion_or_double_counting ... ok
+test engine::metal::qwen_mtplx::policy::sorted_moe_switch_and_family_rehearsal ... ok
+test engine::metal::qwen_mtplx::fused::mtplx_compiled_emitter_matches_all_pinned_silu_bodies ... ok
+test engine::metal::qwen_mtplx::prefill::mtplx_mtp_prefill_driver_matches_reference ... ignored, requires Apple Metal; original streaming MTP history prefill
+test engine::metal::qwen_mtplx::prefill::mtplx_prefill_driver_matches_reference ... ignored, requires Apple Metal; original _prefill control flow and raw cache roots
+test engine::metal::qwen_mtplx::qsa_attention::mtplx_qsa_attention_module_matches_reference ... ignored, requires Metal and unchanged whole-Attention receipts; no model download
+test engine::metal::qwen_mtplx::qsa_compile::mtplx_compiled_indexer_array_replay_matches_reference ... ignored, requires Metal and original compiled indexer receipts; no model weights
+test engine::metal::qwen_mtplx::qsa_core::mtplx_qsa_core_host_matches_reference ... ignored, requires Metal and original host-core receipts; no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_cache_donation_requires_descriptor_data_and_allocation_contract ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_completion_owns_buffers_until_gpu_finishes ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_evaluator_retains_inputs_and_siblings_not_primary_data ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::qsa_indexer::issue107::issue107_real_geometry_query_and_block_start_positions ... ignored, Metal and independently verified pinned indexer fixtures; external supervisor required
+test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_canonical_indexer_routes_match_reference ... ignored, requires Metal and original ongoing indexer routing receipts; no weights
+test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_compiled_commit_matches_reference ... ignored, requires Metal and original indexer lifecycle receipts; no model weights
+test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_prefix_matches_reference ... ignored, requires Apple Metal, original stateful QSAIndexer receipts, no model weights
+test engine::metal::qwen_mtplx::qsa_prefill::mtplx_qsa_prefill_graph_matches_reference ... ignored, requires Metal and original two-stage prefill receipts, no weights
+test engine::metal::qwen_mtplx::qsa_prepare::mtplx_qsa_prepare_graph_matches_reference ... ignored, requires Apple Metal; parameterized original QSA preparation and traced positions
+test engine::metal::qwen_mtplx::qsa_select::mtplx_qsa_fused_graph_matches_reference ... ignored, requires Apple Metal and original QSA selector receipts; no model weights
+test engine::metal::qwen_mtplx::random::mtplx_sampler_prefix_matches_reference ... ignored, requires Apple Metal; original target-prefix sampler and default RNG
+test engine::metal::qwen_mtplx::random::mtplx_sampler_resident_matches_reference ... ignored, requires Apple Metal; original MTPLX resident sampling and keyed RNG
+test engine::metal::qwen_mtplx::qsa_graph::qsa_schedule_preserves_the_selected_sibling ... ok
+test engine::metal::qwen_mtplx::reduce::mtplx_sampler_argmax_matches_reference ... ignored, requires Apple Metal; original MTPLX greedy ArgReduce and storage
+test engine::metal::qwen_mtplx::policy::retired_ar_pipeline_flag_cannot_change_server_or_turn_policy ... ok
+test engine::metal::qwen_mtplx::reduce::mtplx_sum_graph_matches_reference ... ignored, requires Apple Metal; original shared sum dispatch and QSA mean
+test engine::metal::qwen_mtplx::residency::mtplx_residency_matches_reference_lifecycle_cases ... ignored, requires Apple Metal 3 / macOS 15, no model weights
+test engine::metal::qwen_mtplx::rope::mtplx_rope_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX RoPE, static YaRN and multi-axis receipts
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_batched_matches_reference ... ignored, requires Apple Metal; original MTPLX bound and serial batch distributions
+test engine::metal::qwen_mtplx::request::qwen_request_settings_override_backend_policy ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_entry_matches_reference ... ignored, requires Apple Metal; actual MTPLX generator sampling entry
+test engine::metal::qwen_mtplx::request::qwen_request_progress_uses_ui_frontiers_and_decode_timer ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_fallback_matches_reference ... ignored, requires Apple Metal; original sparse fallback and host dense sampler
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_penalties_match_reference ... ignored, requires Apple Metal; original GPU and CPU raw-logit penalties
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_selection_matches_reference ... ignored, requires Apple Metal; original MTPLX sampler selection, no model weights
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_serial_support_matches_reference ... ignored, requires Apple Metal; actual MTPLX serial candidate route, no weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_dispatch_counts_tenth_and_completes_after_failed_work ... ok
+test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_primitive_holds_data_and_temporaries_in_fifo_order ... ignored, requires Apple Metal shared storage, no GPU dispatch or model weights
+test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_simplify_preserves_reference_parent_rules ... ok
+test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_cpu_events_propagate_errors_without_consuming_source ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_streams_preserve_fifo_errors_and_shutdown ... ok
+test engine::metal::qwen_mtplx::sdpa::mtplx_sdpa_vector_matches_reference ... ignored, requires Apple M5 Max; actual pinned fused SDPA kernels
+test engine::metal::qwen_mtplx::session_cache::mtplx_full_installed_chat_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_CHAT_REFERENCE required
+test engine::metal::profile::tests::profile_reports_locality_and_lru_hits ... ok
+test engine::metal::qwen_mtplx::session_cache::mtplx_session_boundary_policy ... ok
+test engine::metal::qwen_mtplx::session_checkpoint::qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state ... ignored, model-free Metal checkpoint; external supervisor and DS4_QWEN38_CHECKPOINT_DIR required
+test engine::metal::qwen_mtplx::session_cache::mtplx_session_final_state_requires_capture_and_safe_cache ... ok
+test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_host_import_matches_reference ... ignored, model-free Metal imports; external supervisor and DS4_HOST_IMPORT_REFERENCE required
+test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_matches_original_blobs_and_restore ... ignored, tiny GPU arrays; external supervisor and DS4_SESSION_CODEC_REFERENCE required
+test engine::metal::qwen_mtplx::sort::mtplx_sort_graph_matches_reference ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::sparse_attention::mtplx_sparse_attention_graph_matches_reference ... ignored, requires Apple M5; actual MTPLX sparse Decode and TensorOps Prefill consumers
+test engine::metal::qwen_mtplx::speculative::mtplx_full_installed_mtp_generation_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_MTP_GENERATION_REFERENCE required
+test engine::metal::qwen_mtplx::stream::mtplx_stream_routes_queues_events_and_finalizes_all_under_pressure ... ignored, requires Apple Metal, tiny model-free two-queue copy
+test engine::metal::qwen_mtplx::stream::mtplx_stream_registry_preserves_thread_defaults_templates_and_clear ... ok
+test database::tests::chat_and_compaction_history_survive_reopen_and_session_deletion ... ok
+test engine::metal::qwen_mtplx::submission::mtplx_submission_attaches_at_flush_eval_finish_and_drop ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::unary::mtplx_graph_unary_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph unary layout, donation and kernel boundaries
+test engine::metal::qwen_mtplx::verify_sdpa::issue113_head_chunk_outputs_and_dispatch ... ignored, Metal, pinned head-chunk numerical and dispatch fixtures; serial supervisor
+test engine::metal::qwen_mtplx::views::mtplx_dtype_conversion_and_views_preserve_values_layout_and_donation ... ignored, requires Apple Metal; original conversion/view kernels and donation in lazy eval
+test engine::metal::qwen_mtplx::views::mtplx_dynamic_slice_graph_matches_reference ... ignored, requires Metal and original dynamic slice receipts; no model weights
+test engine::metal::qwen_mtplx::views::mtplx_reshape_shares_or_copies_in_the_connected_evaluator ... ignored, requires Apple Metal; original Reshape copy kernels and shared backing
+test engine::metal::qwen_mtplx::views::mtplx_views_preserve_aliases_offsets_strides_and_gpu_output ... ignored, requires Apple Metal; connected zero-copy views and offset-aware original copy kernel
+test engine::metal::qwen_mtplx::vision::qwen_vision_tower_matches_mtplx_image ... ignored, requires Metal and pinned Qwen vision artifacts/reference values
+test engine::metal::qwen_mtplx::weights::mtplx_checkpoint_projection_fusions_match_reference ... ignored, requires Apple Metal; bounded lazy checkpoint Load and fusion
+test engine::metal::qwen_mtplx::weights::mtplx_installed_checkpoint_fusion_metadata_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS; headers only, never evaluate weights
+test engine::metal::qwen_mtplx_tests::mtplx_affine_selection_preserves_reference_boundaries ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_canonical_hyper_and_gdn_kernels_match_reference ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_canonical_source_bodies_preserve_pinned_hashes ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_connected_delta_update_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gather_sort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_decode_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_forward_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_staged_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_norm_modules_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_router_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_sparse_moe_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_switch_glu_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_gate_up_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_cache_lifecycle_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_backings_and_dynamic_copies_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_routing_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_window_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_capacity_matches_reference ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_indexer_modes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_scalar_sources_match_reference ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_connected_cache_selection_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_eager_topk_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_f32_scores_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_preparation_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_routes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_producer_dispatch_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx::verify_sdpa::issue113_plan_and_policy_match_pinned_oracle ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_selector_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_affine_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_argsort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_gather_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_activity_preserves_reference_pressure_and_wait_rules ... ok
+test engine::metal::tests::batch_raw_span_includes_the_chunk_and_previous_window ... ok
+test engine::metal::tests::compression_schedule_tracks_the_deepseek_model_shape ... ok
+test engine::metal::tests::decode_feature_policy_keeps_device_generations_and_rollbacks_separate ... ok
+test engine::metal::tests::decode_raw_span_tracks_the_logical_sliding_window ... ok
+test engine::metal::tests::directional_steering_matches_the_ds4_token_oracle ... ignored, requires the installed Flash GGUF, DS4 steering fixture, and a Metal device
+test engine::metal::tests::dspark_confidence_quantizes_q8_input_like_ds4 ... ok
+test engine::metal::tests::dspark_markov_activation_matches_q8_rounding ... ok
+test engine::metal::tests::dspark_matches_original_summary_cycles ... ignored, requires installed 0731/DSpark and DS4SERVER_DSPARK_REFERENCE cycle recording
+test engine::metal::tests::dspark_runs_a_target_owned_greedy_cycle ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures
+test engine::metal::tests::dspark_scheduler_matches_the_ds4_default_window ... ok
+test engine::metal::tests::dspark_verifier_selects_only_the_committed_row ... ok
+test engine::metal::tests::flash_0731_long_context_crosses_indexed_prefill_boundary ... ignored, requires a 0731 Flash GGUF and an Apple M5 device
+test engine::metal::tests::flash_0731_m5_decode_performance_gate ... ignored, requires a 0731 Flash GGUF and an Apple M5 device
+test engine::metal::tests::flash_0731_resident_and_ssd_streaming_choose_the_same_tokens ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
+test engine::metal::tests::flash_0731_runs_exact_sampled_dspark ... ignored, requires the installed 0731 target and checkpoint-specific DSpark GGUF fixtures
+test engine::metal::tests::flash_0731_ssd_streaming_maps_and_seeds_batched_prefill_layers ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
+test engine::metal::tests::generated_hotlists_match_each_model_shape ... ok
+test engine::metal::tests::half_conversion_handles_normal_and_subnormal_values ... ok
+test engine::metal::tests::pro_q4_model_spans_remain_isolated ... ok
+test engine::metal::tests::pro_resident_admission_includes_weights_context_and_scratch ... ok
+test engine::metal::tests::pro_ssd_streaming_full_layer_and_selected_batch_agree ... ignored, requires the installed DeepSeek V4 Pro GGUF and Apple Metal
+test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
+test engine::metal::tests::ssd_streaming_supports_dspark ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures
+test engine::metal::tests::streaming_memory_plan_matches_ds4_graph_formulas ... ok
+test engine::metal::vision::tests::glm53_resize_matches_reference_token_grids ... ok
+test engine::metal::qwen_mtplx::submission::mtplx_all_native_command_commits_pass_the_submission_hook ... ok
+test engine::qwen::tests::metadata_and_safetensors_boundaries_fail_closed ... ok
+test engine::qwen::tests::pinned_artifact_set_loads_and_renders_goldens ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
+test engine::qwen::tests::pinned_qwen_metadata_matches_product_without_executor_state ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
+test engine::qwen_vision::qwen_image_cache_keys_distinguish_pixels_and_preserve_prior_text ... ok
+test engine::qwen::tests::frozen_inventory_and_memory_categories_are_exact ... ok
+test engine::qwen_vision::qwen_vision_positions_and_resize_contract ... ok
+test engine::sampling_tests::checkpoint_headers_identify_the_exact_model ... ok
+test engine::sampling_tests::checkpoint_rebuilds_explain_compatibility_and_history_misses ... ok
+test engine::sampling_tests::checkpoint_tag_accepts_an_unchanged_prefix_before_reminders ... ok
+test engine::sampling_tests::checkpoint_tag_covers_the_canonical_chat_state ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_distribution_and_rng_match_reference ... ok
+test engine::sampling_tests::ds4_sampler_real_logits_cpu_benchmark ... ignored, CPU-only; requires DS4_SAMPLER_RECORDING with the recorded GLM logits
+test engine::sampling_tests::exact_delta_sampling_accepts_or_corrects_the_draft ... ok
+test engine::sampling_tests::generated_text_drains_incomplete_bytes_before_phase_changes_and_finish ... ok
+test engine::sampling_tests::generated_text_preserves_thinking_utf8_and_split_stops ... ok
+test engine::sampling_tests::metal_executes_real_flash_token ... ignored, requires the 80 GiB Flash checkpoint and Apple Metal
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_workspace_matches_reference ... ok
+test engine::sampling_tests::qwen_rng_matches_numpy_pcg64 ... ok
+test engine::sampling_tests::qwen_sampling_uses_mtplx_global_nucleus_mass_before_top_k ... ok
+test engine::sampling_tests::resumed_prefill_crossover_matches_each_ds4_graph ... ok
+test engine::sampling_tests::sampling_probabilities_match_ds4_filter_order ... ok
+test engine::sampling_tests::split_utf8_token_bytes_are_joined_before_decoding ... ok
+test engine::sampling_tests::top_k_and_stream_stops_are_applied_before_output ... ok
+test engine::sampling_tests::workspace_instruction_identity_participates_in_append_only_kv_keys ... ok
+test engine::sampling_tests::zero_temperature_is_greedy ... ok
+test engine::tokenizer::issue111::issue111_ds4_families ... ignored, CPU-only; installed DS4 GGUFs and exact oracle IDs
+test engine::tokenizer::issue111::issue111_every_unicode_scalar_split ... ignored, CPU-only; exhaustive pinned Qwen split receipt
+test engine::tokenizer::issue111::issue111_qwen_tokens_and_requests ... ignored, CPU-only; installed Qwen tokenizer and pinned exact IDs
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_cpu_rows_and_raw_history_match_reference ... ok
+test engine::tokenizer::tests::character_helpers_are_total_at_every_byte_offset ... ok
+test engine::tokenizer::tests::ds4_chat_matches_original_session_tokens ... ignored, CPU-only; requires DS4SERVER_CHAT_REFERENCE and its installed GGUF
+test engine::tokenizer::tests::qwen_checkpoint_continuation_matches_full_conversation ... ignored, requires the installed Qwen tokenizer, but no GPU or model weights
+test engine::tokenizer::tests::qwen_plain_chat_matches_mtplx_server_tokens ... ignored, requires local tokenizer and tools/qwen-chat-reference.py output; no inference
+test engine::tokenizer::tests::reasoning_prefixes_match_each_model_family_contract ... ok
+test engine::validation::tests::configured_glm53_main_fixture_passes_the_exact_layout ... ok
+test engine::validation::tests::configured_glm53_vision_fixture_passes_the_exact_layout ... ok
+test dev_brain::tests::parser_and_index_rebuild_ignore_links_in_code ... ok
+test engine::validation::tests::installed_dspark_fixture_passes_the_target_layout ... ok
+test engine::validation::tests::mxfp4_is_limited_to_routed_experts ... ok
+test engine::tokenizer::issue111::qwen_split_boundaries ... ok
+test extensions::tests::duplicate_skill_names_and_update_rollback_are_deterministic ... ok
+test dev_brain::tests::validation_names_every_drifted_source_and_requires_semantic_refresh ... ok
+test extensions::tests::hook_output_is_bounded_typed_and_event_matched ... ok
+test engine::sampling_tests::probability_ratio_speculation_preserves_the_target_distribution ... ok
+test extensions::tests::manifest_and_paths_are_validated_without_following_escapes ... ok
+test extensions::tests::pinned_ponytail_runs_all_lifecycle_modes ... ignored, downloads the pinned Ponytail reference integration
+test extensions::tests::registry_persists_enable_disable_errors_and_uninstall ... ok
+test extensions::tests::symlink_and_variable_escapes_are_rejected ... ok
+test instructions::tests::changes_removals_resume_and_compaction_are_append_only ... ok
+test instructions::tests::discovery_is_global_to_local_deduplicated_and_confined ... ok
+test instructions::tests::empty_and_repaired_baselines_do_not_repeat_prompt_content ... ok
+test engine::metal::qwen_mtplx::array::mtplx_array_releases_sibling_cycles_and_deep_input_graphs ... ok
+test instructions::tests::newly_crossed_scope_is_added_once_and_unavailable_content_is_retained ... ok
+test metrics::tests::cache_report_splits_buckets_and_folds_index_files_into_their_checkpoint ... ok
+test metrics::tests::decode_timer_excludes_later_work_and_resets_between_requests ... ok
+test metrics::tests::prompt_timing_keeps_unmeasured_components_distinct_from_zero ... ok
+test metrics::tests::removed_preview_metric_code_stays_decodable ... ok
+test metrics::tests::snapshots_track_runtime_and_server_counters ... ok
+test model::qwen::qwen_vision_is_an_independent_managed_artifact_set ... ok
+test model::qwen::tests::embedded_manifest_is_the_frozen_complete_set ... ok
+test model::qwen::tests::verification_rejects_hash_mismatch_and_honors_cancellation ... ok
+test model::transfer::tests::cancellation_keeps_the_partial_file_for_the_next_run ... ok
+test model::transfer::tests::catalog_and_checksum_verification_are_explicit ... ok
+test model::transfer::tests::managed_artifact_inventory_and_delete_include_partial_files ... ok
+test model::transfer::tests::qwen_set_verifies_reloads_and_removes_as_one_managed_artifact ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
+test instructions::tests::frames_escape_control_tags_and_enforce_utf8_byte_limits ... ok
+test model::transfer::tests::restart_resumes_at_the_existing_partial_byte ... ok
+test model::transfer::tests::verification_reports_bytes_read ... ok
+test model_eval::tests::acceleration_receipt_reports_effective_sampling_contract ... ok
+test model_eval::tests::acceleration_switch_uses_each_models_native_path ... ok
+test model_eval::tests::canary_cross_phase_samples_keep_their_start_phase ... ok
+test dev_brain::tests::skill_validation_and_default_guide_restore_are_strict ... ok
+test model_eval::tests::evaluations_exclude_overlapping_workers_and_release_on_exit ... ok
+test model_eval::tests::external_canary_observes_phases_and_finishes_on_eof ... ignored, Metal probe only; DS4SERVER_CANARY_BINARY must point to the current release binary
+test model_eval::tests::external_canary_protocol_accepts_only_known_phases ... ok
+test model_eval::tests::parses_safe_defaults_and_overrides ... ok
+test model_eval::tests::rejects_unbounded_or_invalid_limits ... ok
+test model_eval::tests::repeated_prompts_form_one_ordered_conversation ... ok
+test model_eval::tests::warmup_timing_separates_loading_without_inventing_missing_durations ... ok
+test dev_brain::tests::source_hash_drift_excludes_page_from_authoritative_queries ... ok
+test model_eval::tests::worker_progress_events_advance_the_watchdog ... ok
+test native_edit::tests::edit_history_supports_undo_and_redo ... ok
+test native_edit::tests::paste_replays_a_complete_command_shortcut ... ok
+test native_menu::tests::edit_items_are_custom_menu_events ... ok
+test native_menu::tests::user_guide_uses_unmodified_f1 ... ok
+test native_media::tests::media_player_only_accepts_remote_http_urls ... ok
+test native_menu::tests::view_items_keep_the_selected_pane_checked ... ok
+test runtime::tests::dropping_active_generation_cancels_it ... ok
+test runtime::tests::operation_failures_use_the_matching_event ... ok
+test runtime::tests::operation_tracking_is_consistent ... ok
+test runtime::tests::runtime_panic_recovery_accepts_the_next_operation ... ok
+test server::request::tests::reasoning_requests_preserve_each_models_exact_modes ... ok
+test server::tests::connection_slots_enforce_the_limit_and_release_on_drop ... ok
+test server::tests::canonical_tool_replay_preserves_argument_types ... ok
+test server::tests::content_arrays_match_ds4_text_projection ... ok
+test server::tests::cors_headers_are_opt_in_and_cover_preflight ... ok
+test server::tests::dsml_tool_stream_projects_incremental_json_arguments ... ok
+test server::tests::context_limit_errors_use_protocol_standard_fields ... ok
+test server::tests::generated_dsml_becomes_openai_tool_calls ... ok
+test server::tests::model_metadata_uses_the_requested_alias_and_default_token_limit ... ok
+test server::tests::generated_tool_parser_accepts_ds4_recovery_syntaxes ... ok
+test server::tests::protocol_tool_results_require_live_or_replayed_call_ids ... ok
+test server::tests::responses_parser_rejects_dropped_state_and_preserves_hosted_calls ... ok
+test server::tests::sampled_tool_replay_survives_server_restart ... ok
+test server::tests::streaming_errors_match_ds4_server_framing ... ok
+test dev_brain::tests::validation_keeps_broken_links_editable_and_reports_provenance_errors ... ok
+test server::tests::tool_prompt_matches_ds4_server_text ... ok
+test server::tests::tool_stream_projects_ds4_recovery_syntaxes ... ok
+test settings::tests::captured_ds4_cli_options_are_uniquely_classified ... ok
+test settings::tests::effective_settings_build_one_engine_and_turn_configuration ... ok
+test settings::tests::effective_settings_preserve_unset_model_defaults ... ok
+test settings::tests::execution_settings_keep_ds4_automatic_values_and_glm_rules ... ok
+test settings::tests::kv_cache_gates_follow_the_ds4_store_rules ... ok
+test settings::tests::reasoning_selection_is_exact_and_model_specific ... ok
+test settings::tests::runtime_settings_preserve_automatic_values_and_cross_field_rules ... ok
+test settings::tests::speculative_settings_match_acceleration_defaults_and_dependencies ... ok
+test dev_brain::tests::verified_skills_are_indexed_and_injected_on_demand ... ok
+test server::tests::streaming_prefill_keeps_the_client_connection_alive ... ok
+test dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree ... ok
+test model_eval::tests::cpu_accounting_matches_getrusage_units ... ok
+test engine::validation::tests::installed_ds4_fixture_opens_and_renders_a_prompt ... ok
+test model::transfer::tests::stalled_download_times_out_and_keeps_the_partial_file ... ok
+test extensions::tests::direct_hook_runner_isolated_environment_and_handles_failures ... ok
+test server::tests::request_deadline_stops_slow_clients ... ok
+test engine::sampling_tests::ds4_sampler_matches_original_reference_tokens_and_rng ... ok
+test agent::tests::execution_failures_and_timeouts_have_distinct_codes ... ok
+kill: 95283: No such process
+test agent::tests::dropping_session_tools_stops_live_jobs_and_removes_output ... ok
+test engine::qwen_vision::qwen_image_preprocessing_matches_pillow_mtplx ... ok
+test agent::tests::local_file_and_bash_tools_execute_in_the_project ... ok
+test extensions::tests::lifecycle_matchers_preserve_declared_event_order ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_shaders_are_pinned ... ok
+test extensions::tests::hook_failures_are_bounded_actionable_and_non_shell ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_mixed_distributions_match_reference ... ok
+
+test result: ok. 313 passed; 0 failed; 236 ignored; 0 measured; 0 filtered out; finished in 5.62s
+
+     Running unittests tools/qwen38-artifacts.rs (target/debug/deps/qwen38_artifacts-1288d410762a1a77)
+
+running 4 tests
+test tests::artifact_names_cannot_escape_the_output_directory ... ok
+test tests::affine_sample_digest_uses_packed_values_scales_and_biases ... ok
+test tests::safetensors_validation_rejects_duplicate_names_and_invalid_ranges ... ok
+test tests::artifact_download_resumes_an_authenticated_partial ... ok
+
+test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
+
+     Running unittests tools/test-supervisor.rs (target/debug/deps/test_supervisor-df80f419c8443a12)
+
+running 4 tests
+test supervisor::tests::direct_worker_arguments_are_not_rust_test_arguments ... ok
+test supervisor::tests::split_records_cannot_be_interleaved_with_watchdog_samples ... ok
+test supervisor::tests::split_probe_records_do_not_reset_progress ... ok
+ready{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+ready
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+progressprogressprogressprogressprogressprogressprogressprogresstest supervisor::tests::watchdog_stops_memory_and_silence_but_not_total_runtime ... ok
+
+test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.30s
diff --git a/src/engine/metal/qwen_mtplx.rs b/src/engine/metal/qwen_mtplx.rs
index 6fc3439..038b85e 100644
--- a/src/engine/metal/qwen_mtplx.rs
+++ b/src/engine/metal/qwen_mtplx.rs
@@ -86,6 +86,7 @@ mod stream;
 #[cfg(test)]
 mod submission;
 mod unary;
+mod verify_sdpa;
 mod views;
 mod vision;
 mod vision_kernels;
diff --git a/src/engine/metal/qwen_mtplx/decoder.rs b/src/engine/metal/qwen_mtplx/decoder.rs
index 3275179..68e1cc8 100644
--- a/src/engine/metal/qwen_mtplx/decoder.rs
+++ b/src/engine/metal/qwen_mtplx/decoder.rs
@@ -376,6 +376,7 @@ fn mtplx_decoder_connected_attention_and_moe_advance_caches() {
         },
         qsa: qsa_attention::Options {
             verify_gate: false,
+            head_chunk: false, // Historical unchunked oracle fixture.
             indexer: QsaIndexerOptions {
                 fused: false,
                 compiled: false,
diff --git a/src/engine/metal/qwen_mtplx/execution.rs b/src/engine/metal/qwen_mtplx/execution.rs
index 10ae64c..5535674 100644
--- a/src/engine/metal/qwen_mtplx/execution.rs
+++ b/src/engine/metal/qwen_mtplx/execution.rs
@@ -186,6 +186,7 @@ impl Options {
             },
             qsa: qsa_attention::Options {
                 verify_gate: false,
+                head_chunk: super::policy::sdpa_head_chunk_enabled(&get),
                 indexer: QsaIndexerOptions {
                     fused: flag("MTPLX_FUSED_QSA_INDEXER"),
                     compiled: flag("MTPLX_COMPILED_QSA_INDEXER"),
diff --git a/src/engine/metal/qwen_mtplx/issue103_analysis.rs b/src/engine/metal/qwen_mtplx/issue103_analysis.rs
index 84ff050..876b791 100644
--- a/src/engine/metal/qwen_mtplx/issue103_analysis.rs
+++ b/src/engine/metal/qwen_mtplx/issue103_analysis.rs
@@ -391,6 +391,7 @@ fn issue103_vision_position_receipts() {
     .unwrap();
     let mut options = Options {
         verify_gate: false,
+        head_chunk: false, // Historical unchunked oracle fixture.
         indexer: super::QsaIndexerOptions {
             fused: false,
             compiled: false,
diff --git a/src/engine/metal/qwen_mtplx/issue106.rs b/src/engine/metal/qwen_mtplx/issue106.rs
index 9b90058..9a79c5b 100644
--- a/src/engine/metal/qwen_mtplx/issue106.rs
+++ b/src/engine/metal/qwen_mtplx/issue106.rs
@@ -166,6 +166,7 @@ fn issue106_complete_state_and_checkpoint_recovery() {
         context: 4096,
         vocabulary: metadata.tokenizer.vocab_size() as u32,
         mtp: true,
+        head_chunk: super::policy::sdpa_head_chunk_enabled(|key| std::env::var(key).ok()),
     };
     let mut live = LiveSession::default();
     assert_eq!(&std::fs::read(&old).unwrap()[..8], b"QWMTKV01");
diff --git a/src/engine/metal/qwen_mtplx/issue109_product.rs b/src/engine/metal/qwen_mtplx/issue109_product.rs
index 1edc64d..682d324 100644
--- a/src/engine/metal/qwen_mtplx/issue109_product.rs
+++ b/src/engine/metal/qwen_mtplx/issue109_product.rs
@@ -8,6 +8,16 @@ use crate::settings::{
 #[test]
 #[ignore = "installed model; serial external memory supervisor required"]
 fn issue109_verify_rows_and_warm_checkpoint() {
+    verify_rows_and_warm_checkpoint(true);
+}
+
+#[test]
+#[ignore = "installed model and actual pre-head-chunk checkpoint; serial external memory supervisor"]
+fn issue113_default_verify_and_checkpoint() {
+    verify_rows_and_warm_checkpoint(false);
+}
+
+fn verify_rows_and_warm_checkpoint(old_gate: bool) {
     assert_eq!(std::env::var("MTPLX_CONTEXT_COPY").as_deref(), Ok("0"));
     let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
     let directory = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE109_DIR").unwrap());
@@ -151,13 +161,25 @@ fn issue109_verify_rows_and_warm_checkpoint() {
     owner
         .save_checkpoint(&directory.join("new.bin"), [109; 32], &mut |_| {})
         .unwrap();
-    let old = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE109_OLD").unwrap());
-    assert!(
-        owner
-            .load_checkpoint(&old, &mut |_| {})
-            .unwrap_err()
-            .contains("obsolete MTP attention-gate")
+    let old = std::path::PathBuf::from(
+        std::env::var_os(if old_gate {
+            "DS4_ISSUE109_OLD"
+        } else {
+            "DS4_ISSUE113_OLD"
+        })
+        .unwrap(),
     );
+    let reject_old = old_gate || owner.identity.head_chunk;
+    let loaded = owner.load_checkpoint(&old, &mut |_| {});
+    if reject_old {
+        assert!(loaded.unwrap_err().contains(if old_gate {
+            "obsolete MTP attention-gate"
+        } else {
+            "SDPA head-chunk policy differs"
+        }));
+    } else {
+        assert!(loaded.unwrap());
+    }
     assert_eq!(original, state(&owner));
     let mut next = messages.clone();
     next.push(first.message.clone());
@@ -179,6 +201,6 @@ fn issue109_verify_rows_and_warm_checkpoint() {
     assert_eq!(expected, state(&owner));
     println!(
         "{}",
-        serde_json::json!({"case":"checkpoint","actual_old_rejected":true,"warm_reopened_state_exact":true})
+        serde_json::json!({"case":"checkpoint","actual_old_rejected":reject_old,"warm_reopened_state_exact":true})
     );
 }
diff --git a/src/engine/metal/qwen_mtplx/owner.rs b/src/engine/metal/qwen_mtplx/owner.rs
index a774d07..4e130f8 100644
--- a/src/engine/metal/qwen_mtplx/owner.rs
+++ b/src/engine/metal/qwen_mtplx/owner.rs
@@ -42,6 +42,7 @@ impl QwenExecutor {
             vocabulary: u32::try_from(metadata.tokenizer.vocab_size())
                 .map_err(|e| e.to_string())?,
             mtp: settings.speculative.glm_mtp,
+            head_chunk: super::policy::sdpa_head_chunk_enabled(|key| std::env::var(key).ok()),
         };
         let (execution, policy) = Execution::open(settings)?;
         Ok(Self {
diff --git a/src/engine/metal/qwen_mtplx/policy.rs b/src/engine/metal/qwen_mtplx/policy.rs
index 6d9fc58..5010a2b 100644
--- a/src/engine/metal/qwen_mtplx/policy.rs
+++ b/src/engine/metal/qwen_mtplx/policy.rs
@@ -25,6 +25,10 @@ fn falsey(value: Option<&str>) -> bool {
     })
 }
 
+pub(super) fn sdpa_head_chunk_enabled(get: impl Fn(&'static str) -> Option<String>) -> bool {
+    !falsey(get("MTPLX_QWEN4_VERIFY_SDPA_HEAD_CHUNK").as_deref())
+}
+
 pub(super) fn sorted_gather_enabled(get: impl Fn(&'static str) -> Option<String>) -> bool {
     !falsey(get("MTPLX_MOE_SORTED_GATHER_KERNEL").as_deref())
         && !truthy(get("MTPLX_FORCE_GPU_FAMILY_FALLBACK").as_deref())
diff --git a/src/engine/metal/qwen_mtplx/qsa_attention.rs b/src/engine/metal/qwen_mtplx/qsa_attention.rs
index 5cf34a8..e8b8100 100644
--- a/src/engine/metal/qwen_mtplx/qsa_attention.rs
+++ b/src/engine/metal/qwen_mtplx/qsa_attention.rs
@@ -33,6 +33,7 @@ pub(super) struct Attention<'a> {
 /// Resolved original request-phase/environment switches, not new heuristics.
 pub(super) struct Options {
     pub(super) verify_gate: bool,
+    pub(super) head_chunk: bool,
     pub(super) indexer: QsaIndexerOptions,
     pub(super) flash_min_context: u32,
     pub(super) prefill_gather: bool,
@@ -351,8 +352,15 @@ impl Attention<'_> {
             }
             _ => return Err("unconsumed QSA attention selection".into()),
         };
-        let out =
-            attention::scaled_dot_product(&q, &k, &v, mask.as_ref(), self.scale as f32, stream)?;
+        let out = super::verify_sdpa::call(
+            &q,
+            &k,
+            &v,
+            mask.as_ref(),
+            self.scale as f32,
+            options.head_chunk,
+            stream,
+        )?;
         self.finish(
             &ops::transpose(&out, &[0, 2, 1, 3], stream)?,
             &gate,
@@ -546,6 +554,7 @@ fn mtplx_qsa_attention_module_matches_reference() {
             };
             let options = Options {
                 verify_gate: false,
+                head_chunk: false, // Historical unchunked oracle fixture.
                 indexer: qsa_indexer_options(&first["options"]),
                 flash_min_context: first["flash_min"].as_u64().unwrap() as u32,
                 prefill_gather: first["prefill_gather"].as_bool().unwrap(),
diff --git a/src/engine/metal/qwen_mtplx/session_cache.rs b/src/engine/metal/qwen_mtplx/session_cache.rs
index f062c17..cdc9b29 100644
--- a/src/engine/metal/qwen_mtplx/session_cache.rs
+++ b/src/engine/metal/qwen_mtplx/session_cache.rs
@@ -532,6 +532,9 @@ fn mtplx_full_installed_chat_matches_reference() {
                     context: settings.context_tokens as u32,
                     vocabulary: metadata.tokenizer.vocab_size() as u32,
                     mtp,
+                    head_chunk: super::policy::sdpa_head_chunk_enabled(|key| {
+                        std::env::var(key).ok()
+                    }),
                 };
                 let key = crate::engine::conversation_key(
                     &settings.system_prompt,
diff --git a/src/engine/metal/qwen_mtplx/session_checkpoint.rs b/src/engine/metal/qwen_mtplx/session_checkpoint.rs
index c13c51e..82ba15b 100644
--- a/src/engine/metal/qwen_mtplx/session_checkpoint.rs
+++ b/src/engine/metal/qwen_mtplx/session_checkpoint.rs
@@ -23,6 +23,8 @@ pub(super) struct Identity {
     pub(super) context: u32,
     pub(super) vocabulary: u32,
     pub(super) mtp: bool,
+    #[serde(default)]
+    pub(super) head_chunk: bool,
 }
 
 #[derive(Serialize, Deserialize)]
@@ -182,7 +184,11 @@ impl LiveSession {
         file.read_exact(&mut raw).map_err(|e| e.to_string())?;
         let mut manifest: Manifest = serde_json::from_slice(&raw).map_err(|e| e.to_string())?;
         progress(HEADER_BYTES + raw.len() as u64);
-        if manifest.identity != identity {
+        if (Identity {
+            head_chunk: identity.head_chunk,
+            ..manifest.identity
+        }) != identity
+        {
             return Err(
                 "Qwen checkpoint does not match the current model/context/MTP policy".into(),
             );
@@ -196,6 +202,14 @@ impl LiveSession {
                 "Qwen checkpoint uses obsolete MTP attention-gate math; rebuild the cache".into(),
             );
         }
+        // The route also changes short dense prefills, not just MTP verify.
+        // A <=3-token AR prefix cannot have executed a >=3-row body, whereas
+        // MTP can retain a short frontier from a wider verification window.
+        if manifest.identity.head_chunk != identity.head_chunk
+            && (identity.mtp || manifest.tokens.len() >= 4)
+        {
+            return Err("Qwen checkpoint SDPA head-chunk policy differs; rebuild the cache".into());
+        }
         // Old files do not record prefill chunk sizes or producer hardware.
         // At ten routes/token this length can contain an affected chunk;
         // shorter sessions cannot. Reject before allocating/restoring state.
@@ -312,6 +326,7 @@ fn qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state() {
         context: 64,
         vocabulary: 256,
         mtp: true,
+        head_chunk: super::policy::sdpa_head_chunk_enabled(|key| std::env::var(key).ok()),
     };
     let directory =
         std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_CHECKPOINT_DIR").unwrap());
@@ -522,6 +537,87 @@ fn qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state() {
             .contains("sorted-gather row guard")
     );
     assert_eq!(fingerprint(&live), ar_original);
+    // Policy changes in either direction invalidate affected checkpoints;
+    // missing metadata is the actual pre-port (unchunked) policy.
+    for mtp in [false, true] {
+        for count in [1, 3, 4] {
+            for produced in [false, true] {
+                let saved = live.current.as_mut().unwrap();
+                saved.token_ids = vec![1; count];
+                saved.mtp = mtp.then(|| vec![None]);
+                saved.boundaries.clear();
+                let producer = Identity {
+                    mtp,
+                    head_chunk: produced,
+                    ..identity
+                };
+                live.save_checkpoint(
+                    &path,
+                    producer,
+                    [113; 32],
+                    &streams,
+                    stream,
+                    |_| {},
+                    || false,
+                )
+                .unwrap();
+                let original = fingerprint(&live);
+                let bytes = fs::read(&path).unwrap();
+                let offset = u64::from_le_bytes(bytes[8..16].try_into().unwrap()) as usize;
+                for legacy in [false, true] {
+                    if legacy && produced {
+                        continue;
+                    }
+                    let mut manifest: serde_json::Value =
+                        serde_json::from_slice(&bytes[offset..]).unwrap();
+                    if legacy {
+                        manifest["identity"]
+                            .as_object_mut()
+                            .unwrap()
+                            .remove("head_chunk");
+                    }
+                    let raw = serde_json::to_vec(&manifest).unwrap();
+                    let mut file = bytes[..offset].to_vec();
+                    file.extend(&raw);
+                    fs::write(&broken, file).unwrap();
+                    for consumed in [false, true] {
+                        let consumer = Identity {
+                            head_chunk: consumed,
+                            ..producer
+                        };
+                        let mut read = 0;
+                        let result = live.load_checkpoint(
+                            &broken,
+                            consumer,
+                            &streams,
+                            stream,
+                            |n| read += n,
+                            || false,
+                        );
+                        if produced != consumed && (mtp || count >= 4) {
+                            assert!(
+                                result
+                                    .unwrap_err()
+                                    .contains("SDPA head-chunk policy differs")
+                            );
+                            assert_eq!(
+                                read,
+                                HEADER_BYTES + raw.len() as u64,
+                                "rejected before tensor reads"
+                            );
+                        } else {
+                            assert!(result.unwrap());
+                        }
+                        assert_eq!(fingerprint(&live), original);
+                        assert_eq!(live.current.as_ref().unwrap().token_ids, vec![1; count]);
+                    }
+                }
+            }
+        }
+    }
+    println!(
+        "#113 SDPA policy: legacy/current, both directions, AR/MTP, 1/3/4-token frontiers; exact compatible restore and incompatible rejection before tensor reads"
+    );
     println!(
         "Qwen sorted-gather checkpoint policy: old short state exact, possibly affected long state rejected before restore"
     );
diff --git a/src/engine/metal/qwen_mtplx/verify_sdpa.rs b/src/engine/metal/qwen_mtplx/verify_sdpa.rs
new file mode 100644
index 0000000..f73eb47
--- /dev/null
+++ b/src/engine/metal/qwen_mtplx/verify_sdpa.rs
@@ -0,0 +1,267 @@
+//! Pinned qwen4_exp._verify_sdpa; selected by geometry, not attention phase.
+use super::array::Array;
+use super::stream::Stream;
+use super::{attention, ops};
+
+pub(super) fn plan(rows: i32, heads: i32, kv_heads: i32) -> Option<(i32, i32)> {
+    if kv_heads <= 0 || heads % kv_heads != 0 {
+        return None;
+    }
+    let gqa = heads / kv_heads;
+    if gqa <= 0 || i64::from(rows) * i64::from(gqa) <= 32 || rows > 32 {
+        return None;
+    }
+    let width = (32 / rows.max(1)).min(gqa).max(1);
+    Some((width, kv_heads * ((gqa - 1) / width + 1)))
+}
+
+pub(super) fn call(
+    q: &Array,
+    k: &Array,
+    v: &Array,
+    mask: Option<&Array>,
+    scale: f32,
+    enabled: bool,
+    stream: Stream,
+) -> Result<Array, String> {
+    let qs = q.layout().shape().to_vec();
+    let ks = k.layout().shape().to_vec();
+    if qs.len() != 4 || ks.len() != 4 {
+        return Err("Qwen SDPA expects rank-four queries and keys".into());
+    }
+    let Some((width, chunks)) = enabled.then(|| plan(qs[2], qs[1], ks[1])).flatten() else {
+        return attention::scaled_dot_product(q, k, v, mask, scale, stream);
+    };
+    static ENGAGED: std::sync::Once = std::sync::Once::new();
+    ENGAGED.call_once(|| {
+        eprintln!(
+            "[mtplx] verify SDPA head-chunked: q_len={} heads_per_chunk={width} chunks={chunks}",
+            qs[2]
+        );
+    });
+    let gqa = qs[1] / ks[1];
+    let qg = ops::reshape(q, &[qs[0], ks[1], gqa, qs[2], qs[3]], stream)?;
+    let mut outputs = Vec::with_capacity(chunks as usize);
+    for group in 0..ks[1] {
+        let kv = |a: &Array| {
+            let mut end = a.layout().shape().to_vec();
+            end[1] = group + 1;
+            ops::slice(a, &[0, group, 0, 0], &end, &[1; 4], stream)
+        };
+        let kj = kv(k)?;
+        let vj = kv(v)?;
+        for start in (0..gqa).step_by(width as usize) {
+            let end = (start + width).min(gqa);
+            let qc = ops::squeeze(
+                &ops::slice(
+                    &qg,
+                    &[0, group, start, 0, 0],
+                    &[qs[0], group + 1, end, qs[2], qs[3]],
+                    &[1; 5],
+                    stream,
+                )?,
+                Some(&[1]),
+                stream,
+            )?;
+            // The pinned wrapper passes the original mask unchanged. The
+            // primitive broadcasts it or rejects incompatible head dimensions.
+            outputs.push(attention::scaled_dot_product(
+                &qc, &kj, &vj, mask, scale, stream,
+            )?);
+        }
+    }
+    ops::concatenate(&outputs, 1, stream)
+}
+
+#[test]
+fn issue113_plan_and_policy_match_pinned_oracle() {
+    let fixture: serde_json::Value = serde_json::from_str(include_str!(
+        "../../../../tests/fixtures/mtplx-head-chunk.json"
+    ))
+    .unwrap();
+    let plans = fixture["plans"].as_array().unwrap();
+    assert_eq!(plans.len(), 520);
+    for p in plans {
+        let n = |k| p[k].as_i64().unwrap() as i32;
+        assert_eq!(
+            serde_json::to_value(plan(n("rows"), n("heads"), n("kv"))).unwrap(),
+            p["plan"]
+        );
+    }
+    for (value, expected) in [
+        (None, true),
+        (Some(""), true),
+        (Some("unexpected"), true),
+        (Some("0"), false),
+        (Some(" FALSE "), false),
+        (Some("Off"), false),
+        (Some("NO"), false),
+    ] {
+        assert_eq!(
+            super::policy::sdpa_head_chunk_enabled(|_| value.map(str::to_owned)),
+            expected
+        );
+    }
+}
+
+#[test]
+#[ignore = "Metal, pinned head-chunk numerical and dispatch fixtures; serial supervisor"]
+fn issue113_head_chunk_outputs_and_dispatch() {
+    use super::super::{
+        gpu::Context,
+        qwen_mtplx_tests::{capture_dispatches, hex_digest, pattern},
+    };
+    use super::array::Dtype;
+    super::configure_sources().unwrap();
+    let _context = Context::open_qwen(0).unwrap();
+    let streams = super::stream::Streams::new(Some(super::allocator::Allocator::new().unwrap()));
+    let gpu = streams.default_stream(super::stream::Device::Gpu).unwrap();
+    let fixture: serde_json::Value = serde_json::from_str(include_str!(
+        "../../../../tests/fixtures/mtplx-head-chunk.json"
+    ))
+    .unwrap();
+    let cases = fixture["cases"].as_array().unwrap();
+    assert_eq!(cases.len(), 216);
+    for (index, c) in cases.iter().enumerate() {
+        let n = |k| c[k].as_i64().unwrap() as i32;
+        let (b, h, kv, r, t) = (n("batch"), n("heads"), n("kv"), n("rows"), n("total"));
+        let bf = |shape: &[i32], salt| {
+            Array::new(
+                shape,
+                Dtype::BF16,
+                pattern(shape.iter().map(|&x| x as u32).product(), salt),
+            )
+            .unwrap()
+        };
+        let q = ops::transpose(&bf(&[b, r, h, 256], 93), &[0, 2, 1, 3], gpu).unwrap();
+        let source = |salt| {
+            ops::slice(
+                &bf(&[b, kv, t + 7, 256], salt),
+                &[0, 0, 2, 0],
+                &[b, kv, t + 2, 256],
+                &[1; 4],
+                gpu,
+            )
+            .unwrap()
+        };
+        let (k, v) = (source(94), source(95));
+        let mode = c["mode"].as_str().unwrap();
+        let mask = if mode == "none" {
+            None
+        } else {
+            let mh = if mode == "heads" {
+                h
+            } else if mode == "local" {
+                4
+            } else {
+                1
+            };
+            let mut bytes = Vec::new();
+            for head in 0..mh {
+                for row in 0..r {
+                    for col in 0..t {
+                        let on = col <= t - r + row && (col + head) % 3 != 1 && row != 1;
+                        if mode == "add" {
+                            bytes.extend((if on { 0u16 } else { 0xc120 }).to_le_bytes());
+                        } else {
+                            bytes.push(u8::from(on));
+                        }
+                    }
+                }
+            }
+            let shape = if mode == "two_dim" {
+                vec![r, t]
+            } else {
+                vec![1, mh, r, t]
+            };
+            Some(
+                Array::new(
+                    &shape,
+                    if mode == "add" {
+                        Dtype::BF16
+                    } else {
+                        Dtype::Bool
+                    },
+                    super::scalar_buffer(&bytes).unwrap(),
+                )
+                .unwrap(),
+            )
+        };
+        let result = call(
+            &q,
+            &k,
+            &v,
+            mask.as_ref(),
+            0.0625,
+            c["enabled"].as_bool().unwrap(),
+            gpu,
+        );
+        if c.get("error").is_some() {
+            assert!(
+                result.is_err(),
+                "case {index}: expected oracle mask/shape rejection"
+            );
+            continue;
+        }
+        let output = result.unwrap_or_else(|e| panic!("case {index}: {e}"));
+        let chunked = c["enabled"].as_bool().unwrap() && plan(r, h, kv).is_some();
+        let outputs = if chunked {
+            output.inputs().clone()
+        } else {
+            vec![output.clone()]
+        };
+        let calls = c["calls"].as_array().unwrap();
+        assert_eq!(outputs.len(), calls.len(), "case {index}");
+        for (part, expected) in outputs.iter().zip(calls) {
+            assert_eq!(
+                serde_json::json!(part.layout().shape()),
+                expected["q"],
+                "case {index}"
+            );
+            if matches!(
+                part.operation().as_deref(),
+                Some(super::ops::Operation::SdpaVector { .. })
+            ) {
+                for (input, name) in part.inputs().iter().take(3).zip(["q", "k", "v"]) {
+                    assert_eq!(
+                        serde_json::json!(input.layout().shape()),
+                        expected[name],
+                        "case {index}"
+                    );
+                }
+            }
+        }
+        drop(outputs);
+        let (_, dispatches) = capture_dispatches(|| {
+            ops::evaluate(&streams, std::slice::from_ref(&output), gpu, false).unwrap()
+        });
+        let fused = dispatches
+            .iter()
+            .filter(|d| d.0.starts_with("sdpa_vector") && !d.0.starts_with("sdpa_vector_2pass_2"))
+            .count();
+        assert_eq!(
+            fused,
+            c["fused_primitives"].as_u64().unwrap() as usize,
+            "case {index}: {dispatches:?}"
+        );
+        assert_eq!(
+            dispatches
+                .iter()
+                .filter(|d| d.0.contains("softmax"))
+                .count(),
+            calls.len() - fused,
+            "case {index}"
+        );
+        let mut bytes = vec![0; output.layout().nbytes()];
+        output.buffer().read(output.offset(), &mut bytes).unwrap();
+        assert_eq!(
+            hex_digest(&bytes),
+            c["sha256"].as_str().unwrap(),
+            "case {index}: {c}"
+        );
+        println!(
+            "{}",
+            serde_json::json!({"case":"head_chunk","index":index,"exact":true,"fused":fused,"subcalls":calls.len()})
+        );
+    }
+}
diff --git a/tests/fixtures/mtplx-head-chunk.json b/tests/fixtures/mtplx-head-chunk.json
new file mode 100644
index 0000000..85b5e59
--- /dev/null
+++ b/tests/fixtures/mtplx-head-chunk.json
@@ -0,0 +1 @@
+{"revision":"9882703f3105363ddc37eca9f97aa09a1d387112","mlx":"0.32.2","plans":[{"rows":-1,"heads":0,"kv":0,"plan":null},{"rows":-1,"heads":0,"kv":1,"plan":null},{"rows":-1,"heads":0,"kv":2,"plan":null},{"rows":-1,"heads":0,"kv":3,"plan":null},{"rows":-1,"heads":0,"kv":5,"plan":null},{"rows":-1,"heads":1,"kv":0,"plan":null},{"rows":-1,"heads":1,"kv":1,"plan":null},{"rows":-1,"heads":1,"kv":2,"plan":null},{"rows":-1,"heads":1,"kv":3,"plan":null},{"rows":-1,"heads":1,"kv":5,"plan":null},{"rows":-1,"heads":2,"kv":0,"plan":null},{"rows":-1,"heads":2,"kv":1,"plan":null},{"rows":-1,"heads":2,"kv":2,"plan":null},{"rows":-1,"heads":2,"kv":3,"plan":null},{"rows":-1,"heads":2,"kv":5,"plan":null},{"rows":-1,"heads":12,"kv":0,"plan":null},{"rows":-1,"heads":12,"kv":1,"plan":null},{"rows":-1,"heads":12,"kv":2,"plan":null},{"rows":-1,"heads":12,"kv":3,"plan":null},{"rows":-1,"heads":12,"kv":5,"plan":null},{"rows":-1,"heads":24,"kv":0,"plan":null},{"rows":-1,"heads":24,"kv":1,"plan":null},{"rows":-1,"heads":24,"kv":2,"plan":null},{"rows":-1,"heads":24,"kv":3,"plan":null},{"rows":-1,"heads":24,"kv":5,"plan":null},{"rows":-1,"heads":33,"kv":0,"plan":null},{"rows":-1,"heads":33,"kv":1,"plan":null},{"rows":-1,"heads":33,"kv":2,"plan":null},{"rows":-1,"heads":33,"kv":3,"plan":null},{"rows":-1,"heads":33,"kv":5,"plan":null},{"rows":-1,"heads":36,"kv":0,"plan":null},{"rows":-1,"heads":36,"kv":1,"plan":null},{"rows":-1,"heads":36,"kv":2,"plan":null},{"rows":-1,"heads":36,"kv":3,"plan":null},{"rows":-1,"heads":36,"kv":5,"plan":null},{"rows":-1,"heads":64,"kv":0,"plan":null},{"rows":-1,"heads":64,"kv":1,"plan":null},{"rows":-1,"heads":64,"kv":2,"plan":null},{"rows":-1,"heads":64,"kv":3,"plan":null},{"rows":-1,"heads":64,"kv":5,"plan":null},{"rows":0,"heads":0,"kv":0,"plan":null},{"rows":0,"heads":0,"kv":1,"plan":null},{"rows":0,"heads":0,"kv":2,"plan":null},{"rows":0,"heads":0,"kv":3,"plan":null},{"rows":0,"heads":0,"kv":5,"plan":null},{"rows":0,"heads":1,"kv":0,"plan":null},{"rows":0,"heads":1,"kv":1,"plan":null},{"rows":0,"heads":1,"kv":2,"plan":null},{"rows":0,"heads":1,"kv":3,"plan":null},{"rows":0,"heads":1,"kv":5,"plan":null},{"rows":0,"heads":2,"kv":0,"plan":null},{"rows":0,"heads":2,"kv":1,"plan":null},{"rows":0,"heads":2,"kv":2,"plan":null},{"rows":0,"heads":2,"kv":3,"plan":null},{"rows":0,"heads":2,"kv":5,"plan":null},{"rows":0,"heads":12,"kv":0,"plan":null},{"rows":0,"heads":12,"kv":1,"plan":null},{"rows":0,"heads":12,"kv":2,"plan":null},{"rows":0,"heads":12,"kv":3,"plan":null},{"rows":0,"heads":12,"kv":5,"plan":null},{"rows":0,"heads":24,"kv":0,"plan":null},{"rows":0,"heads":24,"kv":1,"plan":null},{"rows":0,"heads":24,"kv":2,"plan":null},{"rows":0,"heads":24,"kv":3,"plan":null},{"rows":0,"heads":24,"kv":5,"plan":null},{"rows":0,"heads":33,"kv":0,"plan":null},{"rows":0,"heads":33,"kv":1,"plan":null},{"rows":0,"heads":33,"kv":2,"plan":null},{"rows":0,"heads":33,"kv":3,"plan":null},{"rows":0,"heads":33,"kv":5,"plan":null},{"rows":0,"heads":36,"kv":0,"plan":null},{"rows":0,"heads":36,"kv":1,"plan":null},{"rows":0,"heads":36,"kv":2,"plan":null},{"rows":0,"heads":36,"kv":3,"plan":null},{"rows":0,"heads":36,"kv":5,"plan":null},{"rows":0,"heads":64,"kv":0,"plan":null},{"rows":0,"heads":64,"kv":1,"plan":null},{"rows":0,"heads":64,"kv":2,"plan":null},{"rows":0,"heads":64,"kv":3,"plan":null},{"rows":0,"heads":64,"kv":5,"plan":null},{"rows":1,"heads":0,"kv":0,"plan":null},{"rows":1,"heads":0,"kv":1,"plan":null},{"rows":1,"heads":0,"kv":2,"plan":null},{"rows":1,"heads":0,"kv":3,"plan":null},{"rows":1,"heads":0,"kv":5,"plan":null},{"rows":1,"heads":1,"kv":0,"plan":null},{"rows":1,"heads":1,"kv":1,"plan":null},{"rows":1,"heads":1,"kv":2,"plan":null},{"rows":1,"heads":1,"kv":3,"plan":null},{"rows":1,"heads":1,"kv":5,"plan":null},{"rows":1,"heads":2,"kv":0,"plan":null},{"rows":1,"heads":2,"kv":1,"plan":null},{"rows":1,"heads":2,"kv":2,"plan":null},{"rows":1,"heads":2,"kv":3,"plan":null},{"rows":1,"heads":2,"kv":5,"plan":null},{"rows":1,"heads":12,"kv":0,"plan":null},{"rows":1,"heads":12,"kv":1,"plan":null},{"rows":1,"heads":12,"kv":2,"plan":null},{"rows":1,"heads":12,"kv":3,"plan":null},{"rows":1,"heads":12,"kv":5,"plan":null},{"rows":1,"heads":24,"kv":0,"plan":null},{"rows":1,"heads":24,"kv":1,"plan":null},{"rows":1,"heads":24,"kv":2,"plan":null},{"rows":1,"heads":24,"kv":3,"plan":null},{"rows":1,"heads":24,"kv":5,"plan":null},{"rows":1,"heads":33,"kv":0,"plan":null},{"rows":1,"heads":33,"kv":1,"plan":[32,2]},{"rows":1,"heads":33,"kv":2,"plan":null},{"rows":1,"heads":33,"kv":3,"plan":null},{"rows":1,"heads":33,"kv":5,"plan":null},{"rows":1,"heads":36,"kv":0,"plan":null},{"rows":1,"heads":36,"kv":1,"plan":[32,2]},{"rows":1,"heads":36,"kv":2,"plan":null},{"rows":1,"heads":36,"kv":3,"plan":null},{"rows":1,"heads":36,"kv":5,"plan":null},{"rows":1,"heads":64,"kv":0,"plan":null},{"rows":1,"heads":64,"kv":1,"plan":[32,2]},{"rows":1,"heads":64,"kv":2,"plan":null},{"rows":1,"heads":64,"kv":3,"plan":null},{"rows":1,"heads":64,"kv":5,"plan":null},{"rows":2,"heads":0,"kv":0,"plan":null},{"rows":2,"heads":0,"kv":1,"plan":null},{"rows":2,"heads":0,"kv":2,"plan":null},{"rows":2,"heads":0,"kv":3,"plan":null},{"rows":2,"heads":0,"kv":5,"plan":null},{"rows":2,"heads":1,"kv":0,"plan":null},{"rows":2,"heads":1,"kv":1,"plan":null},{"rows":2,"heads":1,"kv":2,"plan":null},{"rows":2,"heads":1,"kv":3,"plan":null},{"rows":2,"heads":1,"kv":5,"plan":null},{"rows":2,"heads":2,"kv":0,"plan":null},{"rows":2,"heads":2,"kv":1,"plan":null},{"rows":2,"heads":2,"kv":2,"plan":null},{"rows":2,"heads":2,"kv":3,"plan":null},{"rows":2,"heads":2,"kv":5,"plan":null},{"rows":2,"heads":12,"kv":0,"plan":null},{"rows":2,"heads":12,"kv":1,"plan":null},{"rows":2,"heads":12,"kv":2,"plan":null},{"rows":2,"heads":12,"kv":3,"plan":null},{"rows":2,"heads":12,"kv":5,"plan":null},{"rows":2,"heads":24,"kv":0,"plan":null},{"rows":2,"heads":24,"kv":1,"plan":[16,2]},{"rows":2,"heads":24,"kv":2,"plan":null},{"rows":2,"heads":24,"kv":3,"plan":null},{"rows":2,"heads":24,"kv":5,"plan":null},{"rows":2,"heads":33,"kv":0,"plan":null},{"rows":2,"heads":33,"kv":1,"plan":[16,3]},{"rows":2,"heads":33,"kv":2,"plan":null},{"rows":2,"heads":33,"kv":3,"plan":null},{"rows":2,"heads":33,"kv":5,"plan":null},{"rows":2,"heads":36,"kv":0,"plan":null},{"rows":2,"heads":36,"kv":1,"plan":[16,3]},{"rows":2,"heads":36,"kv":2,"plan":[16,4]},{"rows":2,"heads":36,"kv":3,"plan":null},{"rows":2,"heads":36,"kv":5,"plan":null},{"rows":2,"heads":64,"kv":0,"plan":null},{"rows":2,"heads":64,"kv":1,"plan":[16,4]},{"rows":2,"heads":64,"kv":2,"plan":[16,4]},{"rows":2,"heads":64,"kv":3,"plan":null},{"rows":2,"heads":64,"kv":5,"plan":null},{"rows":3,"heads":0,"kv":0,"plan":null},{"rows":3,"heads":0,"kv":1,"plan":null},{"rows":3,"heads":0,"kv":2,"plan":null},{"rows":3,"heads":0,"kv":3,"plan":null},{"rows":3,"heads":0,"kv":5,"plan":null},{"rows":3,"heads":1,"kv":0,"plan":null},{"rows":3,"heads":1,"kv":1,"plan":null},{"rows":3,"heads":1,"kv":2,"plan":null},{"rows":3,"heads":1,"kv":3,"plan":null},{"rows":3,"heads":1,"kv":5,"plan":null},{"rows":3,"heads":2,"kv":0,"plan":null},{"rows":3,"heads":2,"kv":1,"plan":null},{"rows":3,"heads":2,"kv":2,"plan":null},{"rows":3,"heads":2,"kv":3,"plan":null},{"rows":3,"heads":2,"kv":5,"plan":null},{"rows":3,"heads":12,"kv":0,"plan":null},{"rows":3,"heads":12,"kv":1,"plan":[10,2]},{"rows":3,"heads":12,"kv":2,"plan":null},{"rows":3,"heads":12,"kv":3,"plan":null},{"rows":3,"heads":12,"kv":5,"plan":null},{"rows":3,"heads":24,"kv":0,"plan":null},{"rows":3,"heads":24,"kv":1,"plan":[10,3]},{"rows":3,"heads":24,"kv":2,"plan":[10,4]},{"rows":3,"heads":24,"kv":3,"plan":null},{"rows":3,"heads":24,"kv":5,"plan":null},{"rows":3,"heads":33,"kv":0,"plan":null},{"rows":3,"heads":33,"kv":1,"plan":[10,4]},{"rows":3,"heads":33,"kv":2,"plan":null},{"rows":3,"heads":33,"kv":3,"plan":[10,6]},{"rows":3,"heads":33,"kv":5,"plan":null},{"rows":3,"heads":36,"kv":0,"plan":null},{"rows":3,"heads":36,"kv":1,"plan":[10,4]},{"rows":3,"heads":36,"kv":2,"plan":[10,4]},{"rows":3,"heads":36,"kv":3,"plan":[10,6]},{"rows":3,"heads":36,"kv":5,"plan":null},{"rows":3,"heads":64,"kv":0,"plan":null},{"rows":3,"heads":64,"kv":1,"plan":[10,7]},{"rows":3,"heads":64,"kv":2,"plan":[10,8]},{"rows":3,"heads":64,"kv":3,"plan":null},{"rows":3,"heads":64,"kv":5,"plan":null},{"rows":4,"heads":0,"kv":0,"plan":null},{"rows":4,"heads":0,"kv":1,"plan":null},{"rows":4,"heads":0,"kv":2,"plan":null},{"rows":4,"heads":0,"kv":3,"plan":null},{"rows":4,"heads":0,"kv":5,"plan":null},{"rows":4,"heads":1,"kv":0,"plan":null},{"rows":4,"heads":1,"kv":1,"plan":null},{"rows":4,"heads":1,"kv":2,"plan":null},{"rows":4,"heads":1,"kv":3,"plan":null},{"rows":4,"heads":1,"kv":5,"plan":null},{"rows":4,"heads":2,"kv":0,"plan":null},{"rows":4,"heads":2,"kv":1,"plan":null},{"rows":4,"heads":2,"kv":2,"plan":null},{"rows":4,"heads":2,"kv":3,"plan":null},{"rows":4,"heads":2,"kv":5,"plan":null},{"rows":4,"heads":12,"kv":0,"plan":null},{"rows":4,"heads":12,"kv":1,"plan":[8,2]},{"rows":4,"heads":12,"kv":2,"plan":null},{"rows":4,"heads":12,"kv":3,"plan":null},{"rows":4,"heads":12,"kv":5,"plan":null},{"rows":4,"heads":24,"kv":0,"plan":null},{"rows":4,"heads":24,"kv":1,"plan":[8,3]},{"rows":4,"heads":24,"kv":2,"plan":[8,4]},{"rows":4,"heads":24,"kv":3,"plan":null},{"rows":4,"heads":24,"kv":5,"plan":null},{"rows":4,"heads":33,"kv":0,"plan":null},{"rows":4,"heads":33,"kv":1,"plan":[8,5]},{"rows":4,"heads":33,"kv":2,"plan":null},{"rows":4,"heads":33,"kv":3,"plan":[8,6]},{"rows":4,"heads":33,"kv":5,"plan":null},{"rows":4,"heads":36,"kv":0,"plan":null},{"rows":4,"heads":36,"kv":1,"plan":[8,5]},{"rows":4,"heads":36,"kv":2,"plan":[8,6]},{"rows":4,"heads":36,"kv":3,"plan":[8,6]},{"rows":4,"heads":36,"kv":5,"plan":null},{"rows":4,"heads":64,"kv":0,"plan":null},{"rows":4,"heads":64,"kv":1,"plan":[8,8]},{"rows":4,"heads":64,"kv":2,"plan":[8,8]},{"rows":4,"heads":64,"kv":3,"plan":null},{"rows":4,"heads":64,"kv":5,"plan":null},{"rows":8,"heads":0,"kv":0,"plan":null},{"rows":8,"heads":0,"kv":1,"plan":null},{"rows":8,"heads":0,"kv":2,"plan":null},{"rows":8,"heads":0,"kv":3,"plan":null},{"rows":8,"heads":0,"kv":5,"plan":null},{"rows":8,"heads":1,"kv":0,"plan":null},{"rows":8,"heads":1,"kv":1,"plan":null},{"rows":8,"heads":1,"kv":2,"plan":null},{"rows":8,"heads":1,"kv":3,"plan":null},{"rows":8,"heads":1,"kv":5,"plan":null},{"rows":8,"heads":2,"kv":0,"plan":null},{"rows":8,"heads":2,"kv":1,"plan":null},{"rows":8,"heads":2,"kv":2,"plan":null},{"rows":8,"heads":2,"kv":3,"plan":null},{"rows":8,"heads":2,"kv":5,"plan":null},{"rows":8,"heads":12,"kv":0,"plan":null},{"rows":8,"heads":12,"kv":1,"plan":[4,3]},{"rows":8,"heads":12,"kv":2,"plan":[4,4]},{"rows":8,"heads":12,"kv":3,"plan":null},{"rows":8,"heads":12,"kv":5,"plan":null},{"rows":8,"heads":24,"kv":0,"plan":null},{"rows":8,"heads":24,"kv":1,"plan":[4,6]},{"rows":8,"heads":24,"kv":2,"plan":[4,6]},{"rows":8,"heads":24,"kv":3,"plan":[4,6]},{"rows":8,"heads":24,"kv":5,"plan":null},{"rows":8,"heads":33,"kv":0,"plan":null},{"rows":8,"heads":33,"kv":1,"plan":[4,9]},{"rows":8,"heads":33,"kv":2,"plan":null},{"rows":8,"heads":33,"kv":3,"plan":[4,9]},{"rows":8,"heads":33,"kv":5,"plan":null},{"rows":8,"heads":36,"kv":0,"plan":null},{"rows":8,"heads":36,"kv":1,"plan":[4,9]},{"rows":8,"heads":36,"kv":2,"plan":[4,10]},{"rows":8,"heads":36,"kv":3,"plan":[4,9]},{"rows":8,"heads":36,"kv":5,"plan":null},{"rows":8,"heads":64,"kv":0,"plan":null},{"rows":8,"heads":64,"kv":1,"plan":[4,16]},{"rows":8,"heads":64,"kv":2,"plan":[4,16]},{"rows":8,"heads":64,"kv":3,"plan":null},{"rows":8,"heads":64,"kv":5,"plan":null},{"rows":9,"heads":0,"kv":0,"plan":null},{"rows":9,"heads":0,"kv":1,"plan":null},{"rows":9,"heads":0,"kv":2,"plan":null},{"rows":9,"heads":0,"kv":3,"plan":null},{"rows":9,"heads":0,"kv":5,"plan":null},{"rows":9,"heads":1,"kv":0,"plan":null},{"rows":9,"heads":1,"kv":1,"plan":null},{"rows":9,"heads":1,"kv":2,"plan":null},{"rows":9,"heads":1,"kv":3,"plan":null},{"rows":9,"heads":1,"kv":5,"plan":null},{"rows":9,"heads":2,"kv":0,"plan":null},{"rows":9,"heads":2,"kv":1,"plan":null},{"rows":9,"heads":2,"kv":2,"plan":null},{"rows":9,"heads":2,"kv":3,"plan":null},{"rows":9,"heads":2,"kv":5,"plan":null},{"rows":9,"heads":12,"kv":0,"plan":null},{"rows":9,"heads":12,"kv":1,"plan":[3,4]},{"rows":9,"heads":12,"kv":2,"plan":[3,4]},{"rows":9,"heads":12,"kv":3,"plan":[3,6]},{"rows":9,"heads":12,"kv":5,"plan":null},{"rows":9,"heads":24,"kv":0,"plan":null},{"rows":9,"heads":24,"kv":1,"plan":[3,8]},{"rows":9,"heads":24,"kv":2,"plan":[3,8]},{"rows":9,"heads":24,"kv":3,"plan":[3,9]},{"rows":9,"heads":24,"kv":5,"plan":null},{"rows":9,"heads":33,"kv":0,"plan":null},{"rows":9,"heads":33,"kv":1,"plan":[3,11]},{"rows":9,"heads":33,"kv":2,"plan":null},{"rows":9,"heads":33,"kv":3,"plan":[3,12]},{"rows":9,"heads":33,"kv":5,"plan":null},{"rows":9,"heads":36,"kv":0,"plan":null},{"rows":9,"heads":36,"kv":1,"plan":[3,12]},{"rows":9,"heads":36,"kv":2,"plan":[3,12]},{"rows":9,"heads":36,"kv":3,"plan":[3,12]},{"rows":9,"heads":36,"kv":5,"plan":null},{"rows":9,"heads":64,"kv":0,"plan":null},{"rows":9,"heads":64,"kv":1,"plan":[3,22]},{"rows":9,"heads":64,"kv":2,"plan":[3,22]},{"rows":9,"heads":64,"kv":3,"plan":null},{"rows":9,"heads":64,"kv":5,"plan":null},{"rows":16,"heads":0,"kv":0,"plan":null},{"rows":16,"heads":0,"kv":1,"plan":null},{"rows":16,"heads":0,"kv":2,"plan":null},{"rows":16,"heads":0,"kv":3,"plan":null},{"rows":16,"heads":0,"kv":5,"plan":null},{"rows":16,"heads":1,"kv":0,"plan":null},{"rows":16,"heads":1,"kv":1,"plan":null},{"rows":16,"heads":1,"kv":2,"plan":null},{"rows":16,"heads":1,"kv":3,"plan":null},{"rows":16,"heads":1,"kv":5,"plan":null},{"rows":16,"heads":2,"kv":0,"plan":null},{"rows":16,"heads":2,"kv":1,"plan":null},{"rows":16,"heads":2,"kv":2,"plan":null},{"rows":16,"heads":2,"kv":3,"plan":null},{"rows":16,"heads":2,"kv":5,"plan":null},{"rows":16,"heads":12,"kv":0,"plan":null},{"rows":16,"heads":12,"kv":1,"plan":[2,6]},{"rows":16,"heads":12,"kv":2,"plan":[2,6]},{"rows":16,"heads":12,"kv":3,"plan":[2,6]},{"rows":16,"heads":12,"kv":5,"plan":null},{"rows":16,"heads":24,"kv":0,"plan":null},{"rows":16,"heads":24,"kv":1,"plan":[2,12]},{"rows":16,"heads":24,"kv":2,"plan":[2,12]},{"rows":16,"heads":24,"kv":3,"plan":[2,12]},{"rows":16,"heads":24,"kv":5,"plan":null},{"rows":16,"heads":33,"kv":0,"plan":null},{"rows":16,"heads":33,"kv":1,"plan":[2,17]},{"rows":16,"heads":33,"kv":2,"plan":null},{"rows":16,"heads":33,"kv":3,"plan":[2,18]},{"rows":16,"heads":33,"kv":5,"plan":null},{"rows":16,"heads":36,"kv":0,"plan":null},{"rows":16,"heads":36,"kv":1,"plan":[2,18]},{"rows":16,"heads":36,"kv":2,"plan":[2,18]},{"rows":16,"heads":36,"kv":3,"plan":[2,18]},{"rows":16,"heads":36,"kv":5,"plan":null},{"rows":16,"heads":64,"kv":0,"plan":null},{"rows":16,"heads":64,"kv":1,"plan":[2,32]},{"rows":16,"heads":64,"kv":2,"plan":[2,32]},{"rows":16,"heads":64,"kv":3,"plan":null},{"rows":16,"heads":64,"kv":5,"plan":null},{"rows":31,"heads":0,"kv":0,"plan":null},{"rows":31,"heads":0,"kv":1,"plan":null},{"rows":31,"heads":0,"kv":2,"plan":null},{"rows":31,"heads":0,"kv":3,"plan":null},{"rows":31,"heads":0,"kv":5,"plan":null},{"rows":31,"heads":1,"kv":0,"plan":null},{"rows":31,"heads":1,"kv":1,"plan":null},{"rows":31,"heads":1,"kv":2,"plan":null},{"rows":31,"heads":1,"kv":3,"plan":null},{"rows":31,"heads":1,"kv":5,"plan":null},{"rows":31,"heads":2,"kv":0,"plan":null},{"rows":31,"heads":2,"kv":1,"plan":[1,2]},{"rows":31,"heads":2,"kv":2,"plan":null},{"rows":31,"heads":2,"kv":3,"plan":null},{"rows":31,"heads":2,"kv":5,"plan":null},{"rows":31,"heads":12,"kv":0,"plan":null},{"rows":31,"heads":12,"kv":1,"plan":[1,12]},{"rows":31,"heads":12,"kv":2,"plan":[1,12]},{"rows":31,"heads":12,"kv":3,"plan":[1,12]},{"rows":31,"heads":12,"kv":5,"plan":null},{"rows":31,"heads":24,"kv":0,"plan":null},{"rows":31,"heads":24,"kv":1,"plan":[1,24]},{"rows":31,"heads":24,"kv":2,"plan":[1,24]},{"rows":31,"heads":24,"kv":3,"plan":[1,24]},{"rows":31,"heads":24,"kv":5,"plan":null},{"rows":31,"heads":33,"kv":0,"plan":null},{"rows":31,"heads":33,"kv":1,"plan":[1,33]},{"rows":31,"heads":33,"kv":2,"plan":null},{"rows":31,"heads":33,"kv":3,"plan":[1,33]},{"rows":31,"heads":33,"kv":5,"plan":null},{"rows":31,"heads":36,"kv":0,"plan":null},{"rows":31,"heads":36,"kv":1,"plan":[1,36]},{"rows":31,"heads":36,"kv":2,"plan":[1,36]},{"rows":31,"heads":36,"kv":3,"plan":[1,36]},{"rows":31,"heads":36,"kv":5,"plan":null},{"rows":31,"heads":64,"kv":0,"plan":null},{"rows":31,"heads":64,"kv":1,"plan":[1,64]},{"rows":31,"heads":64,"kv":2,"plan":[1,64]},{"rows":31,"heads":64,"kv":3,"plan":null},{"rows":31,"heads":64,"kv":5,"plan":null},{"rows":32,"heads":0,"kv":0,"plan":null},{"rows":32,"heads":0,"kv":1,"plan":null},{"rows":32,"heads":0,"kv":2,"plan":null},{"rows":32,"heads":0,"kv":3,"plan":null},{"rows":32,"heads":0,"kv":5,"plan":null},{"rows":32,"heads":1,"kv":0,"plan":null},{"rows":32,"heads":1,"kv":1,"plan":null},{"rows":32,"heads":1,"kv":2,"plan":null},{"rows":32,"heads":1,"kv":3,"plan":null},{"rows":32,"heads":1,"kv":5,"plan":null},{"rows":32,"heads":2,"kv":0,"plan":null},{"rows":32,"heads":2,"kv":1,"plan":[1,2]},{"rows":32,"heads":2,"kv":2,"plan":null},{"rows":32,"heads":2,"kv":3,"plan":null},{"rows":32,"heads":2,"kv":5,"plan":null},{"rows":32,"heads":12,"kv":0,"plan":null},{"rows":32,"heads":12,"kv":1,"plan":[1,12]},{"rows":32,"heads":12,"kv":2,"plan":[1,12]},{"rows":32,"heads":12,"kv":3,"plan":[1,12]},{"rows":32,"heads":12,"kv":5,"plan":null},{"rows":32,"heads":24,"kv":0,"plan":null},{"rows":32,"heads":24,"kv":1,"plan":[1,24]},{"rows":32,"heads":24,"kv":2,"plan":[1,24]},{"rows":32,"heads":24,"kv":3,"plan":[1,24]},{"rows":32,"heads":24,"kv":5,"plan":null},{"rows":32,"heads":33,"kv":0,"plan":null},{"rows":32,"heads":33,"kv":1,"plan":[1,33]},{"rows":32,"heads":33,"kv":2,"plan":null},{"rows":32,"heads":33,"kv":3,"plan":[1,33]},{"rows":32,"heads":33,"kv":5,"plan":null},{"rows":32,"heads":36,"kv":0,"plan":null},{"rows":32,"heads":36,"kv":1,"plan":[1,36]},{"rows":32,"heads":36,"kv":2,"plan":[1,36]},{"rows":32,"heads":36,"kv":3,"plan":[1,36]},{"rows":32,"heads":36,"kv":5,"plan":null},{"rows":32,"heads":64,"kv":0,"plan":null},{"rows":32,"heads":64,"kv":1,"plan":[1,64]},{"rows":32,"heads":64,"kv":2,"plan":[1,64]},{"rows":32,"heads":64,"kv":3,"plan":null},{"rows":32,"heads":64,"kv":5,"plan":null},{"rows":33,"heads":0,"kv":0,"plan":null},{"rows":33,"heads":0,"kv":1,"plan":null},{"rows":33,"heads":0,"kv":2,"plan":null},{"rows":33,"heads":0,"kv":3,"plan":null},{"rows":33,"heads":0,"kv":5,"plan":null},{"rows":33,"heads":1,"kv":0,"plan":null},{"rows":33,"heads":1,"kv":1,"plan":null},{"rows":33,"heads":1,"kv":2,"plan":null},{"rows":33,"heads":1,"kv":3,"plan":null},{"rows":33,"heads":1,"kv":5,"plan":null},{"rows":33,"heads":2,"kv":0,"plan":null},{"rows":33,"heads":2,"kv":1,"plan":null},{"rows":33,"heads":2,"kv":2,"plan":null},{"rows":33,"heads":2,"kv":3,"plan":null},{"rows":33,"heads":2,"kv":5,"plan":null},{"rows":33,"heads":12,"kv":0,"plan":null},{"rows":33,"heads":12,"kv":1,"plan":null},{"rows":33,"heads":12,"kv":2,"plan":null},{"rows":33,"heads":12,"kv":3,"plan":null},{"rows":33,"heads":12,"kv":5,"plan":null},{"rows":33,"heads":24,"kv":0,"plan":null},{"rows":33,"heads":24,"kv":1,"plan":null},{"rows":33,"heads":24,"kv":2,"plan":null},{"rows":33,"heads":24,"kv":3,"plan":null},{"rows":33,"heads":24,"kv":5,"plan":null},{"rows":33,"heads":33,"kv":0,"plan":null},{"rows":33,"heads":33,"kv":1,"plan":null},{"rows":33,"heads":33,"kv":2,"plan":null},{"rows":33,"heads":33,"kv":3,"plan":null},{"rows":33,"heads":33,"kv":5,"plan":null},{"rows":33,"heads":36,"kv":0,"plan":null},{"rows":33,"heads":36,"kv":1,"plan":null},{"rows":33,"heads":36,"kv":2,"plan":null},{"rows":33,"heads":36,"kv":3,"plan":null},{"rows":33,"heads":36,"kv":5,"plan":null},{"rows":33,"heads":64,"kv":0,"plan":null},{"rows":33,"heads":64,"kv":1,"plan":null},{"rows":33,"heads":64,"kv":2,"plan":null},{"rows":33,"heads":64,"kv":3,"plan":null},{"rows":33,"heads":64,"kv":5,"plan":null},{"rows":64,"heads":0,"kv":0,"plan":null},{"rows":64,"heads":0,"kv":1,"plan":null},{"rows":64,"heads":0,"kv":2,"plan":null},{"rows":64,"heads":0,"kv":3,"plan":null},{"rows":64,"heads":0,"kv":5,"plan":null},{"rows":64,"heads":1,"kv":0,"plan":null},{"rows":64,"heads":1,"kv":1,"plan":null},{"rows":64,"heads":1,"kv":2,"plan":null},{"rows":64,"heads":1,"kv":3,"plan":null},{"rows":64,"heads":1,"kv":5,"plan":null},{"rows":64,"heads":2,"kv":0,"plan":null},{"rows":64,"heads":2,"kv":1,"plan":null},{"rows":64,"heads":2,"kv":2,"plan":null},{"rows":64,"heads":2,"kv":3,"plan":null},{"rows":64,"heads":2,"kv":5,"plan":null},{"rows":64,"heads":12,"kv":0,"plan":null},{"rows":64,"heads":12,"kv":1,"plan":null},{"rows":64,"heads":12,"kv":2,"plan":null},{"rows":64,"heads":12,"kv":3,"plan":null},{"rows":64,"heads":12,"kv":5,"plan":null},{"rows":64,"heads":24,"kv":0,"plan":null},{"rows":64,"heads":24,"kv":1,"plan":null},{"rows":64,"heads":24,"kv":2,"plan":null},{"rows":64,"heads":24,"kv":3,"plan":null},{"rows":64,"heads":24,"kv":5,"plan":null},{"rows":64,"heads":33,"kv":0,"plan":null},{"rows":64,"heads":33,"kv":1,"plan":null},{"rows":64,"heads":33,"kv":2,"plan":null},{"rows":64,"heads":33,"kv":3,"plan":null},{"rows":64,"heads":33,"kv":5,"plan":null},{"rows":64,"heads":36,"kv":0,"plan":null},{"rows":64,"heads":36,"kv":1,"plan":null},{"rows":64,"heads":36,"kv":2,"plan":null},{"rows":64,"heads":36,"kv":3,"plan":null},{"rows":64,"heads":36,"kv":5,"plan":null},{"rows":64,"heads":64,"kv":0,"plan":null},{"rows":64,"heads":64,"kv":1,"plan":null},{"rows":64,"heads":64,"kv":2,"plan":null},{"rows":64,"heads":64,"kv":3,"plan":null},{"rows":64,"heads":64,"kv":5,"plan":null}],"cases":[{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"none","enabled":false,"fused_primitives":1,"sha256":"6dc3adca00be6915fff3bb8a727075b15420e42215bca1caaa89a3d739ae6749","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"none","enabled":true,"fused_primitives":1,"sha256":"6dc3adca00be6915fff3bb8a727075b15420e42215bca1caaa89a3d739ae6749","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"bool","enabled":false,"fused_primitives":1,"sha256":"e7bfb5930b3039fe685b8bbb3051fd16986c3d2bdf4646f1ec50c11b237920fb","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"bool","enabled":true,"fused_primitives":1,"sha256":"e7bfb5930b3039fe685b8bbb3051fd16986c3d2bdf4646f1ec50c11b237920fb","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"add","enabled":false,"fused_primitives":1,"sha256":"3134ea314b83fd44c069b7ad777b8bfdafed01e7fd2fc547492d254df5a076bb","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"add","enabled":true,"fused_primitives":1,"sha256":"3134ea314b83fd44c069b7ad777b8bfdafed01e7fd2fc547492d254df5a076bb","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"heads","enabled":false,"fused_primitives":1,"sha256":"fa3c8e33d58031a6979a751182e51f20a3f5809f50b7e1f1e6e4970b579ebca9","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"heads","enabled":true,"fused_primitives":1,"sha256":"fa3c8e33d58031a6979a751182e51f20a3f5809f50b7e1f1e6e4970b579ebca9","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,1,65) and (1,24,1,65) cannot be broadcast.","calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,1,65) and (1,24,1,65) cannot be broadcast.","calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"two_dim","enabled":false,"fused_primitives":1,"sha256":"e7bfb5930b3039fe685b8bbb3051fd16986c3d2bdf4646f1ec50c11b237920fb","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":1,"total":65,"mode":"two_dim","enabled":true,"fused_primitives":1,"sha256":"e7bfb5930b3039fe685b8bbb3051fd16986c3d2bdf4646f1ec50c11b237920fb","shape":[1,24,1,256],"calls":[{"q":[1,24,1,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"none","enabled":false,"fused_primitives":1,"sha256":"ce8ece909459a899a6443709c0d3cd467e37e975467b3d4791bd2b980668e44b","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"none","enabled":true,"fused_primitives":1,"sha256":"ce8ece909459a899a6443709c0d3cd467e37e975467b3d4791bd2b980668e44b","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"bool","enabled":false,"fused_primitives":1,"sha256":"bfb9b4ade86c3d6e244a876d1492cd66ac37b9d530fa915ad811bff09ca61f94","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"bool","enabled":true,"fused_primitives":1,"sha256":"bfb9b4ade86c3d6e244a876d1492cd66ac37b9d530fa915ad811bff09ca61f94","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"add","enabled":false,"fused_primitives":1,"sha256":"f9dc581dad8afef430d330240e590bc6593f5cc81486f60fc5919d02959aa456","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"add","enabled":true,"fused_primitives":1,"sha256":"f9dc581dad8afef430d330240e590bc6593f5cc81486f60fc5919d02959aa456","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"heads","enabled":false,"fused_primitives":1,"sha256":"1436e03ed34cdb12c713062dc228767cc7100173b31d2e7537902278aa4130cb","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"heads","enabled":true,"fused_primitives":1,"sha256":"1436e03ed34cdb12c713062dc228767cc7100173b31d2e7537902278aa4130cb","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,2,65) and (1,24,2,65) cannot be broadcast.","calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,2,65) and (1,24,2,65) cannot be broadcast.","calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"two_dim","enabled":false,"fused_primitives":1,"sha256":"bfb9b4ade86c3d6e244a876d1492cd66ac37b9d530fa915ad811bff09ca61f94","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":2,"total":65,"mode":"two_dim","enabled":true,"fused_primitives":1,"sha256":"bfb9b4ade86c3d6e244a876d1492cd66ac37b9d530fa915ad811bff09ca61f94","shape":[1,24,2,256],"calls":[{"q":[1,24,2,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[2,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"23aec22c4a9b071f3f68b898cdd0003ffbc6781afc4ebb6e83701cafe21f4ead","shape":[1,24,3,256],"calls":[{"q":[1,24,3,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"none","enabled":true,"fused_primitives":4,"sha256":"255e6e29e2ffd7c36f8b65f1bab4b5ebf2e9e9dff4e55cf9797e00153e085e40","shape":[1,24,3,256],"calls":[{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"c07246da70d7cd70ec4c94287fd42800d8382fcfcbc39910c896e8d10db444bc","shape":[1,24,3,256],"calls":[{"q":[1,24,3,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"bool","enabled":true,"fused_primitives":4,"sha256":"3e5aa9bc711caec5110dbf404c27d5682655be4116cee5bb68d262365329e7e4","shape":[1,24,3,256],"calls":[{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,3,65]},{"q":[1,2,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,3,65]},{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,3,65]},{"q":[1,2,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"88c445380d3618ae4e5d9c75654c81f064c5f0f60d0faec6214861f3a69eeba2","shape":[1,24,3,256],"calls":[{"q":[1,24,3,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"add","enabled":true,"fused_primitives":4,"sha256":"b06da381759f2fcffe7376f5c94e43b978bb000db10a76871ea4729fc39be65f","shape":[1,24,3,256],"calls":[{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,3,65]},{"q":[1,2,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,3,65]},{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,3,65]},{"q":[1,2,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"edf79cc764671c43548c122d4b8d00fac202b409e363e639433748a535651007","shape":[1,24,3,256],"calls":[{"q":[1,24,3,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,24,3,65) and (1,10,3,65) cannot be broadcast.","calls":[{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,24,3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,3,65) and (1,24,3,65) cannot be broadcast.","calls":[{"q":[1,24,3,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,3,65) and (1,10,3,65) cannot be broadcast.","calls":[{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"c07246da70d7cd70ec4c94287fd42800d8382fcfcbc39910c896e8d10db444bc","shape":[1,24,3,256],"calls":[{"q":[1,24,3,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":3,"total":65,"mode":"two_dim","enabled":true,"fused_primitives":4,"sha256":"3e5aa9bc711caec5110dbf404c27d5682655be4116cee5bb68d262365329e7e4","shape":[1,24,3,256],"calls":[{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[3,65]},{"q":[1,2,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[3,65]},{"q":[1,10,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[3,65]},{"q":[1,2,3,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[3,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"6dd129041c2d3d5b9e463e099eaf055e6cc224e7f399f460eab9b90507e877eb","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"none","enabled":true,"fused_primitives":4,"sha256":"d7e25ac9e5acaa066bb39cb5f4d44ce66e6ed9ecf65cd6739e0e85c5e9e07f5c","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,4,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,4,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"defeb30d1a00e48c77964849f103e8581620ef3eb2eab50c1d8c4941ac14da9c","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"bool","enabled":true,"fused_primitives":4,"sha256":"653936b24faea1777b8a2d60142b15976b4a9dd07073f0632cafd3d274c3d1ba","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,4,65]},{"q":[1,4,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,4,65]},{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,4,65]},{"q":[1,4,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"b2d6984d7745d029ef944c1b71b117be4f65c942d75ff856b897f40ffde786ca","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"add","enabled":true,"fused_primitives":4,"sha256":"5c141d52df37b7e3b69d29fcafefa01590b886df719814a4b5394cbbcafb5107","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,4,65]},{"q":[1,4,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,4,65]},{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,4,65]},{"q":[1,4,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"5c751a3a29dfc238be07c3f5e42b27f0f413d82391ec75a02c6e0436c1743b31","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,24,4,65) and (1,8,4,65) cannot be broadcast.","calls":[{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,24,4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,4,65) and (1,24,4,65) cannot be broadcast.","calls":[{"q":[1,24,4,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,4,65) and (1,8,4,65) cannot be broadcast.","calls":[{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"defeb30d1a00e48c77964849f103e8581620ef3eb2eab50c1d8c4941ac14da9c","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":65,"mode":"two_dim","enabled":true,"fused_primitives":4,"sha256":"653936b24faea1777b8a2d60142b15976b4a9dd07073f0632cafd3d274c3d1ba","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[4,65]},{"q":[1,4,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[4,65]},{"q":[1,8,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[4,65]},{"q":[1,4,4,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[4,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"a5fc56a33e7aa0ef2491a3412ad781255cc70f5287d91f4a822b224c8c61a4e1","shape":[1,24,8,256],"calls":[{"q":[1,24,8,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"none","enabled":true,"fused_primitives":6,"sha256":"460a99fb105fb36cacede40c92566b1db999661cf69f2f515eab665180a7a604","shape":[1,24,8,256],"calls":[{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"bf9e1c78c65ad8840d66979f2d6b9d3d2dcbdf21915fe4e2b811f8f9011f9343","shape":[1,24,8,256],"calls":[{"q":[1,24,8,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"bool","enabled":true,"fused_primitives":6,"sha256":"cc94951369563aa234c01eeff23ac5f33feb06ce72350b5d2f8e8e0f720ec5bf","shape":[1,24,8,256],"calls":[{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"bf49bbc6059b1adc89117f1a44efdbc36f71567b416e73a2298de5e5d806f5bd","shape":[1,24,8,256],"calls":[{"q":[1,24,8,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"add","enabled":true,"fused_primitives":6,"sha256":"27e7a1bcca8d43d69767e04e01b790684f4a38278686876a7c04a06b6b7c3431","shape":[1,24,8,256],"calls":[{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"d90beb996ec414e06f0ce6236c0f5b6bd0014f23223e51d55a145928d9fc2db1","shape":[1,24,8,256],"calls":[{"q":[1,24,8,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,24,8,65) and (1,4,8,65) cannot be broadcast.","calls":[{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,24,8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,8,65) and (1,24,8,65) cannot be broadcast.","calls":[{"q":[1,24,8,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"local","enabled":true,"fused_primitives":6,"sha256":"9867b0ea57370a7922fae310ddfea235a77113a889f81defaddd620a6cc3c928","shape":[1,24,8,256],"calls":[{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"bf9e1c78c65ad8840d66979f2d6b9d3d2dcbdf21915fe4e2b811f8f9011f9343","shape":[1,24,8,256],"calls":[{"q":[1,24,8,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":8,"total":65,"mode":"two_dim","enabled":true,"fused_primitives":6,"sha256":"cc94951369563aa234c01eeff23ac5f33feb06ce72350b5d2f8e8e0f720ec5bf","shape":[1,24,8,256],"calls":[{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[8,65]},{"q":[1,4,8,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[8,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"8462161a75cc9d55073f1293f3ffe3f0444e6aedaf66aea875450c80b7949108","shape":[1,24,9,256],"calls":[{"q":[1,24,9,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"none","enabled":true,"fused_primitives":0,"sha256":"8462161a75cc9d55073f1293f3ffe3f0444e6aedaf66aea875450c80b7949108","shape":[1,24,9,256],"calls":[{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"c2d8ef7c9c200b42b2c791a2efd02a07eed8784d426cb7a1c0270154c33df1a4","shape":[1,24,9,256],"calls":[{"q":[1,24,9,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"bool","enabled":true,"fused_primitives":0,"sha256":"c2d8ef7c9c200b42b2c791a2efd02a07eed8784d426cb7a1c0270154c33df1a4","shape":[1,24,9,256],"calls":[{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"63543217d76cfa29e395e772288912fd3f5b3a580460b0c579af16afb8484233","shape":[1,24,9,256],"calls":[{"q":[1,24,9,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"add","enabled":true,"fused_primitives":0,"sha256":"63543217d76cfa29e395e772288912fd3f5b3a580460b0c579af16afb8484233","shape":[1,24,9,256],"calls":[{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"e9b44cc48f22152a0d99119ae637b55d5515842104693d78bc35d5ef1171aa4e","shape":[1,24,9,256],"calls":[{"q":[1,24,9,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,24,9,65) and (1,3,9,65) cannot be broadcast.","calls":[{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,24,9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,9,65) and (1,24,9,65) cannot be broadcast.","calls":[{"q":[1,24,9,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,9,65) and (1,3,9,65) cannot be broadcast.","calls":[{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"c2d8ef7c9c200b42b2c791a2efd02a07eed8784d426cb7a1c0270154c33df1a4","shape":[1,24,9,256],"calls":[{"q":[1,24,9,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":9,"total":65,"mode":"two_dim","enabled":true,"fused_primitives":0,"sha256":"c2d8ef7c9c200b42b2c791a2efd02a07eed8784d426cb7a1c0270154c33df1a4","shape":[1,24,9,256],"calls":[{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[9,65]},{"q":[1,3,9,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[9,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"a7968b708ea023727a8b9c931ccaa8bf92b0080a1f998ef9de9aebbc3dbb1f9e","shape":[1,24,16,256],"calls":[{"q":[1,24,16,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"none","enabled":true,"fused_primitives":0,"sha256":"a7968b708ea023727a8b9c931ccaa8bf92b0080a1f998ef9de9aebbc3dbb1f9e","shape":[1,24,16,256],"calls":[{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"3a589c35762acaa0c4746f59ed34cb579073733a10173a68d6afe20b849496db","shape":[1,24,16,256],"calls":[{"q":[1,24,16,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"bool","enabled":true,"fused_primitives":0,"sha256":"3a589c35762acaa0c4746f59ed34cb579073733a10173a68d6afe20b849496db","shape":[1,24,16,256],"calls":[{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"580ec6d25eb2e97e58590bcc8cec04274f1e16bb63807af64662a5d05668f460","shape":[1,24,16,256],"calls":[{"q":[1,24,16,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"add","enabled":true,"fused_primitives":0,"sha256":"580ec6d25eb2e97e58590bcc8cec04274f1e16bb63807af64662a5d05668f460","shape":[1,24,16,256],"calls":[{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"58bbb8260c05c9df28a262903b7c672ac600d25e48394bff17bfb5a4817d5bf8","shape":[1,24,16,256],"calls":[{"q":[1,24,16,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,24,16,65) and (1,2,16,65) cannot be broadcast.","calls":[{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,24,16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,16,65) and (1,24,16,65) cannot be broadcast.","calls":[{"q":[1,24,16,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,16,65) and (1,2,16,65) cannot be broadcast.","calls":[{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"3a589c35762acaa0c4746f59ed34cb579073733a10173a68d6afe20b849496db","shape":[1,24,16,256],"calls":[{"q":[1,24,16,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":16,"total":65,"mode":"two_dim","enabled":true,"fused_primitives":0,"sha256":"3a589c35762acaa0c4746f59ed34cb579073733a10173a68d6afe20b849496db","shape":[1,24,16,256],"calls":[{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]},{"q":[1,2,16,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[16,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"21922541613e65fbca7591b72d5c629c90e3b6d17daf935b66ce58a62f5bccc8","shape":[1,24,32,256],"calls":[{"q":[1,24,32,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"none","enabled":true,"fused_primitives":0,"sha256":"21922541613e65fbca7591b72d5c629c90e3b6d17daf935b66ce58a62f5bccc8","shape":[1,24,32,256],"calls":[{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"f1dfe9ab25a67108baf404a2be4ad13fd4e43da3582223cf5d92ba4c03444612","shape":[1,24,32,256],"calls":[{"q":[1,24,32,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"bool","enabled":true,"fused_primitives":0,"sha256":"f1dfe9ab25a67108baf404a2be4ad13fd4e43da3582223cf5d92ba4c03444612","shape":[1,24,32,256],"calls":[{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"724b38797dabc3d2544fd617ebbdb9fe514abc05ec37827c5c352f63175cd30f","shape":[1,24,32,256],"calls":[{"q":[1,24,32,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"add","enabled":true,"fused_primitives":0,"sha256":"724b38797dabc3d2544fd617ebbdb9fe514abc05ec37827c5c352f63175cd30f","shape":[1,24,32,256],"calls":[{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,1,32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"744037fd36cf35ebcaf2f4c5789f581948a23555757e34602100ef1b549c6331","shape":[1,24,32,256],"calls":[{"q":[1,24,32,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"heads","enabled":true,"error":"Cannot broadcast array of shape (1,24,32,65) into shape (1,1,32,65).","calls":[{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,24,32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,32,65) and (1,24,32,65) cannot be broadcast.","calls":[{"q":[1,24,32,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"local","enabled":true,"error":"Cannot broadcast array of shape (1,4,32,65) into shape (1,1,32,65).","calls":[{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[1,4,32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"f1dfe9ab25a67108baf404a2be4ad13fd4e43da3582223cf5d92ba4c03444612","shape":[1,24,32,256],"calls":[{"q":[1,24,32,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":32,"total":65,"mode":"two_dim","enabled":true,"fused_primitives":0,"sha256":"f1dfe9ab25a67108baf404a2be4ad13fd4e43da3582223cf5d92ba4c03444612","shape":[1,24,32,256],"calls":[{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]},{"q":[1,1,32,256],"k":[1,1,65,256],"v":[1,1,65,256],"mask":[32,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"a23070399be2b7f7ac466986549551792e60de8811b8737ff96ae5b8a7949d03","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"none","enabled":true,"fused_primitives":0,"sha256":"a23070399be2b7f7ac466986549551792e60de8811b8737ff96ae5b8a7949d03","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"2cbbbd8a49d7514ba43eab2ab2532043e13387bdff7e8c2379d9a06485bcf93d","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,33,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"bool","enabled":true,"fused_primitives":0,"sha256":"2cbbbd8a49d7514ba43eab2ab2532043e13387bdff7e8c2379d9a06485bcf93d","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,33,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"b84378926aa673fda24c46f231d3c6597509f52a34ba5b4f116078f4635b23ab","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,33,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"add","enabled":true,"fused_primitives":0,"sha256":"b84378926aa673fda24c46f231d3c6597509f52a34ba5b4f116078f4635b23ab","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,1,33,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"5fe96745350d0d8aa29e40d67af8e8a5b6376170b379e60d94b010161344462d","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,33,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"heads","enabled":true,"fused_primitives":0,"sha256":"5fe96745350d0d8aa29e40d67af8e8a5b6376170b379e60d94b010161344462d","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,24,33,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,33,65) and (1,24,33,65) cannot be broadcast.","calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,33,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,33,65) and (1,24,33,65) cannot be broadcast.","calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[1,4,33,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"2cbbbd8a49d7514ba43eab2ab2532043e13387bdff7e8c2379d9a06485bcf93d","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[33,65]}]},{"batch":1,"heads":24,"kv":2,"rows":33,"total":65,"mode":"two_dim","enabled":true,"fused_primitives":0,"sha256":"2cbbbd8a49d7514ba43eab2ab2532043e13387bdff7e8c2379d9a06485bcf93d","shape":[1,24,33,256],"calls":[{"q":[1,24,33,256],"k":[1,2,65,256],"v":[1,2,65,256],"mask":[33,65]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"6d4f0b5f3eca5552d0cd276983e693a8e82de4d11963bb3405a86711bace2604","shape":[2,24,4,256],"calls":[{"q":[2,24,4,256],"k":[2,2,17,256],"v":[2,2,17,256],"mask":null}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"none","enabled":true,"fused_primitives":4,"sha256":"bf4d9268ebddd31551c89d88682dbabf38020bae709facb53677bff4d32f2989","shape":[2,24,4,256],"calls":[{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":null},{"q":[2,4,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":null},{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":null},{"q":[2,4,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":null}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"cdd8edcc3e0c44922e824838b3023315b089c8a972711318f919c98c5c9b0c6b","shape":[2,24,4,256],"calls":[{"q":[2,24,4,256],"k":[2,2,17,256],"v":[2,2,17,256],"mask":[1,1,4,17]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"bool","enabled":true,"fused_primitives":4,"sha256":"c3365af829ccc7c875f2f07a1db9caf5799195617de4771299d2252c94b65929","shape":[2,24,4,256],"calls":[{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,1,4,17]},{"q":[2,4,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,1,4,17]},{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,1,4,17]},{"q":[2,4,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,1,4,17]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"72d3e9619fb92e6cd02172851718826598250a6f1e1fc639574dff77ffbef04a","shape":[2,24,4,256],"calls":[{"q":[2,24,4,256],"k":[2,2,17,256],"v":[2,2,17,256],"mask":[1,1,4,17]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"add","enabled":true,"fused_primitives":4,"sha256":"4da6828e970632e921b5f421aaa36ba4105d945801df29e593e3949fddf26afe","shape":[2,24,4,256],"calls":[{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,1,4,17]},{"q":[2,4,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,1,4,17]},{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,1,4,17]},{"q":[2,4,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,1,4,17]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"eb92d8436d484eaa7d11898be87c4cef0cf29e267b0f64e5c8a82a367993b8af","shape":[2,24,4,256],"calls":[{"q":[2,24,4,256],"k":[2,2,17,256],"v":[2,2,17,256],"mask":[1,24,4,17]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,24,4,17) and (2,8,4,17) cannot be broadcast.","calls":[{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,24,4,17]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,4,17) and (2,24,4,17) cannot be broadcast.","calls":[{"q":[2,24,4,256],"k":[2,2,17,256],"v":[2,2,17,256],"mask":[1,4,4,17]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,4,17) and (2,8,4,17) cannot be broadcast.","calls":[{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[1,4,4,17]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"cdd8edcc3e0c44922e824838b3023315b089c8a972711318f919c98c5c9b0c6b","shape":[2,24,4,256],"calls":[{"q":[2,24,4,256],"k":[2,2,17,256],"v":[2,2,17,256],"mask":[4,17]}]},{"batch":2,"heads":24,"kv":2,"rows":4,"total":17,"mode":"two_dim","enabled":true,"fused_primitives":4,"sha256":"c3365af829ccc7c875f2f07a1db9caf5799195617de4771299d2252c94b65929","shape":[2,24,4,256],"calls":[{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[4,17]},{"q":[2,4,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[4,17]},{"q":[2,8,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[4,17]},{"q":[2,4,4,256],"k":[2,1,17,256],"v":[2,1,17,256],"mask":[4,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"5388dffa3c4a3c23a6011c615ed1ef493612a161d73a45ee0874e85ba692a093","shape":[1,64,1,256],"calls":[{"q":[1,64,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":null}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"none","enabled":true,"fused_primitives":2,"sha256":"fd7d194718bf5a3847cb47ea0dbc1327e1edb08b99ba4416f001dc8aff235b4c","shape":[1,64,1,256],"calls":[{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":null},{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":null}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"b184e7fa8e0058d3eebc51921147b7d2972da367d18bfbba035c54efbd9eae0c","shape":[1,64,1,256],"calls":[{"q":[1,64,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,1,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"bool","enabled":true,"fused_primitives":2,"sha256":"9280bb2d418a4fd58666670cbaa1d74e405431d386b66ddbf81882d911b85a0c","shape":[1,64,1,256],"calls":[{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,1,17]},{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,1,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"c269091e61a5de720d674c4b5fbb50ab2208d0b4ff7d44de8728f92454868251","shape":[1,64,1,256],"calls":[{"q":[1,64,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,1,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"add","enabled":true,"fused_primitives":2,"sha256":"d8463a24126df751474d5a4f093a33950c8f8957eb15e886ffe53f44b82fe6a8","shape":[1,64,1,256],"calls":[{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,1,17]},{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,1,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"16a295dcbb0b0ea59297c8cee02832f2e2061ab6bfcffe92c4c5b959a65a2e96","shape":[1,64,1,256],"calls":[{"q":[1,64,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,64,1,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,64,1,17) and (1,32,1,17) cannot be broadcast.","calls":[{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,64,1,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,1,17) and (1,64,1,17) cannot be broadcast.","calls":[{"q":[1,64,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,4,1,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,1,17) and (1,32,1,17) cannot be broadcast.","calls":[{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,4,1,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"b184e7fa8e0058d3eebc51921147b7d2972da367d18bfbba035c54efbd9eae0c","shape":[1,64,1,256],"calls":[{"q":[1,64,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,17]}]},{"batch":1,"heads":64,"kv":1,"rows":1,"total":17,"mode":"two_dim","enabled":true,"fused_primitives":2,"sha256":"9280bb2d418a4fd58666670cbaa1d74e405431d386b66ddbf81882d911b85a0c","shape":[1,64,1,256],"calls":[{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,17]},{"q":[1,32,1,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"2e31aae2377ebbf08014d4e0c44531970f3d91b1acb85479ec283d824cc7e57c","shape":[1,14,5,256],"calls":[{"q":[1,14,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":null}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"none","enabled":true,"fused_primitives":4,"sha256":"803045579d411990fce804bf90d6beb915d7598d785d5e8d93a28e3aff2d2c93","shape":[1,14,5,256],"calls":[{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":null},{"q":[1,1,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":null},{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":null},{"q":[1,1,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":null}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"2aaf90c29c51a3f91da4065d1e5793d62154f80917287d6faee5e6cf09c4d9fa","shape":[1,14,5,256],"calls":[{"q":[1,14,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,5,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"bool","enabled":true,"fused_primitives":4,"sha256":"b1e5a6df8c160eb2c5d605e26eba91a8e6de80482cca0b5fa2a00ff3fab29d66","shape":[1,14,5,256],"calls":[{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,5,17]},{"q":[1,1,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,5,17]},{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,5,17]},{"q":[1,1,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,5,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"62ce03d01e008b8eb4e9cef613d025e84dcebee3f8a6fe300c0075fec0b7b790","shape":[1,14,5,256],"calls":[{"q":[1,14,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,5,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"add","enabled":true,"fused_primitives":4,"sha256":"539b12bb24dbb2ab55428c4290f12bf8e9484bcba1b1110c31a06d6c9370ff70","shape":[1,14,5,256],"calls":[{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,5,17]},{"q":[1,1,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,5,17]},{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,5,17]},{"q":[1,1,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,5,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"c9d291f11f9b3384b01bca322cf2ebbf5baab5d0f45cab8554eb94f078ed2b46","shape":[1,14,5,256],"calls":[{"q":[1,14,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,14,5,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,14,5,17) and (1,6,5,17) cannot be broadcast.","calls":[{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,14,5,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,5,17) and (1,14,5,17) cannot be broadcast.","calls":[{"q":[1,14,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,4,5,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,5,17) and (1,6,5,17) cannot be broadcast.","calls":[{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,4,5,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"2aaf90c29c51a3f91da4065d1e5793d62154f80917287d6faee5e6cf09c4d9fa","shape":[1,14,5,256],"calls":[{"q":[1,14,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[5,17]}]},{"batch":1,"heads":14,"kv":2,"rows":5,"total":17,"mode":"two_dim","enabled":true,"fused_primitives":4,"sha256":"b1e5a6df8c160eb2c5d605e26eba91a8e6de80482cca0b5fa2a00ff3fab29d66","shape":[1,14,5,256],"calls":[{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[5,17]},{"q":[1,1,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[5,17]},{"q":[1,6,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[5,17]},{"q":[1,1,5,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"none","enabled":false,"fused_primitives":1,"sha256":"d8b8c37d52a674aeab3f3ae671952ad973f0276c90e15c98a58e4bc51eda7575","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":null}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"none","enabled":true,"fused_primitives":1,"sha256":"d8b8c37d52a674aeab3f3ae671952ad973f0276c90e15c98a58e4bc51eda7575","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":null}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"bool","enabled":false,"fused_primitives":1,"sha256":"c48850d4a4ae1801ade250bae92bb2d6cf9bde3505b4e77aa437edba42b603f0","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"bool","enabled":true,"fused_primitives":1,"sha256":"c48850d4a4ae1801ade250bae92bb2d6cf9bde3505b4e77aa437edba42b603f0","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"add","enabled":false,"fused_primitives":1,"sha256":"e09f1cef51c10ce34deaf06b31d20c07527edd1b2f2e7e62612d02b037fbbcca","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"add","enabled":true,"fused_primitives":1,"sha256":"e09f1cef51c10ce34deaf06b31d20c07527edd1b2f2e7e62612d02b037fbbcca","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"heads","enabled":false,"fused_primitives":1,"sha256":"0f5d4ef3eb21018e7fc23d4912af571790d26248e121874a139ca25291248e48","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,12,5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"heads","enabled":true,"fused_primitives":1,"sha256":"0f5d4ef3eb21018e7fc23d4912af571790d26248e121874a139ca25291248e48","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,12,5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,5,17) and (1,12,5,17) cannot be broadcast.","calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,4,5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,5,17) and (1,12,5,17) cannot be broadcast.","calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,4,5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"two_dim","enabled":false,"fused_primitives":1,"sha256":"c48850d4a4ae1801ade250bae92bb2d6cf9bde3505b4e77aa437edba42b603f0","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[5,17]}]},{"batch":1,"heads":12,"kv":2,"rows":5,"total":17,"mode":"two_dim","enabled":true,"fused_primitives":1,"sha256":"c48850d4a4ae1801ade250bae92bb2d6cf9bde3505b4e77aa437edba42b603f0","shape":[1,12,5,256],"calls":[{"q":[1,12,5,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[5,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"none","enabled":false,"fused_primitives":1,"sha256":"3fccad2f3146a9c6fac1f433a24a43d3ad9fa96bef2ced5689586edd054d8d3c","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":null}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"none","enabled":true,"fused_primitives":1,"sha256":"3fccad2f3146a9c6fac1f433a24a43d3ad9fa96bef2ced5689586edd054d8d3c","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":null}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"bool","enabled":false,"fused_primitives":1,"sha256":"ae02ac3a0596eeb604d53f48955939b5a57bbc620acd4c7a35e41ef0d00d4ac1","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"bool","enabled":true,"fused_primitives":1,"sha256":"ae02ac3a0596eeb604d53f48955939b5a57bbc620acd4c7a35e41ef0d00d4ac1","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"add","enabled":false,"fused_primitives":1,"sha256":"9f3ed8ccd4d5ec8b3eb62d75afddbabb47c34736e7fb41a6220b8ba02da02d13","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"add","enabled":true,"fused_primitives":1,"sha256":"9f3ed8ccd4d5ec8b3eb62d75afddbabb47c34736e7fb41a6220b8ba02da02d13","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"heads","enabled":false,"fused_primitives":1,"sha256":"ae02ac3a0596eeb604d53f48955939b5a57bbc620acd4c7a35e41ef0d00d4ac1","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"heads","enabled":true,"fused_primitives":1,"sha256":"ae02ac3a0596eeb604d53f48955939b5a57bbc620acd4c7a35e41ef0d00d4ac1","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"local","enabled":false,"error":"Cannot broadcast array of shape (1,4,4,17) into shape (1,1,4,17).","calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,4,4,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"local","enabled":true,"error":"Cannot broadcast array of shape (1,4,4,17) into shape (1,1,4,17).","calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[1,4,4,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"two_dim","enabled":false,"fused_primitives":1,"sha256":"ae02ac3a0596eeb604d53f48955939b5a57bbc620acd4c7a35e41ef0d00d4ac1","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[4,17]}]},{"batch":1,"heads":1,"kv":1,"rows":4,"total":17,"mode":"two_dim","enabled":true,"fused_primitives":1,"sha256":"ae02ac3a0596eeb604d53f48955939b5a57bbc620acd4c7a35e41ef0d00d4ac1","shape":[1,1,4,256],"calls":[{"q":[1,1,4,256],"k":[1,1,17,256],"v":[1,1,17,256],"mask":[4,17]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"dc729500dc8a68f06fa5346083c445191640b7acf2538b17270f8e4dadb2cdc8","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,2,256],"v":[1,2,2,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"none","enabled":true,"fused_primitives":0,"sha256":"dc729500dc8a68f06fa5346083c445191640b7acf2538b17270f8e4dadb2cdc8","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":null},{"q":[1,4,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":null},{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":null},{"q":[1,4,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"37ccffe1b48c96ecc32ae036a9359fa71d8959c832affeb7f2a89dd85bfded29","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,2,256],"v":[1,2,2,256],"mask":[1,1,4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"bool","enabled":true,"fused_primitives":0,"sha256":"37ccffe1b48c96ecc32ae036a9359fa71d8959c832affeb7f2a89dd85bfded29","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,1,4,2]},{"q":[1,4,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,1,4,2]},{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,1,4,2]},{"q":[1,4,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,1,4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"0bb33a928006437520d2b72a45594ef906e0340b9ffab13aeb827edd9c4f30db","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,2,256],"v":[1,2,2,256],"mask":[1,1,4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"add","enabled":true,"fused_primitives":0,"sha256":"0bb33a928006437520d2b72a45594ef906e0340b9ffab13aeb827edd9c4f30db","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,1,4,2]},{"q":[1,4,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,1,4,2]},{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,1,4,2]},{"q":[1,4,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,1,4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"afc5f68a25ddcb5a1d44fe72219614e94527bd3502ca68de065ed8816eac9b9f","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,2,256],"v":[1,2,2,256],"mask":[1,24,4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,24,4,2) and (1,8,4,2) cannot be broadcast.","calls":[{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,24,4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,4,2) and (1,24,4,2) cannot be broadcast.","calls":[{"q":[1,24,4,256],"k":[1,2,2,256],"v":[1,2,2,256],"mask":[1,4,4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,4,2) and (1,8,4,2) cannot be broadcast.","calls":[{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[1,4,4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"37ccffe1b48c96ecc32ae036a9359fa71d8959c832affeb7f2a89dd85bfded29","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,2,256],"v":[1,2,2,256],"mask":[4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":2,"mode":"two_dim","enabled":true,"fused_primitives":0,"sha256":"37ccffe1b48c96ecc32ae036a9359fa71d8959c832affeb7f2a89dd85bfded29","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[4,2]},{"q":[1,4,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[4,2]},{"q":[1,8,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[4,2]},{"q":[1,4,4,256],"k":[1,1,2,256],"v":[1,1,2,256],"mask":[4,2]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"46ba42eddc8ce47f67d48d26e8b5c4d40edad2a0d077741b057b016d6669ccf1","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,1024,256],"v":[1,2,1024,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"none","enabled":true,"fused_primitives":4,"sha256":"2377053d4f1104b7cf88566a5e888bc2c7f129b19c208b0c89d84c9448a61c46","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":null},{"q":[1,4,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":null},{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":null},{"q":[1,4,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"5f6e592dbfb87906de746d9a03c4fd0e01f1caa6a269be3fcaddc3719c1e1890","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,1024,256],"v":[1,2,1024,256],"mask":[1,1,4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"bool","enabled":true,"fused_primitives":4,"sha256":"7f4dbd3acd492928b7d4253aa3b98b64346f5e8b9d3933cb66f0f8f6e32900d0","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,1,4,1024]},{"q":[1,4,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,1,4,1024]},{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,1,4,1024]},{"q":[1,4,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,1,4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"30ff1e5a6c339a6eeaad483c6d4d95ac91924250c58b16dde2cb13be50df34fd","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,1024,256],"v":[1,2,1024,256],"mask":[1,1,4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"add","enabled":true,"fused_primitives":4,"sha256":"d9c1b4a4577dff07ea5084b77515c8865bd353b3e28562d8192bb5061d362efe","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,1,4,1024]},{"q":[1,4,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,1,4,1024]},{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,1,4,1024]},{"q":[1,4,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,1,4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"d0635700d416de1e6d66a48a1395cac01f82a6e8f702ce7ffd1abc5b898f1e47","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,1024,256],"v":[1,2,1024,256],"mask":[1,24,4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,24,4,1024) and (1,8,4,1024) cannot be broadcast.","calls":[{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,24,4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,4,1024) and (1,24,4,1024) cannot be broadcast.","calls":[{"q":[1,24,4,256],"k":[1,2,1024,256],"v":[1,2,1024,256],"mask":[1,4,4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,4,1024) and (1,8,4,1024) cannot be broadcast.","calls":[{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[1,4,4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"5f6e592dbfb87906de746d9a03c4fd0e01f1caa6a269be3fcaddc3719c1e1890","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,1024,256],"v":[1,2,1024,256],"mask":[4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":1024,"mode":"two_dim","enabled":true,"fused_primitives":4,"sha256":"7f4dbd3acd492928b7d4253aa3b98b64346f5e8b9d3933cb66f0f8f6e32900d0","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[4,1024]},{"q":[1,4,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[4,1024]},{"q":[1,8,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[4,1024]},{"q":[1,4,4,256],"k":[1,1,1024,256],"v":[1,1,1024,256],"mask":[4,1024]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"none","enabled":false,"fused_primitives":0,"sha256":"c9b4166aaa0cac5cc5c48668736a26b9b8554d8db5c0bf1603ac88cdd3298bea","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,4097,256],"v":[1,2,4097,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"none","enabled":true,"fused_primitives":4,"sha256":"82bb5bbfe25668aa7f94f75c0295b4f7ce839367f45214981c55b4e2341faa90","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":null},{"q":[1,4,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":null},{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":null},{"q":[1,4,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":null}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"bool","enabled":false,"fused_primitives":0,"sha256":"8f8c8fccb9d4e404616270c7f882c76a5da8c8d7a012e325b1c65f97d6a00a8d","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,4097,256],"v":[1,2,4097,256],"mask":[1,1,4,4097]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"bool","enabled":true,"fused_primitives":4,"sha256":"4a0fe947c36a4869d4eb82c4dc6f32bfeb79d54245f6a9e7d3c17d9000e68589","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,1,4,4097]},{"q":[1,4,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,1,4,4097]},{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,1,4,4097]},{"q":[1,4,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,1,4,4097]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"add","enabled":false,"fused_primitives":0,"sha256":"3800a0e9c3d0d18c08d8ce1b6c3c191f8de48a801fde31a101bef79d6d257c32","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,4097,256],"v":[1,2,4097,256],"mask":[1,1,4,4097]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"add","enabled":true,"fused_primitives":4,"sha256":"d32250c4803dd0967e94b6c1508c9f934aeddfc83f7cc8851718ab9477a87dc0","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,1,4,4097]},{"q":[1,4,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,1,4,4097]},{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,1,4,4097]},{"q":[1,4,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,1,4,4097]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"heads","enabled":false,"fused_primitives":0,"sha256":"4963c84777e3e7150f6207dbf2664e300480b515f8d0025b9e3c5fc31ba2ef35","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,4097,256],"v":[1,2,4097,256],"mask":[1,24,4,4097]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"heads","enabled":true,"error":"[broadcast_shapes] Shapes (1,24,4,4097) and (1,8,4,4097) cannot be broadcast.","calls":[{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,24,4,4097]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"local","enabled":false,"error":"[broadcast_shapes] Shapes (1,4,4,4097) and (1,24,4,4097) cannot be broadcast.","calls":[{"q":[1,24,4,256],"k":[1,2,4097,256],"v":[1,2,4097,256],"mask":[1,4,4,4097]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"local","enabled":true,"error":"[broadcast_shapes] Shapes (1,4,4,4097) and (1,8,4,4097) cannot be broadcast.","calls":[{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[1,4,4,4097]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"two_dim","enabled":false,"fused_primitives":0,"sha256":"8f8c8fccb9d4e404616270c7f882c76a5da8c8d7a012e325b1c65f97d6a00a8d","shape":[1,24,4,256],"calls":[{"q":[1,24,4,256],"k":[1,2,4097,256],"v":[1,2,4097,256],"mask":[4,4097]}]},{"batch":1,"heads":24,"kv":2,"rows":4,"total":4097,"mode":"two_dim","enabled":true,"fused_primitives":4,"sha256":"4a0fe947c36a4869d4eb82c4dc6f32bfeb79d54245f6a9e7d3c17d9000e68589","shape":[1,24,4,256],"calls":[{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[4,4097]},{"q":[1,4,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[4,4097]},{"q":[1,8,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[4,4097]},{"q":[1,4,4,256],"k":[1,1,4097,256],"v":[1,1,4097,256],"mask":[4,4097]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"none","enabled":false,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":null}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"none","enabled":true,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":null}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"bool","enabled":false,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"bool","enabled":true,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"add","enabled":false,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"add","enabled":true,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,1,4,17]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"heads","enabled":false,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,5,4,17]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"heads","enabled":true,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,5,4,17]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"local","enabled":false,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,4,4,17]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"local","enabled":true,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[1,4,4,17]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"two_dim","enabled":false,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[4,17]}]},{"batch":1,"heads":5,"kv":2,"rows":4,"total":17,"mode":"two_dim","enabled":true,"error":"[scaled_dot_product_attention] n_heads must be a multiple of n_kv_heads, found n_heads 5 for n_kv_heads 2.","calls":[{"q":[1,5,4,256],"k":[1,2,17,256],"v":[1,2,17,256],"mask":[4,17]}]}]}
diff --git a/tools/issue113-model-reference.py b/tools/issue113-model-reference.py
new file mode 100644
index 0000000..1f39271
--- /dev/null
+++ b/tools/issue113-model-reference.py
@@ -0,0 +1,132 @@
+"""Strict default/off head-chunk oracle for real verify rows and matching bank snapshots."""
+import argparse,hashlib,json,os,subprocess,sys
+from pathlib import Path
+checkout,root,native,out=map(Path,sys.argv[1:5])
+assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
+sys.path.insert(0,str(checkout))
+from mtplx.server.openai import _server_runtime_env_overrides
+os.environ.update(_server_runtime_env_overrides(argparse.Namespace(model=str(root),generation_mode='mtp'),None))
+os.environ['MTPLX_SUSTAINED_PREFILL']='1'
+os.environ['MTPLX_PREFILL_CHUNK_SIZE']='2048'
+os.environ['MTPLX_CONTEXT_COPY']='0'
+import mlx.core as mx
+import numpy as np
+from mlx_lm.utils import load_model
+from mtplx.models import qwen4_exp as qwen
+from mtplx.runtime import _load_tokenizer_resilient,MTPLXRuntime
+from mtplx.mtp_patch import MTPContract
+from mtplx.generation import generate_mtpk
+from mtplx.sampling import SamplerConfig
+from mtplx.session_bank import SessionBank,_lazy_snapshot_enabled
+from mtplx.cache_state import snapshot_cache,snapshot_cache_lazy_hybrid
+from mtplx.attention_context import current_attention_phase
+out.mkdir(parents=True,exist_ok=True)
+print(json.dumps(dict(case='environment',overrides={k:v for k,v in os.environ.items() if k.startswith('MTPLX_')})),flush=True)
+tokenizer=_load_tokenizer_resilient(root,json.loads((root/'config.json').read_text()))
+model,_=load_model(root,lazy=False,strict=True,get_model_classes=lambda **_:(qwen.Model,qwen.ModelArgs))
+model.post_weight_load(root)
+assert model.attach_mtp(root)
+rt=MTPLXRuntime(model,tokenizer,root,True,MTPContract())
+phase='cold'; index=0; nested=0
+
+def wrap(fn):
+    def call(input_ids,*args,**kwargs):
+        global index,nested
+        outer=nested==0
+        nested+=1
+        try: result=fn(input_ids,*args,**kwargs)
+        finally: nested-=1
+        if outer and current_attention_phase() == 'prefill' and isinstance(result,tuple) and result[0] is not None and result[1] is not None:
+            cache=kwargs.get('cache',args[0] if args else None)
+            if cache is not None:
+                key=np.asarray((snapshot_cache_lazy_hybrid(cache) if _lazy_snapshot_enabled() else snapshot_cache(cache)).states[35][0].astype(mx.float32))
+                (out/f'{phase}-prefill-key35.f32').write_bytes(key.tobytes())
+            for part,a in zip(['logits','hidden'],result[:2]):
+                # The suffix owner retains only the final hidden row (generation.py:4689).
+                if part=='hidden':a=a[:,-1:,:]
+                data=np.asarray(a.astype(mx.float32)).tobytes()
+                (out/f'{phase}-prefill-{part}.f32').write_bytes(data)
+                path=native/f'{phase}-prefill-{part}.f32'
+                print(json.dumps(dict(case='prefill_compare',phase=phase,part=part,shape=list(a.shape),exact=path.exists() and path.read_bytes()==data)),flush=True)
+        if outer and current_attention_phase()=='decode_verify':
+            expected=json.loads((native/phase/f'{index}.json').read_text())
+            ids=np.asarray(input_ids).reshape(-1).tolist()
+            assert expected['verify'] and ids==expected['ids'], (phase,index,ids,expected['ids'])
+            receipt=dict(case='verify_forward',phase=phase,index=index,ids=ids)
+            for part,a in zip(['logits','hidden'],result[:2]):
+                data=np.asarray(a.astype(mx.float32)).tobytes()
+                (out/f'{phase}-{index}-{part}.f32').write_bytes(data)
+                ref=(native/phase/f'{index}-{part}.f32').read_bytes()
+                receipt[part+'_exact']=data==ref
+                receipt[part+'_sha256']=hashlib.sha256(data).hexdigest()
+                if data!=ref:
+                    x=np.frombuffer(data,np.float32); y=np.frombuffer(ref,np.float32)
+                    receipt[part+'_different']=int(np.count_nonzero(x!=y))
+                    receipt[part+'_max_abs']=float(np.max(np.abs(x-y)))
+            print(json.dumps(receipt),flush=True)
+            assert receipt['logits_exact'] and receipt['hidden_exact'],receipt
+            index+=1
+        return result
+    return call
+rt.forward_ar=wrap(rt.forward_ar)
+rt.forward_ar_capture=wrap(rt.forward_ar_capture)
+bank=SessionBank()
+for phase in ['cold','warm']:
+    index=0
+    expected=json.loads((native/f'{phase}.json').read_text())
+    ids=expected['prompt']
+    result=generate_mtpk(rt,ids,max_tokens=12,sampler=SamplerConfig(temperature=0.,top_p=.95,top_k=20),speculative_depth=3,
+        seed=1,stop_token_ids=set(tokenizer.eos_token_ids),mtp_history_policy='committed',verify_strategy='batched',capture_final_state=True,
+        session_bank=bank,session_id='issue113',store_prefix_snapshot=False,
+        token_callback=lambda ids:print(json.dumps(dict(case='tokens',phase=phase,ids=ids)),flush=True))
+    assert result.tokens==expected['tokens'],(result.tokens,expected['tokens'])
+    assert result.stats.cached_tokens==expected['cached'],(result.stats.cached_tokens,expected['cached'])
+    assert index==len(list((native/phase).glob('*.json')))
+    final=result.final_state
+    assert final.safe_to_commit
+    for part,a in [('logits',final.final_logits),('hidden',final.final_hidden)]:
+        data=np.asarray(a.astype(mx.float32)).tobytes()
+        assert data==(native/f'{phase}-{part}.f32').read_bytes(),(phase,part)
+    def states(cache,lazy=False):
+        def array(a):
+            if a is None: return None
+            values=np.asarray(a.astype(mx.float32)).copy()
+            bits=hashlib.sha256(values.tobytes()).hexdigest()
+            values[values==0]=0
+            return dict(shape=list(a.shape),sha256=bits,value_sha256=hashlib.sha256(values.tobytes()).hexdigest())
+        return [[array(a) for a in state] if state is not None else None for state in (snapshot_cache_lazy_hybrid(cache) if lazy else snapshot_cache(cache)).states]
+    arrays=dict(trunk=states(final.final_trunk_cache,_lazy_snapshot_enabled()),mtp=states(final.final_committed_mtp_cache))
+    (out/f'{phase}-arrays.json').write_text(json.dumps(arrays))
+    native_arrays=json.loads((native/f'{phase}-arrays.json').read_text())
+    zero_sign_differences=[]
+    for family in arrays:
+        assert len(arrays[family])==len(native_arrays[family])
+        for layer,(actual,want) in enumerate(zip(arrays[family],native_arrays[family])):
+            if actual is None or want is None:
+                assert actual==want
+                continue
+            assert len(actual)==len(want)
+            for slot,(a,b) in enumerate(zip(actual,want)):
+                if a is None or b is None:
+                    assert a==b
+                    continue
+                assert a['shape']==b['shape'] and a['value_sha256']==b['value_sha256'],(phase,family,layer,slot,a,b)
+                if a['sha256']!=b['sha256']: zero_sign_differences.append([family,layer,slot])
+    assert zero_sign_differences==[], zero_sign_differences
+    # Compare the same lazy trunk snapshot used by the server/native bank.
+    # An extra deep clone would add zero and change negative-zero bits.
+    key=np.asarray((snapshot_cache_lazy_hybrid(final.final_trunk_cache) if _lazy_snapshot_enabled() else snapshot_cache(final.final_trunk_cache)).states[35][0].astype(mx.float32))
+    (out/f'{phase}-key35.f32').write_bytes(key.tobytes())
+    want=np.fromfile(native/f'{phase}-key35.f32',np.float32).reshape(key.shape)
+    different=key.view(np.uint32)!=want.view(np.uint32)
+    assert np.all(key[different]==0) and np.all(want[different]==0)
+    assert not np.any(different)
+    for part in ["key35","logits","hidden"]:
+        assert (out/f"{phase}-prefill-{part}.f32").read_bytes()==(native/f"{phase}-prefill-{part}.f32").read_bytes(),(phase,part,"prefill differs")
+    print(json.dumps(dict(case='cache_arrays',phase=phase,values_exact=True,zero_sign_differences=zero_sign_differences,key35_bit_coordinates=np.argwhere(different).tolist())),flush=True)
+    head=snapshot_cache(final.final_committed_mtp_cache)
+    key=ids+result.tokens
+    assert bank.put(runtime=rt,token_ids=key,cache=final.final_trunk_cache,logits=final.final_logits,hidden=final.final_hidden,
+        hidden_variant='post_norm',session_id='issue113',mtp_history_policy='committed',mtp_history_snapshot=head,
+        snapshot_epoch=len(key),mtp_snapshot_epoch=len(key)) is not None
+    print(json.dumps(dict(case='turn',phase=phase,tokens=result.tokens,cached=result.stats.cached_tokens,verify_calls=index,final_exact=True)),flush=True)
diff --git a/tools/issue113-reference.py b/tools/issue113-reference.py
new file mode 100644
index 0000000..6b01c69
--- /dev/null
+++ b/tools/issue113-reference.py
@@ -0,0 +1,51 @@
+"""Pinned head-chunk plan, actual SDPA subcall shapes and exact BF16 outputs."""
+import hashlib,io,json,os,subprocess,sys
+from pathlib import Path
+checkout,out=map(Path,sys.argv[1:3])
+assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
+sys.path.insert(0,str(checkout))
+import mlx.core as mx
+import numpy as np
+from mtplx.models import qwen4_exp as qwen
+assert mx.__version__=='0.32.2'
+plans=[dict(rows=r,heads=h,kv=kv,plan=qwen._verify_sdpa_head_chunk_plan(r,h,kv)) for r in [-1,0,1,2,3,4,8,9,16,31,32,33,64] for h in [0,1,2,12,24,33,36,64] for kv in [0,1,2,3,5]]
+def pattern(shape,salt):
+ n=int(np.prod(shape));v=((np.arange(n,dtype=np.uint32)*17+salt*13)%257).astype(np.int32)-128
+ return mx.array((v.astype(np.float32)/128).reshape(shape)).astype(mx.bfloat16)
+geometries=[(1,24,2,r,65) for r in [1,2,3,4,8,9,16,32,33]]+[(2,24,2,4,17),(1,64,1,1,17),(1,14,2,5,17),(1,12,2,5,17),(1,1,1,4,17),(1,24,2,4,2),(1,24,2,4,1024),(1,24,2,4,4097),(1,5,2,4,17)]
+original=mx.fast.scaled_dot_product_attention
+calls=[]
+def traced(q,k,v,**kwargs):
+ calls.append(dict(q=list(q.shape),k=list(k.shape),v=list(v.shape),mask=None if kwargs['mask'] is None else list(kwargs['mask'].shape)))
+ return original(q,k,v,**kwargs)
+mx.fast.scaled_dot_product_attention=traced
+cases=[]
+for batch,heads,kv,rows,total in geometries:
+ for mode in ['none','bool','add','heads','local','two_dim']:
+  for enabled in [False,True]:
+   os.environ['MTPLX_QWEN4_VERIFY_SDPA_HEAD_CHUNK']='1' if enabled else '0'
+   calls.clear();case=dict(batch=batch,heads=heads,kv=kv,rows=rows,total=total,mode=mode,enabled=enabled)
+   q=pattern((batch,rows,heads,256),93).transpose(0,2,1,3)
+   k=pattern((batch,kv,total+7,256),94)[:,:,2:total+2]
+   v=pattern((batch,kv,total+7,256),95)[:,:,2:total+2]
+   mask=None
+   if mode!='none':
+    mh=heads if mode=='heads' else (4 if mode=='local' else 1)
+    mask=np.array([j<=total-rows+i and (j+h)%3!=1 and i!=1 for h in range(mh) for i in range(rows) for j in range(total)],dtype=np.bool_).reshape(1,mh,rows,total)
+    mask=mx.array(mask)
+    if mode=='add':mask=mx.where(mask,mx.array(0,dtype=mx.bfloat16),mx.array(-10,dtype=mx.bfloat16))
+    if mode=='two_dim':mask=mask.reshape(rows,total)
+   try:
+    y=qwen._verify_sdpa(q,k,v,scale=.0625,mask=mask)
+    dot=io.StringIO();mx.export_to_dot(dot,output=y)
+    case['fused_primitives']=dot.getvalue().count('ScaledDotProductAttention')
+    mx.eval(y)
+    case['sha256']=hashlib.sha256(np.asarray(y.view(mx.uint16)).tobytes()).hexdigest()
+    case['shape']=list(y.shape)
+   except ValueError as e:case['error']=str(e)
+   case['calls']=list(calls);cases.append(case)
+   del q,k,v,mask
+   mx.clear_cache()
+assert len(cases)==216
+out.write_text(json.dumps(dict(revision='9882703f3105363ddc37eca9f97aa09a1d387112',mlx=mx.__version__,plans=plans,cases=cases),separators=(',',':'))+'\n')
+print(json.dumps(dict(plans=len(plans),cases=len(cases),errors=sum('error' in c for c in cases),output=str(out))))