Preserve image-aware sparse QSA for Qwen vision

docs/issue107-vision-qsa/README.md

@@ -0,0 +1,104 @@
+# Issue 107: image-aware sparse QSA
+
+Qwen vision now retains the indexer's sparse selection. The indexer receives the
+request's position table and delta, rotates queries row by row and pooled keys at
+each block's first token, and uses the eager preparation route selected by pinned
+MTPLX `9882703f3105363ddc37eca9f97aa09a1d387112`. Its compiled/fused preparation
+is ineligible with image positions; the existing sparse selectors/consumers remain
+eligible. Budgets, thresholds, text preparation and attention projection RoPE are
+unchanged. Issue 108 still owns the attention projection's table-straddling case.
+
+The indexer's new helper implements `_vision_position_cos_sin` using the native
+array operations, including I32 positions, clipped table gather, per-row selection,
+axis gather, trigonometry and YaRN amplitude. Signed I32 Gather reuses the existing
+U32 payload-copy kernel: it performs no value arithmetic or float conversion.
+Tests include signed extrema on both front-axis and general gather routes.
+
+Image identities now include a numerical policy: 1 for image-aware sparse QSA,
+0 for the old dense policy (also MTPLX's explicit diagnostic rollback flag).
+Missing fields deserialize as 0. Old vision checkpoints are rejected before
+tensor materialization, obsolete live/resident owners are discarded, and stale
+owners cannot save a checkpoint. Identical image bytes and token spans do not
+bypass this policy. Text-only identities remain empty and compatible. The V1
+signature rejection delivered in issue 106 remains in force; this issue also
+rejects affected V2 vision files without invalidating unaffected V2 text files.
+
+Request prefix matching additionally rejects a restore ending strictly inside an
+image span, matching MTPLX's image-safe restore boundary. Complete image spans and
+preceding text prefixes remain eligible. Existing image digests continue to
+distinguish different pixels with identical image-pad token IDs.
+
+## Acceptance and review
+
+- The owned `vision_sparse_qsa` regression now requires equal sparse output for
+  text and a vision context with equal text positions. Separate issue 108/109
+  diagnostic assertions remain unchanged.
+- Real indexer geometry: four heads, dim128, ratio4, top-k512, 2,072 raw key rows,
+  518 pooled blocks. Query rows and block starts cross a 2,060-row position table.
+  Table, equal-axis, delta-only and empty-table cases match pinned MTPLX exactly
+  in F32 and BF16; independent NumPy math verifies the F32 values. Normal fused
+  selectors remain active while compiled preparation is correctly bypassed.
+- A real 2,095-token, two-image request exceeds the sparse budget. Corrected final
+  model logits match MTPLX byte-for-byte: SHA256
+  `8b97664316bec6113c85b7dfca9b09967391f019ea1064b83d81a3d9d5191354`.
+  Pre-fix logits differ (`a79aa02a98542adeea636f22c85fc5f929308dc5dc75658d4072e6f275d22abe`).
+- The actual pre-fix dense-vision checkpoint is 412,628,836 bytes, SHA256
+  `7ddc663d4f337ff272000a7237dac05a224262bf25bc88257e784547b2056326`.
+  Tests reject its normal disk load and import its original tensor payload only
+  through a test helper to verify stale live/resident rejection and save refusal.
+- Product tests cover unchanged images, identical complete-state restore/reopen
+  and follow-up, text edits before/between/after two images, different pixels with
+  identical pad IDs, cancellation/resend, a prefix ending inside an image, and
+  unaffected text checkpoints. Comparisons cover all serialized tensors and state
+  metadata, not only response text.
+- DS4/GLM do not call the modified Qwen modules. Their sampling, RNG, vision and
+  checkpoint formats are unchanged. This is MTPLX behavior plus DS4Server's
+  application-specific checkpoint policy integration.
+- The 12 affected whole-attention fixture rows were regenerated from the pinned
+  oracle (profiles 7 and 9, TF32 on/off). All 56 fixture rows pass with both lazy
+  and prepared inputs, including output/state hashes, consumer routes, compiled
+  core lifecycle and counters. Existing text-only indexer route fixtures pass.
+
+All four commit gates passed. The initial general test run hit an unrelated
+`dev_brain` fixture whose temporary short Git hash was all digits and therefore
+parsed as a YAML integer. Its focused retry and the subsequent full suite passed;
+no unrelated source change was included. `checks.json` records both outcomes.
+
+The tests and receipts are correctness evidence, not performance measurements.
+See `checks.json` and the accompanying JSONL files for actual run outcomes.
+
+## Reproduction
+
+Use the same unchanged Bare-Speed artifact and environment documented in
+`../issue104-ar/environment.json`, including the pinned vision tower. The input
+image is `local-eval-results/qwen-vision-input/image.png` (1024 square pixels),
+repeated twice; tests generate an independent 1024-square blue control.
+
+An isolated checkout of parent `a8cad88` plus `baseline-capture.patch` regenerates
+the actual old snapshot with `issue107_vision_product`. Set `DS4_ISSUE107_DIR` to
+its output directory. Do not replace a new file's header to fake old state.
+
+Generate fresh reference results in the isolated MLX 0.32.2 environment:
+
+```sh
+python tools/issue107-model-reference.py PINNED_MTPLX MODEL IMAGE ORACLE_DIR
+python tools/issue107-indexer-reference.py PINNED_MTPLX INDEXER_DIR
+```
+
+Build native tests with `cargo test --all-features --bin ds4-server --no-run`.
+Run ignored tests one at a time via `test-supervisor MEMORY_MIB 60 120 --command
+TEST_EXECUTABLE TEST_NAME --ignored --nocapture --test-threads=1`. Use 114688 MiB
+for the product test and 4096 MiB for model-free tests. Set
+`DS4_QWEN38_ARTIFACTS`, `DS4_QWEN_VISION_IMAGE`, `DS4_ISSUE107_DIR`,
+`DS4_ISSUE107_BAD_CHECKPOINT`, `DS4_ISSUE107_ORACLE_DIR`, and
+`DS4_ISSUE107_INDEXER` to the corresponding files/directories.
+
+Owned tests: `issue107_vision_product`,
+`issue107_real_geometry_query_and_block_start_positions`, and the
+`vision_sparse_qsa` case in `issue103_vision_position_receipts`.
+
+To refresh just the affected whole-attention receipts, set `PYTHONPATH` to the
+pinned checkout and run `qsa_attention_module_fixtures(profile_filter=7)` and
+`qsa_attention_module_fixtures(profile_filter=9)` from
+`runpy.run_path("tools/mtplx-kernel-fixtures.py")` in separate processes with
+`MLX_ENABLE_TF32=1` and `0`. Replace only those profile/TF32 rows in the fixture.

docs/issue107-vision-qsa/baseline-capture.patch

@@ -0,0 +1,102 @@
+--- a/src/engine/metal/qwen_mtplx/owner.rs
++++ b/src/engine/metal/qwen_mtplx/owner.rs
+@@ -1068,3 +1068,7 @@
+ #[cfg(test)]
+ #[path = "issue106_product.rs"]
+ mod issue106_product;
++
++#[cfg(test)]
++#[path = "issue107_product.rs"]
++mod issue107_product;
+--- /dev/null
++++ b/src/engine/metal/qwen_mtplx/issue107_product.rs
+@@ -0,0 +1,89 @@
++//! Long multi-image request and cache compatibility through the product owner.
++use super::*;
++use crate::settings::{
++    GenerationPreferences, ReasoningMode, RuntimePreferences, SpeculativePreferences,
++    effective_settings,
++};
++use base64::Engine as _;
++
++#[test]
++#[ignore = "installed model plus vision tower; serial external memory supervisor required"]
++fn issue107_vision_product() {
++    let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
++    let image = std::fs::read(std::env::var_os("DS4_QWEN_VISION_IMAGE").unwrap()).unwrap();
++    let directory = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE107_DIR").unwrap());
++    std::fs::create_dir_all(&directory).unwrap();
++    let settings = effective_settings(
++        crate::model::ModelChoice::Qwen38FlashNext,
++        &GenerationPreferences {
++            context_tokens: 8192,
++            max_generated_tokens: 1,
++            system_prompt: String::new(),
++            temperature: Some(0.),
++            top_p: Some(0.95),
++            min_p: Some(0.),
++            seed: Some(1),
++            reasoning_mode: ReasoningMode::Low,
++        },
++        &RuntimePreferences {
++            speculative: SpeculativePreferences {
++                glm_mtp: true,
++                ..Default::default()
++            },
++            ..Default::default()
++        },
++        root.parent().unwrap(),
++    )
++    .unwrap();
++    let marker = crate::engine::vision_data_marker(&format!(
++        "data:image/png;base64,{}",
++        base64::engine::general_purpose::STANDARD.encode(&image)
++    ));
++    let messages = [ChatTurn {
++        user: true,
++        tool: false,
++        system: false,
++        skip_previous_eos: false,
++        reasoning: None,
++        reasoning_complete: true,
++        content: format!("Describe these images{marker}{marker}"),
++    }];
++    let mut owner = QwenExecutor::open(&settings.engine).unwrap();
++    owner.policy.store_min_suffix = Some(0);
++    let metrics = Metrics::new(&directory.join("metrics"));
++    let (out, complete) = owner
++        .generate(
++            &messages,
++            &settings.turn,
++            &AtomicBool::new(false),
++            &metrics,
++            |p, _, _| {
++                eprintln!("#107 prefill {p}");
++            },
++            |_, _| {},
++        )
++        .unwrap();
++    assert!(complete);
++    assert!(out.prompt_tokens > 2048);
++    assert_eq!(out.cached_tokens, 0);
++    let saved = owner.live.current.as_ref().unwrap();
++    let logits = owner.execution.values_f32(&saved.logits).unwrap();
++    std::fs::write(
++        directory.join("long-final.f32"),
++        logits
++            .iter()
++            .flat_map(|v| v.to_le_bytes())
++            .collect::<Vec<_>>(),
++    )
++    .unwrap();
++    std::fs::write(directory.join("long-input.json"),serde_json::to_vec(&serde_json::json!({"ids":saved.token_ids,"images":owner.live.current_images,"prompt_tokens":out.prompt_tokens})).unwrap()).unwrap();
++    owner
++        .save_checkpoint(&directory.join("long.bin"), [107; 32], &mut |_| {})
++        .unwrap();
++    println!(
++        "{}",
++        serde_json::json!({"case":"vision_long","prompt_tokens":out.prompt_tokens,"cached_tokens":out.cached_tokens,
++        "tokens":owner.live.current.as_ref().unwrap().token_ids,"text":out.message.content,"reasoning":out.message.reasoning,
++        "logits_sha256":crate::engine::metal::qwen_mtplx_tests::hex_digest(&logits.iter().flat_map(|v|v.to_le_bytes()).collect::<Vec<_>>())})
++    );
++}

docs/issue107-vision-qsa/before.jsonl

@@ -0,0 +1 @@
+{"case":"vision_long","prompt_tokens":2095,"cached_tokens":0,"text":"","reasoning":"The","logits_sha256":"a79aa02a98542adeea636f22c85fc5f929308dc5dc75658d4072e6f275d22abe","bank_token_count":2096,"bank_token_ids_sha256":"a757dd27268f65ff6926bc0b2d6d81faec709b89512673068d3c536056c5105b","last_token":760}

docs/issue107-vision-qsa/checks.json

@@ -0,0 +1,132 @@
+{
+  "before": {
+    "cases": 1,
+    "test_result": [
+      "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 524 filtered out; finished in 11.84s"
+    ],
+    "elapsed_ms": 12922,
+    "peak_physical_bytes": 81646134816,
+    "supervisor_error": null,
+    "log_sha256": "15b89366a4a1488df33e2ada91cc848b9ac6240bf7f8a44f3afe4ff2c2f0c757"
+  },
+  "model-oracle": {
+    "cases": 7,
+    "test_result": [],
+    "elapsed_ms": 15413,
+    "peak_physical_bytes": 80281215320,
+    "supervisor_error": null,
+    "log_sha256": "efe9ea90eaf1d6a3d9e95669cd07f9318fdce6a3ae48d7b8dbecc545622511f2"
+  },
+  "indexer-oracle": {
+    "cases": 17,
+    "test_result": [],
+    "elapsed_ms": 625,
+    "peak_physical_bytes": 215155408,
+    "supervisor_error": null,
+    "log_sha256": "aac8e9211ffc59569716960a5493420ddf9f235e0d54e38ae02a7d27de9cee84"
+  },
+  "indexer-native-final": {
+    "cases": 15,
+    "test_result": [
+      "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 525 filtered out; finished in 0.28s"
+    ],
+    "elapsed_ms": 315,
+    "peak_physical_bytes": 181633888,
+    "supervisor_error": null,
+    "log_sha256": "1e634d173e8a5e4b6e50785ffcb31a70ac3438292293dddfa63316563d208b1e"
+  },
+  "small": {
+    "cases": 4,
+    "test_result": [
+      "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 525 filtered out; finished in 0.14s"
+    ],
+    "elapsed_ms": 210,
+    "peak_physical_bytes": 37094192,
+    "supervisor_error": null,
+    "log_sha256": "afb6f32dc619c6e3ec70ca877e08aeae1fece96c64dc12019f0704354c0bfde6"
+  },
+  "product": {
+    "cases": 16,
+    "test_result": [
+      "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 525 filtered out; finished in 165.40s"
+    ],
+    "elapsed_ms": 166771,
+    "peak_physical_bytes": 82617511280,
+    "supervisor_error": null,
+    "log_sha256": "844160c173fc4ba29b3475e871e940376b2ad553f0be75c17cf0fd049fe862de"
+  },
+  "text-indexer": {
+    "cases": 0,
+    "test_result": [
+      "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 525 filtered out; finished in 38.29s"
+    ],
+    "elapsed_ms": 38368,
+    "peak_physical_bytes": 1108821168,
+    "supervisor_error": null,
+    "log_sha256": "0494120021c6accce4e4976465b6a7c6c2ddbe94af829bd0ca71dd1ef17f2809"
+  },
+  "attention-native-1": {
+    "cases": 0,
+    "test_result": [
+      "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 525 filtered out; finished in 14.09s"
+    ],
+    "elapsed_ms": 14204,
+    "peak_physical_bytes": 1608435056,
+    "supervisor_error": null,
+    "log_sha256": "2ed6ca37f3700b0780cdf5c2262409ff16d2ed8c172ebd0206795c7a71fb4fea"
+  },
+  "attention-native-0": {
+    "cases": 0,
+    "test_result": [
+      "test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 525 filtered out; finished in 13.94s"
+    ],
+    "elapsed_ms": 14007,
+    "peak_physical_bytes": 1607861592,
+    "supervisor_error": null,
+    "log_sha256": "a32c72594ee7ce00b04553e65bca9de5b6cb0125b1d23b3abc574208fddb6e08"
+  },
+  "pre_fix_checkpoint": {
+    "base": "a8cad8810060d22ee7ccac82c210ef40d4e08785",
+    "bytes": 412628836,
+    "sha256": "7ddc663d4f337ff272000a7237dac05a224262bf25bc88257e784547b2056326",
+    "magic": "QWMTKV02",
+    "vision_policy": "absent (legacy 0)",
+    "images": 2,
+    "prompt_tokens": 2095
+  },
+  "model_logits": {
+    "before": "a79aa02a98542adeea636f22c85fc5f929308dc5dc75658d4072e6f275d22abe",
+    "native": "8b97664316bec6113c85b7dfca9b09967391f019ea1064b83d81a3d9d5191354",
+    "oracle": "8b97664316bec6113c85b7dfca9b09967391f019ea1064b83d81a3d9d5191354"
+  },
+  "input_image_sha256": "b7568d90f4df6180d9af14a824dd553cd995457561d29167354c1ba66b728347",
+  "commit_gates": {
+    "fmt": {
+      "status": "passed",
+      "log_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    },
+    "clippy": {
+      "status": "passed",
+      "log_sha256": "c437a2464fc3d0eb441259bf1126ad32e23230ebfde7e51a6c3fd06259d1bfc9"
+    },
+    "bundle": {
+      "status": "passed",
+      "log_sha256": "0b9900a8392b683ce2804a420fbcbd9cac895658bbad67859d56a9b046110159"
+    },
+    "tests-final": {
+      "status": "passed",
+      "log_sha256": "649b53e5b60e39c4a22a8f76246128b2024f76e6d9c9a9ec65b93e1b638cf261",
+      "results": [
+        "test result: ok. 308 passed; 0 failed; 218 ignored; 0 measured; 0 filtered out; finished in 5.52s",
+        "test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s",
+        "test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.29s"
+      ]
+    }
+  },
+  "unrelated_initial_test_failure": {
+    "test": "dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree",
+    "cause": "Temporary commit short hash 6496041 was parsed as an integer in unquoted YAML; test uses git2::Signature::now, so hash changes with time.",
+    "resolution": "Focused retry passed, followed by full suite passing. No unrelated source change."
+  },
+  "review": "All issue 107 acceptance, oracle route/numerics, Qwen cache policy, and family boundaries reviewed; no remaining gap."
+}

docs/issue107-vision-qsa/indexer-native.jsonl

@@ -0,0 +1,15 @@
+{"case":"indexer","dtype":"f32","mode":"table","part":"pooled","exact_oracle_match":true}
+{"case":"indexer","dtype":"f32","mode":"equal","part":"query","exact_oracle_match":true}
+{"case":"indexer","dtype":"f32","mode":"equal","part":"pooled","exact_oracle_match":true}
+{"case":"indexer","dtype":"f32","mode":"delta","part":"query","exact_oracle_match":true}
+{"case":"indexer","dtype":"f32","mode":"delta","part":"pooled","exact_oracle_match":true}
+{"case":"indexer","dtype":"f32","mode":"empty","part":"query","exact_oracle_match":true}
+{"case":"indexer","dtype":"f32","mode":"empty","part":"pooled","exact_oracle_match":true}
+{"case":"indexer","dtype":"bf16","mode":"table","part":"query","exact_oracle_match":true}
+{"case":"indexer","dtype":"bf16","mode":"table","part":"pooled","exact_oracle_match":true}
+{"case":"indexer","dtype":"bf16","mode":"equal","part":"query","exact_oracle_match":true}
+{"case":"indexer","dtype":"bf16","mode":"equal","part":"pooled","exact_oracle_match":true}
+{"case":"indexer","dtype":"bf16","mode":"delta","part":"query","exact_oracle_match":true}
+{"case":"indexer","dtype":"bf16","mode":"delta","part":"pooled","exact_oracle_match":true}
+{"case":"indexer","dtype":"bf16","mode":"empty","part":"query","exact_oracle_match":true}
+{"case":"indexer","dtype":"bf16","mode":"empty","part":"pooled","exact_oracle_match":true}

docs/issue107-vision-qsa/indexer-oracle.jsonl

@@ -0,0 +1,17 @@
+{"case":"indexer","dtype":"f32","mode":"table","part":"query","shape":[1,16,4,128],"sha256":"00cf519652dde6ac163822403e7bd26c6f131fde0673633d71f9cd6523ebb730"}
+{"case":"indexer","dtype":"f32","mode":"table","part":"pooled","shape":[1,518,128],"sha256":"d4c1f51750ebba1e83c2b5bed38a1962682630f39c0f3beae8455144444d8114"}
+{"case":"indexer","dtype":"f32","mode":"equal","part":"query","shape":[1,16,4,128],"sha256":"6ed0da7d959383ae23041ff321283fd7fa99d4e4539c8a4bd0f22bafd8dd457a"}
+{"case":"indexer","dtype":"f32","mode":"equal","part":"pooled","shape":[1,518,128],"sha256":"a6117d1182ed7abf52af125c9cfccba6bd1e416f3aa54d47b58d261c1b0f498d"}
+{"case":"indexer","dtype":"f32","mode":"delta","part":"query","shape":[1,16,4,128],"sha256":"b7efbed01f410398d6bd2393f45c52071c1f1da109b244f5e9b851814ec7cebc"}
+{"case":"indexer","dtype":"f32","mode":"delta","part":"pooled","shape":[1,518,128],"sha256":"f6533b66111c3230d4a6d1c0d32fbaccb97e9a25a42b5b0498671d83ffaa9bcf"}
+{"case":"indexer","dtype":"f32","mode":"empty","part":"query","shape":[1,16,4,128],"sha256":"b7efbed01f410398d6bd2393f45c52071c1f1da109b244f5e9b851814ec7cebc"}
+{"case":"indexer","dtype":"f32","mode":"empty","part":"pooled","shape":[1,518,128],"sha256":"f6533b66111c3230d4a6d1c0d32fbaccb97e9a25a42b5b0498671d83ffaa9bcf"}
+{"case":"indexer","dtype":"bf16","mode":"table","part":"query","shape":[1,16,4,128],"sha256":"0e32ce79b902a3c1974a682486432f7e719dfe6c560490279948c629d7680a20"}
+{"case":"indexer","dtype":"bf16","mode":"table","part":"pooled","shape":[1,518,128],"sha256":"a71f98ac2d9db6ad26a83daf9d8736c6a3a0ba0e4e03517a464718b4f72c1cad"}
+{"case":"indexer","dtype":"bf16","mode":"equal","part":"query","shape":[1,16,4,128],"sha256":"50533b71c2e539a8f56b8fa5d6a4b4979dafe21b67e8975f8461792dab1f0b67"}
+{"case":"indexer","dtype":"bf16","mode":"equal","part":"pooled","shape":[1,518,128],"sha256":"c57f4d215c9d4336c4250f1f5b494cefe156e43f88e712fcec53e789209b242b"}
+{"case":"indexer","dtype":"bf16","mode":"delta","part":"query","shape":[1,16,4,128],"sha256":"e95114b8b5cf606c714c06c3d23bbadc1126c502527cbf5b84fc5e7d773e218f"}
+{"case":"indexer","dtype":"bf16","mode":"delta","part":"pooled","shape":[1,518,128],"sha256":"b04fe5e8940fcd7cf15760692caa1951bcf7c338ba30d20fe84f31c1b6cca3cb"}
+{"case":"indexer","dtype":"bf16","mode":"empty","part":"query","shape":[1,16,4,128],"sha256":"e95114b8b5cf606c714c06c3d23bbadc1126c502527cbf5b84fc5e7d773e218f"}
+{"case":"indexer","dtype":"bf16","mode":"empty","part":"pooled","shape":[1,518,128],"sha256":"b04fe5e8940fcd7cf15760692caa1951bcf7c338ba30d20fe84f31c1b6cca3cb"}
+{"case":"independent_numpy","passed":true}

docs/issue107-vision-qsa/model-oracle.jsonl

@@ -0,0 +1,7 @@
+{"case":"environment","overrides":{"MTPLX_SKIP_VERIFY_SNAPSHOT":"0","MTPLX_FAMILY_CAPTURE_COMMIT":"1","MTPLX_FUSED_HC_V3":"1","MTPLX_FUSED_GDN_INPROJ":"1","MTPLX_FUSED_GATE_UP":"1","MTPLX_FUSED_GDN_CONVNORM":"1","MTPLX_FUSED_GDN_STEP":"1","MTPLX_FUSED_CONVNORM_VERIFY":"1","MTPLX_QSA_GATHER":"1","MTPLX_COMPILED_GDN":"1","MTPLX_QWEN4EXP_COMPILE":"1","MTPLX_QWEN4_BATCHED_TARGET_DISTRIBUTIONS":"1","MTPLX_QWEN4_FIXED_M4_VERIFY":"1","MTPLX_QWEN4_COMPILED_MTP_PREPARE":"1","MTPLX_QWEN4_RELAXED_DRAFT_TIES":"1","MTPLX_QWEN4_OPDIET":"1","MTPLX_QWEN4_PLE_PREFILL_LOOKAHEAD":"1","MTPLX_QWEN4_PLE_FIRST_GATHER_EARLY":"1","MTPLX_SESSION_BANK_SHED_BOUNDARIES":"1","MTPLX_SESSION_BANK_PROTECTED_TERMINAL":"1","MTPLX_NGRAM_PREWARM":"auto","MTPLX_QWEN4_VERIFY_GLUE":"1","MTPLX_QWEN4_VERIFY_GLUE_ITEMS":"qsa_rope,qsa_rope_idx","MTPLX_QSA_M4_FUSED_KV_GATHER":"1","MTPLX_QSA_GATHER_MAX_ROWS":"32","MTPLX_BATCH_TARGET_ARRAYS":"1","MTPLX_LAZY_TARGET_DISTRIBUTIONS":"0","MTPLX_COMPILED_VERIFY":"1","MTPLX_NAX_VERIFY":"0","MTPLX_QWEN4_PREFILL_WIDE_CHUNK":"4096","MTPLX_QSA_PREFILL_WIDE_MIN_CONTEXT":"16384","MTPLX_SUSTAINED_PREFILL":"1","MTPLX_PREFILL_CHUNK_SIZE":"2048"}}
+{"case":"prefill","phase":"started","tokens_done":0,"tokens_total":2095,"cached_tokens":0,"new_prefill_tokens":2095,"elapsed_s":0.0,"started_s":174050.1008485}
+{"case":"prefill","phase":"chunk","tokens_done":0,"tokens_total":2095,"cached_tokens":0,"new_prefill_tokens":2095,"elapsed_s":3.433300298638642e-05,"cache_hit":false,"reread":{"prompt_tokens":2095,"restore_point_tokens":0,"recompute_tokens":2095,"history_matched_tokens":0,"session_cached_tokens":0,"source":"none","cache_off":true,"image_changed_at":null,"image_limit_at":null}}
+{"case":"prefill","phase":"chunk","tokens_done":2048,"tokens_total":2095,"cached_tokens":0,"elapsed_s":4.264302124996902,"prefill_tok_s":480.266158440049,"cumulative_prefill_tok_s":480.266158440049,"prefill_wall_tok_s":480.266158440049,"live_prefill_tok_s":480.6052339792707,"chunk_size":2048,"chunk_elapsed_s":4.26129358401522,"chunk_prefill_tok_s":480.6052339792707}
+{"case":"prefill","phase":"chunk","tokens_done":2094,"tokens_total":2095,"cached_tokens":0,"elapsed_s":4.605880165996496,"prefill_tok_s":454.63623119403434,"cumulative_prefill_tok_s":454.63623119403434,"prefill_wall_tok_s":454.63623119403434,"live_prefill_tok_s":140.22877867288037,"chunk_size":46,"chunk_elapsed_s":0.32803537501604296,"chunk_prefill_tok_s":140.22877867288037}
+{"case":"prefill","phase":"completed","tokens_total":2095,"cached_tokens":0,"new_prefill_tokens":2095,"elapsed_s":4.6907559579994995,"prompt_eval_time_s":4.686342919012532,"prefill_tok_s":447.04368335927103,"prefill_compute_tok_s":447.04368335927103,"prefill_wall_tok_s":446.6231069700479,"cache_hit":false}
+{"case":"long_vision","prompt_tokens":2095,"text":"The","final_logits_sha256":"8b97664316bec6113c85b7dfca9b09967391f019ea1064b83d81a3d9d5191354","bank_token_count":1,"bank_token_ids_sha256":"a9cf0c5bfacb7118d245ee1a31149cbb264f7138ee585445dd76cc681d755554","last_token":760}

docs/issue107-vision-qsa/product.jsonl

@@ -0,0 +1,16 @@
+{"case":"vision_long","prompt_tokens":2095,"cached_tokens":0,"text":"","reasoning":"The","logits_sha256":"8b97664316bec6113c85b7dfca9b09967391f019ea1064b83d81a3d9d5191354","bank_token_count":2096,"bank_token_ids_sha256":"a757dd27268f65ff6926bc0b2d6d81faec709b89512673068d3c536056c5105b","last_token":760}
+{"case":"unchanged","cached_tokens":2095,"prompt_tokens":2095,"state_sha256":"958ff3ac9ad31c986085a45d31a7424475660263371b5a509b9d0a4ccaf44ab3"}
+{"case":"resident_followup","cached_tokens":2096,"prompt_tokens":2113,"state_sha256":"a5765c3da1ce1a75ce3b57e1b72a7a97b3f5874886c3eccd56c677127adebeef"}
+{"case":"reopened_followup","cached_tokens":2096,"prompt_tokens":2113,"state_sha256":"a5765c3da1ce1a75ce3b57e1b72a7a97b3f5874886c3eccd56c677127adebeef"}
+{"case":"text_before","cached_tokens":0,"prompt_tokens":2096,"state_sha256":"45a34b05a65378353e5c65df8f868dee7e75d38e381c897523058d37b64193d4"}
+{"case":"text_before_cold_control","cached_tokens":0,"prompt_tokens":2096,"state_sha256":"45a34b05a65378353e5c65df8f868dee7e75d38e381c897523058d37b64193d4"}
+{"case":"text_between","cached_tokens":0,"prompt_tokens":2098,"state_sha256":"69121fa827322e5308a065d61bbd9f53e6b85ced209214a0a7db4f711cc95e66"}
+{"case":"text_between_cold_control","cached_tokens":0,"prompt_tokens":2098,"state_sha256":"69121fa827322e5308a065d61bbd9f53e6b85ced209214a0a7db4f711cc95e66"}
+{"case":"text_after","cached_tokens":0,"prompt_tokens":2098,"state_sha256":"156daf37583f00996b960501612a77476e344eaead98726723958b79646ee67d"}
+{"case":"text_after_cold_control","cached_tokens":0,"prompt_tokens":2098,"state_sha256":"156daf37583f00996b960501612a77476e344eaead98726723958b79646ee67d"}
+{"case":"changed_pixels","cached_tokens":0,"prompt_tokens":2095,"state_sha256":"d1f3be2b52ca0314c9ec6c547babccb0847e29c733c9a31fdb4a55930f9af688"}
+{"case":"changed_pixels_cold_control","cached_tokens":0,"prompt_tokens":2095,"state_sha256":"d1f3be2b52ca0314c9ec6c547babccb0847e29c733c9a31fdb4a55930f9af688"}
+{"case":"cancel_resend","cached_tokens":0,"prompt_tokens":2095,"state_sha256":"958ff3ac9ad31c986085a45d31a7424475660263371b5a509b9d0a4ccaf44ab3"}
+{"case":"inside_image_guard","cached_tokens":0,"prompt_tokens":2095,"state_sha256":"958ff3ac9ad31c986085a45d31a7424475660263371b5a509b9d0a4ccaf44ab3"}
+{"case":"text_control","cached_tokens":0,"prompt_tokens":43,"state_sha256":"75bd53e84aa31a13851b02f13b025d5b7b76dddfec5236d2bd70294ab0548850"}
+{"case":"compatibility","old_disk_rejected":true,"old_live_rejected":true,"old_resident_rejected":true,"stale_save_rejected":true,"new_vision_and_text_reopen_exactly":true}

docs/issue107-vision-qsa/small-attention.jsonl

@@ -0,0 +1,4 @@
+{"case":"vision_positions","start":4,"rows":2,"table_end":4,"delta":-1,"different_rows":[],"actual_keys":[-0.3359375,0.02783203125,0.55078125,0.7890625,-0.94921875,1.25,1.3984375,1.5859375,0.3125,0.294921875,0.8203125,0.94140625,-1.203125,1.3203125,1.2109375,1.2578125],"expected_keys":[-0.3359375,0.02783203125,0.55078125,0.7890625,-0.94921875,1.25,1.3984375,1.5859375,0.3125,0.294921875,0.8203125,0.94140625,-1.203125,1.3203125,1.2109375,1.2578125]}
+{"case":"vision_positions","start":2,"rows":4,"table_end":4,"delta":-1,"different_rows":[0,1],"actual_keys":[-0.72265625,0.275390625,0.578125,0.79296875,0.69921875,1.21875,1.390625,1.5859375,-1.2265625,0.55078125,0.84375,0.94140625,0.216796875,1.234375,1.1953125,1.2578125,-0.9609375,0.515625,0.88671875,0.96484375,-0.890625,1.28125,1.140625,1.1640625,0.1962890625,0.4296875,0.90234375,0.9765625,-1.328125,1.3203125,1.125,1.125],"expected_keys":[-0.98046875,0.02783203125,0.5390625,0.7890625,-0.2314453125,1.25,1.40625,1.5859375,-1.2265625,0.423828125,0.8203125,0.94140625,0.216796875,1.28125,1.2109375,1.2578125,-0.9609375,0.515625,0.88671875,0.96484375,-0.890625,1.28125,1.140625,1.1640625,0.1962890625,0.4296875,0.90234375,0.9765625,-1.328125,1.3203125,1.125,1.125]}
+{"case":"vision_sparse_qsa","prefix_rows":12,"topk_blocks":1,"ratio":4,"position_delta":0,"text_output":[0.10986328125,-0.1923828125,0.08544921875,0.498046875,-0.0791015625,0.16796875,0.31640625,-0.03173828125],"vision_output":[0.10986328125,-0.1923828125,0.08544921875,0.498046875,-0.0791015625,0.16796875,0.31640625,-0.03173828125]}
+{"case":"active_attention_gate","gate":-6.84375,"actual":[0.0003185272216796875,-0.00074005126953125,0.00021266937255859375,0.00095367431640625,-0.00021266937255859375,0.0004253387451171875,0.000637054443359375,-0.00010633468627929688],"expected_verify":[0.0003204345703125,-0.0007476806640625,0.000213623046875,0.0009613037109375,-0.000213623046875,0.00042724609375,0.000640869140625,-0.0001068115234375]}