Publication status: analysis commit dcc4c516f0e030566c67472f8f1be0baa21ab629 is durable locally; its push to origin/main awaits explicit approval after automatic review refused the default-branch push. Until published, linked repository evidence files are available locally under /Users/gb/Projects/DS4Server/docs/issue103-analysis/ and the committed test/tool paths, not yet at the remote file URLs.
Review status (updated 2026-10-03)
Confirmed defect or numerical parity difference; implementation still awaits verification. This description incorporates the independent review and the user's explicit model-specific oracle clarification. It supersedes the original issue scope and conflicting conclusions in the initial evidence report. Original raw receipts/diagnostics remain unchanged; no production fix or new runtime/performance result is claimed. The report and AGENTS.md have not been rewritten in this issue-only update.
Model-specific parity boundary
The user explicitly clarified the oracle assignment after independent review:
Qwen 3.8 Flash Next: behavioral/numerical parity with the pinned MTPLX 2.12.2 revision 9882703f3105363ddc37eca9f97aa09a1d387112, including token processing, RNG/sampling, context accounting, inference and cache/session state.
DSv4 Flash and GLM 5.3 Flash: preserve DS4 behavior. A Qwen fix must not change their token IDs, sampler/RNG sequences, numerical inference, context accounting or cache behavior.
Keep changes in the Qwen family/phase/dispatch where applicable. Before changing any shared helper, trace every caller; preserve DS4-backed callers exactly and provide DS4 regression controls. A helper shared by Qwen and GLM must not silently impose MTPLX semantics on GLM. Do not globally change the DS4 sampler, shared RNG, Unicode tables, sigmoid/fast-math rules, or non-Qwen checkpoint formats for this ticket.
This is the user's model-specific clarification of the blanket DS4-oracle wording in AGENTS.md, not permission to weaken DS4 parity. Production remains native Rust/Metal with the existing Objective-C bridge; MTPLX Python is an isolated oracle only.
Broken behavior and evidence
Active qsa_attention::Attention.call executes indexer/cache work, then explicitly sets selected=None whenever vision.is_some(), forcing dense attention. qsa_indexer/qsa_prepare rotate indexer queries and pooled block-start keys with text positions rather than the image-aware context. Caller: multimodal request rendering → model vision context → Decoder/Attention.call → indexer/QSA → dense fallback. These differ from corrected Flash-Next semantics even when vision context has equal text positions.
An actual BF16 Attention module with ratio 4, topk_blocks=1, 12 cached rows and one decode row yields sparse text output [0.10986328125,-0.1923828125,0.08544921875,0.498046875,-0.0791015625,0.16796875,0.31640625,-0.03173828125]. Same input/cache with vision=(no table,delta=0) yields dense output [0.07568359375,-0.1455078125,0.05712890625,0.357421875,-0.034912109375,0.0927734375,0.25390625,-0.048828125]. Equal positions must preserve sparse selection/output. Latest official vision test confirms this contract. Tiny top-k is an isolated geometric reduction; shipped geometry is topk=512, ratio=4, attention 24 heads/dim256 and indexer 4 heads/dim128. No full long-image model logits or image-position indexer mismatch was separately measured.
Native implementation scope
Preserve sparse QSA under Qwen vision; thread image-aware M-RoPE through active indexer query and pooled block-start rotations as well as attention projections. Implement indexer positions per row from the start (table rule for covered rows, text+delta outside), matching upstream _vision_position_cos_sin. Audit actual prepare/select routes and preserve text/equal-axis behavior; do not disable sparse attention or change its budget/threshold as a workaround.
Version the Qwen vision cache policy/identity or checkpoint compatibility so pre-fix dense-vision snapshots cannot be accepted after changing attention semantics. Invalidate affected live/resident and persisted QWMTKV01 states before reuse; identical image digests/spans alone do not distinguish the old numerical policy. Keep invalidation and versioning Qwen-scoped; preserve DS4-family formats/behavior. #108 owns the separate attention table-straddling module defect; do not introduce that defect in the new indexer and do not claim #108 already has a reachable product trigger.
Acceptance
Own vision_sparse_qsa and require corrected selections/outputs with equal text positions. Add independent per-row M-RoPE query/block-start tests at real dim128/topk512 and long-image model logits against latest pinned oracle. Cover multiple images, text edits around spans, changed pixels with same pad IDs, unchanged content, cold/warm and cancel/resend; restores cannot land inside image spans. Capture a pre-fix dense-vision live/persisted snapshot and prove it is rejected after the policy change even for identical pixels/token IDs; a fresh corrected state must restore exactly. Verify unrelated text-only/DSv4/GLM state remains behaviorally correct. Real-geometry/full-image comparisons and stale-state invalidation remain required tests, not completed receipts.
Severity
High: wrong active vision attention output is reproduced. Full long-image quality/performance magnitude is unmeasured.
Shared evidence and reproduction
Analysis parent: #103. Affected DS4Server revision: cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983. Compared MTPLX old e652d55e2652137a4abcf1312357abbf3eb9d692 (2.10.2) with refreshed 9882703f3105363ddc37eca9f97aa09a1d387112 (2.12.2), on MLX exactly 0.32.2. Same Bare-Speed artifact revision 74559cdf34fbfc0b593de72d17e93f37fd4f9ea7; unchanged hashes, compiler/OS/hardware and effective options in environment.json. Hardware: M5 Max128GiB/macOS26.6.2. Synthetic model-free cases use explicit shapes/dtypes in the diagnostic, no template/image/prompt or context setting; model case uses raw IDs, seed0 and restored identical prompt snapshots.
Durable report and instructions, native test, oracle probe. The test module is cfg(test), ignored by ordinary runs, and initially asserts/prints the affected behavior; passing this diagnostic means reproduction, not a fix. Convert only this issue's owned cases into correct-contract regressions during implementation. Shared vision test has separate case ownership listed in the report.
cargo test --all-features --bin ds4-server issue103_vision_position_receipts --no-run
cargo build --bin test-supervisor
exportDS4_QWEN38_ARTIFACTS='/path/to/unchanged/qwen3.8-flash-next'exportDS4_ISSUE103_ORACLE="$PWD/docs/issue103-analysis/latest-oracle.jsonl"# Resolve the emitted test executable; do not pass cargo to the watchdog.
target/debug/test-supervisor 409660120 --command /path/to/ds4_server-test-binary issue103_vision_position_receipts --ignored --nocapture --test-threads=1
Immutable upstream change and latest test. The probe's independent expected values and controls are described above. Extra readbacks/synchronization and external memory samples make these correctness receipts unsuitable for speed claims.
Validation and review boundary
Implement production behavior in Rust with existing Metal/Objective-C bridge only; Python remains an isolated oracle.
Run cargo fmt --all -- --check, cargo clippy --all-targets --all-features -- -D warnings, make bundle, cargo test --all-features before every future commit, plus the owned native/oracle and product regressions above.
Implementation awaits the user's independent-agent review and subsequent instruction. Creating this issue or closing #103 does not authorize a fix. Review the corrected model-specific oracle, active-route applicability and state ownership first; preserve DS4-backed families.
No arbitrary dependencies on sibling issues: coordinate shared source changes where needed; each issue owns its concrete failure and acceptance.
Publication status: analysis commit `dcc4c516f0e030566c67472f8f1be0baa21ab629` is durable locally; its push to origin/main awaits explicit approval after automatic review refused the default-branch push. Until published, linked repository evidence files are available locally under `/Users/gb/Projects/DS4Server/docs/issue103-analysis/` and the committed test/tool paths, not yet at the remote file URLs.
## Review status (updated 2026-10-03)
Confirmed defect or numerical parity difference; implementation still awaits verification. This description incorporates the [independent review](https://git.rfc1437.de/hugo/DS4Server/issues/107#issuecomment-3069) and the user's explicit model-specific oracle clarification. It supersedes the original issue scope and conflicting conclusions in the initial evidence report. Original raw receipts/diagnostics remain unchanged; no production fix or new runtime/performance result is claimed. The report and AGENTS.md have not been rewritten in this issue-only update.
## Model-specific parity boundary
The user explicitly clarified the oracle assignment after independent review:
- Qwen 3.8 Flash Next: behavioral/numerical parity with the pinned MTPLX 2.12.2 revision `9882703f3105363ddc37eca9f97aa09a1d387112`, including token processing, RNG/sampling, context accounting, inference and cache/session state.
- DSv4 Flash and GLM 5.3 Flash: preserve DS4 behavior. A Qwen fix must not change their token IDs, sampler/RNG sequences, numerical inference, context accounting or cache behavior.
- Keep changes in the Qwen family/phase/dispatch where applicable. Before changing any shared helper, trace every caller; preserve DS4-backed callers exactly and provide DS4 regression controls. A helper shared by Qwen and GLM must not silently impose MTPLX semantics on GLM. Do not globally change the DS4 sampler, shared RNG, Unicode tables, sigmoid/fast-math rules, or non-Qwen checkpoint formats for this ticket.
This is the user's model-specific clarification of the blanket DS4-oracle wording in AGENTS.md, not permission to weaken DS4 parity. Production remains native Rust/Metal with the existing Objective-C bridge; MTPLX Python is an isolated oracle only.
## Broken behavior and evidence
Active `qsa_attention::Attention.call` executes indexer/cache work, then explicitly sets selected=None whenever vision.is_some(), forcing dense attention. `qsa_indexer`/qsa_prepare rotate indexer queries and pooled block-start keys with text positions rather than the image-aware context. Caller: multimodal request rendering → model vision context → Decoder/Attention.call → indexer/QSA → dense fallback. These differ from corrected Flash-Next semantics even when vision context has equal text positions.
An actual BF16 Attention module with ratio 4, topk_blocks=1, 12 cached rows and one decode row yields sparse text output [0.10986328125,-0.1923828125,0.08544921875,0.498046875,-0.0791015625,0.16796875,0.31640625,-0.03173828125]. Same input/cache with vision=(no table,delta=0) yields dense output [0.07568359375,-0.1455078125,0.05712890625,0.357421875,-0.034912109375,0.0927734375,0.25390625,-0.048828125]. Equal positions must preserve sparse selection/output. Latest official vision test confirms this contract. Tiny top-k is an isolated geometric reduction; shipped geometry is topk=512, ratio=4, attention 24 heads/dim256 and indexer 4 heads/dim128. No full long-image model logits or image-position indexer mismatch was separately measured.
## Native implementation scope
Preserve sparse QSA under Qwen vision; thread image-aware M-RoPE through active indexer query and pooled block-start rotations as well as attention projections. Implement indexer positions per row from the start (table rule for covered rows, text+delta outside), matching upstream _vision_position_cos_sin. Audit actual prepare/select routes and preserve text/equal-axis behavior; do not disable sparse attention or change its budget/threshold as a workaround.
Version the Qwen vision cache policy/identity or checkpoint compatibility so pre-fix dense-vision snapshots cannot be accepted after changing attention semantics. Invalidate affected live/resident and persisted QWMTKV01 states before reuse; identical image digests/spans alone do not distinguish the old numerical policy. Keep invalidation and versioning Qwen-scoped; preserve DS4-family formats/behavior. #108 owns the separate attention table-straddling module defect; do not introduce that defect in the new indexer and do not claim #108 already has a reachable product trigger.
## Acceptance
Own vision_sparse_qsa and require corrected selections/outputs with equal text positions. Add independent per-row M-RoPE query/block-start tests at real dim128/topk512 and long-image model logits against latest pinned oracle. Cover multiple images, text edits around spans, changed pixels with same pad IDs, unchanged content, cold/warm and cancel/resend; restores cannot land inside image spans. Capture a pre-fix dense-vision live/persisted snapshot and prove it is rejected after the policy change even for identical pixels/token IDs; a fresh corrected state must restore exactly. Verify unrelated text-only/DSv4/GLM state remains behaviorally correct. Real-geometry/full-image comparisons and stale-state invalidation remain required tests, not completed receipts.
## Severity
High: wrong active vision attention output is reproduced. Full long-image quality/performance magnitude is unmeasured.
## Shared evidence and reproduction
Analysis parent: [#103](https://git.rfc1437.de/hugo/DS4Server/issues/103). Affected DS4Server revision: `cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983`. Compared MTPLX old `e652d55e2652137a4abcf1312357abbf3eb9d692` (2.10.2) with refreshed `9882703f3105363ddc37eca9f97aa09a1d387112` (2.12.2), on MLX exactly 0.32.2. Same Bare-Speed artifact revision `74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`; unchanged hashes, compiler/OS/hardware and effective options in [environment.json](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/docs/issue103-analysis/environment.json). Hardware: M5 Max128GiB/macOS26.6.2. Synthetic model-free cases use explicit shapes/dtypes in the diagnostic, no template/image/prompt or context setting; model case uses raw IDs, seed0 and restored identical prompt snapshots.
Durable [report and instructions](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/docs/issue103-analysis/README.md), [native test](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/src/engine/metal/qwen_mtplx/issue103_analysis.rs), [oracle probe](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/tools/issue103-reference.py). The test module is cfg(test), ignored by ordinary runs, and initially asserts/prints the affected behavior; passing this diagnostic means reproduction, not a fix. Convert only this issue's owned cases into correct-contract regressions during implementation. Shared vision test has separate case ownership listed in the report.
```sh
cargo test --all-features --bin ds4-server issue103_vision_position_receipts --no-run
cargo build --bin test-supervisor
export DS4_QWEN38_ARTIFACTS='/path/to/unchanged/qwen3.8-flash-next'
export DS4_ISSUE103_ORACLE="$PWD/docs/issue103-analysis/latest-oracle.jsonl"
# Resolve the emitted test executable; do not pass cargo to the watchdog.
target/debug/test-supervisor 4096 60 120 --command /path/to/ds4_server-test-binary issue103_vision_position_receipts --ignored --nocapture --test-threads=1
```
Immutable upstream [change](https://github.com/youssofal/mtplx/commit/03a0a0c3727cb8d7d2957f97dedabc50ff65194f) and [latest test](https://github.com/youssofal/mtplx/blob/9882703f3105363ddc37eca9f97aa09a1d387112/tests/test_qwen4_vision_mrope.py). The probe's independent expected values and controls are described above. Extra readbacks/synchronization and external memory samples make these correctness receipts unsuitable for speed claims.
## Validation and review boundary
- Implement production behavior in Rust with existing Metal/Objective-C bridge only; Python remains an isolated oracle.
- Run `cargo fmt --all -- --check`, `cargo clippy --all-targets --all-features -- -D warnings`, `make bundle`, `cargo test --all-features` before every future commit, plus the owned native/oracle and product regressions above.
- Implementation awaits the user's independent-agent review and subsequent instruction. Creating this issue or closing #103 does not authorize a fix. Review the corrected model-specific oracle, active-route applicability and state ownership first; preserve DS4-backed families.
- No arbitrary dependencies on sibling issues: coordinate shared source changes where needed; each issue owns its concrete failure and acceptance.
hugo
added the bug label 2026-10-03 09:20:32 +00:00
Reviewed against DS4Server dcc4c51 and MTPLX 2.12.2 9882703f3105363ddc37eca9f97aa09a1d387112. Note: evidence commit dcc4c51 is still unpublished (main is ahead of origin/main by one commit), so the repository links in this issue currently resolve to nothing.
Defect claim: confirmed.qsa_attention::Attention.call sets selected = None whenever vision.is_some() (qsa_attention.rs:196). qsa_indexer::call takes no vision context and rotates with plain rope::cos_sin (qsa_indexer.rs:591/635). Upstream 03a0a0c states the corrected contract: keep sparse selection under multimodal input and rotate indexer queries and pooled block starts with M-RoPE.
Proposed fix: correct, one gap. Upstream 03a0a0c also "Version[s] the cache policy so older dense-vision state cannot be reused". Our image reuse identity (qwen_vision::same_prefix, request.rs:169) only compares image digests and spans. After the fix, live and persisted (QWMTKV01) snapshots computed with dense vision attention would still be restored as valid. Please add invalidation of pre-fix vision state (checkpoint version bump or an identity/policy version) to scope and acceptance.
Also note: upstream's indexer already chooses positions per row via _vision_position_cos_sin (mx.where). Implement the indexer threading per row from the start so it does not need the #108 fix separately.
## Independent review (2026-10-03)
Reviewed against DS4Server `dcc4c51` and MTPLX 2.12.2 `9882703f3105363ddc37eca9f97aa09a1d387112`. Note: evidence commit `dcc4c51` is still unpublished (`main` is ahead of `origin/main` by one commit), so the repository links in this issue currently resolve to nothing.
**Defect claim: confirmed.** `qsa_attention::Attention.call` sets `selected = None` whenever `vision.is_some()` (qsa_attention.rs:196). `qsa_indexer::call` takes no vision context and rotates with plain `rope::cos_sin` (qsa_indexer.rs:591/635). Upstream `03a0a0c` states the corrected contract: keep sparse selection under multimodal input and rotate indexer queries and pooled block starts with M-RoPE.
**Proposed fix: correct, one gap.** Upstream `03a0a0c` also "Version[s] the cache policy so older dense-vision state cannot be reused". Our image reuse identity (`qwen_vision::same_prefix`, request.rs:169) only compares image digests and spans. After the fix, live and persisted (`QWMTKV01`) snapshots computed with dense vision attention would still be restored as valid. Please add invalidation of pre-fix vision state (checkpoint version bump or an identity/policy version) to scope and acceptance.
Also note: upstream's indexer already chooses positions per row via `_vision_position_cos_sin` (mx.where). Implement the indexer threading per row from the start so it does not need the #108 fix separately.
Required row-wise indexer position handling from the start and kept #108 as a separate latent attention issue.
Added old-cache rejection and corrected new-cache restore acceptance, scoped to Qwen.
All eight issue descriptions now state the user's contract explicitly: Qwen 3.8 Flash Next follows pinned MTPLX; DSv4 Flash and GLM 5.3 Flash retain DS4 parity. Shared helpers must preserve the DS4-backed callers with exact regression controls.
For renewed verification: Check parity with 03a0a0c including its cache-policy versioning, per-row indexer positions, and rejection of old dense-vision snapshots with otherwise identical image identity.
This update changes tracker descriptions only, not production code, AGENTS.md or the committed raw receipts. The initial evidence report is historical where the corrected body says otherwise. Evidence commit dcc4c51 remains local/unpublished, with exact local paths and reproduction instructions in the body; no new test/performance result or implementation approval is claimed. Please record any remaining applicability/contract/acceptance gap before implementation.
## Description corrected after independent review (2026-10-03)
Updated the issue body in response to [your review](https://git.rfc1437.de/hugo/DS4Server/issues/107#issuecomment-3069). Changes:
- Added invalidation/versioning of pre-fix dense-vision live and persisted cache state.
- Explained why image digest/span equality alone cannot identify the attention-policy version.
- Required row-wise indexer position handling from the start and kept #108 as a separate latent attention issue.
- Added old-cache rejection and corrected new-cache restore acceptance, scoped to Qwen.
All eight issue descriptions now state the user's contract explicitly: Qwen 3.8 Flash Next follows pinned MTPLX; DSv4 Flash and GLM 5.3 Flash retain DS4 parity. Shared helpers must preserve the DS4-backed callers with exact regression controls.
For renewed verification: Check parity with 03a0a0c including its cache-policy versioning, per-row indexer positions, and rejection of old dense-vision snapshots with otherwise identical image identity.
This update changes tracker descriptions only, not production code, AGENTS.md or the committed raw receipts. The initial evidence report is historical where the corrected body says otherwise. Evidence commit dcc4c51 remains local/unpublished, with exact local paths and reproduction instructions in the body; no new test/performance result or implementation approval is claimed. Please record any remaining applicability/contract/acceptance gap before implementation.
Implemented and pushed in dccf81971f (Preserve image-aware sparse QSA for Qwen vision).
Qwen vision retains sparse selection and uses the pinned MTPLX image-aware, per-row query and pooled block-start positions, including the eager preparation route. Sparse budgets and thresholds are unchanged. Qwen image identities now carry a numerical policy: actual pre-fix live/resident and disk snapshots are rejected, stale owners cannot save, and restores cannot stop inside an image span. Text-only checkpoints remain compatible; DS4/GLM behavior and formats are unchanged. The separate attention table-straddling defect remains owned by #108.
Verification completed against MTPLX 9882703f3105363ddc37eca9f97aa09a1d387112 / MLX 0.32.2:
Real indexer geometry (4 heads, dim128, ratio4, top-k512, 518 blocks), F32/BF16, table/straddle/equal-axis/delta/empty cases: exact oracle matches, plus independent F32 math controls.
A real 2,095-token two-image model request now has byte-identical final logits to MTPLX (SHA256 8b97664316bec6113c85b7dfca9b09967391f019ea1064b83d81a3d9d5191354); the actual pre-fix logits differ.
Product regressions compare complete serialized state for unchanged images, disk reopen and follow-up, edits before/between/after images, changed pixels with identical pad IDs, cancellation/resend, unsafe image-prefix rejection, actual old snapshot invalidation, and unaffected text state.
All 56 whole-attention fixtures pass with lazy/prepared inputs across both TF32 modes; text-only indexer controls and the owned sparse-vision regression pass.
All commit gates pass: rustfmt, warning-free all-target/all-feature Clippy, signed bundle, and cargo test --all-features (308 application + 8 tool tests). An unrelated temporary-Git-hash YAML fixture flaked once; focused and full retries passed, with both outcomes documented.
Implementation review found the issue acceptance complete. Reproduction instructions, raw receipts, baseline capture, and check results: docs/issue107-vision-qsa.
Implemented and pushed in dccf81971fab7165927592366a5a65e76b6486c1 (Preserve image-aware sparse QSA for Qwen vision).
Qwen vision retains sparse selection and uses the pinned MTPLX image-aware, per-row query and pooled block-start positions, including the eager preparation route. Sparse budgets and thresholds are unchanged. Qwen image identities now carry a numerical policy: actual pre-fix live/resident and disk snapshots are rejected, stale owners cannot save, and restores cannot stop inside an image span. Text-only checkpoints remain compatible; DS4/GLM behavior and formats are unchanged. The separate attention table-straddling defect remains owned by #108.
Verification completed against MTPLX 9882703f3105363ddc37eca9f97aa09a1d387112 / MLX 0.32.2:
- Real indexer geometry (4 heads, dim128, ratio4, top-k512, 518 blocks), F32/BF16, table/straddle/equal-axis/delta/empty cases: exact oracle matches, plus independent F32 math controls.
- A real 2,095-token two-image model request now has byte-identical final logits to MTPLX (SHA256 8b97664316bec6113c85b7dfca9b09967391f019ea1064b83d81a3d9d5191354); the actual pre-fix logits differ.
- Product regressions compare complete serialized state for unchanged images, disk reopen and follow-up, edits before/between/after images, changed pixels with identical pad IDs, cancellation/resend, unsafe image-prefix rejection, actual old snapshot invalidation, and unaffected text state.
- All 56 whole-attention fixtures pass with lazy/prepared inputs across both TF32 modes; text-only indexer controls and the owned sparse-vision regression pass.
- All commit gates pass: rustfmt, warning-free all-target/all-feature Clippy, signed bundle, and cargo test --all-features (308 application + 8 tool tests). An unrelated temporary-Git-hash YAML fixture flaked once; focused and full retries passed, with both outcomes documented.
Implementation review found the issue acceptance complete. Reproduction instructions, raw receipts, baseline capture, and check results: [docs/issue107-vision-qsa](https://git.rfc1437.de/hugo/DS4Server/src/commit/dccf81971fab7165927592366a5a65e76b6486c1/docs/issue107-vision-qsa).
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Publication status: analysis commit
dcc4c516f0e030566c67472f8f1be0baa21ab629is durable locally; its push to origin/main awaits explicit approval after automatic review refused the default-branch push. Until published, linked repository evidence files are available locally under/Users/gb/Projects/DS4Server/docs/issue103-analysis/and the committed test/tool paths, not yet at the remote file URLs.Review status (updated 2026-10-03)
Confirmed defect or numerical parity difference; implementation still awaits verification. This description incorporates the independent review and the user's explicit model-specific oracle clarification. It supersedes the original issue scope and conflicting conclusions in the initial evidence report. Original raw receipts/diagnostics remain unchanged; no production fix or new runtime/performance result is claimed. The report and AGENTS.md have not been rewritten in this issue-only update.
Model-specific parity boundary
The user explicitly clarified the oracle assignment after independent review:
9882703f3105363ddc37eca9f97aa09a1d387112, including token processing, RNG/sampling, context accounting, inference and cache/session state.This is the user's model-specific clarification of the blanket DS4-oracle wording in AGENTS.md, not permission to weaken DS4 parity. Production remains native Rust/Metal with the existing Objective-C bridge; MTPLX Python is an isolated oracle only.
Broken behavior and evidence
Active
qsa_attention::Attention.callexecutes indexer/cache work, then explicitly sets selected=None whenever vision.is_some(), forcing dense attention.qsa_indexer/qsa_prepare rotate indexer queries and pooled block-start keys with text positions rather than the image-aware context. Caller: multimodal request rendering → model vision context → Decoder/Attention.call → indexer/QSA → dense fallback. These differ from corrected Flash-Next semantics even when vision context has equal text positions.An actual BF16 Attention module with ratio 4, topk_blocks=1, 12 cached rows and one decode row yields sparse text output [0.10986328125,-0.1923828125,0.08544921875,0.498046875,-0.0791015625,0.16796875,0.31640625,-0.03173828125]. Same input/cache with vision=(no table,delta=0) yields dense output [0.07568359375,-0.1455078125,0.05712890625,0.357421875,-0.034912109375,0.0927734375,0.25390625,-0.048828125]. Equal positions must preserve sparse selection/output. Latest official vision test confirms this contract. Tiny top-k is an isolated geometric reduction; shipped geometry is topk=512, ratio=4, attention 24 heads/dim256 and indexer 4 heads/dim128. No full long-image model logits or image-position indexer mismatch was separately measured.
Native implementation scope
Preserve sparse QSA under Qwen vision; thread image-aware M-RoPE through active indexer query and pooled block-start rotations as well as attention projections. Implement indexer positions per row from the start (table rule for covered rows, text+delta outside), matching upstream _vision_position_cos_sin. Audit actual prepare/select routes and preserve text/equal-axis behavior; do not disable sparse attention or change its budget/threshold as a workaround.
Version the Qwen vision cache policy/identity or checkpoint compatibility so pre-fix dense-vision snapshots cannot be accepted after changing attention semantics. Invalidate affected live/resident and persisted QWMTKV01 states before reuse; identical image digests/spans alone do not distinguish the old numerical policy. Keep invalidation and versioning Qwen-scoped; preserve DS4-family formats/behavior. #108 owns the separate attention table-straddling module defect; do not introduce that defect in the new indexer and do not claim #108 already has a reachable product trigger.
Acceptance
Own vision_sparse_qsa and require corrected selections/outputs with equal text positions. Add independent per-row M-RoPE query/block-start tests at real dim128/topk512 and long-image model logits against latest pinned oracle. Cover multiple images, text edits around spans, changed pixels with same pad IDs, unchanged content, cold/warm and cancel/resend; restores cannot land inside image spans. Capture a pre-fix dense-vision live/persisted snapshot and prove it is rejected after the policy change even for identical pixels/token IDs; a fresh corrected state must restore exactly. Verify unrelated text-only/DSv4/GLM state remains behaviorally correct. Real-geometry/full-image comparisons and stale-state invalidation remain required tests, not completed receipts.
Severity
High: wrong active vision attention output is reproduced. Full long-image quality/performance magnitude is unmeasured.
Shared evidence and reproduction
Analysis parent: #103. Affected DS4Server revision:
cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983. Compared MTPLX olde652d55e2652137a4abcf1312357abbf3eb9d692(2.10.2) with refreshed9882703f3105363ddc37eca9f97aa09a1d387112(2.12.2), on MLX exactly 0.32.2. Same Bare-Speed artifact revision74559cdf34fbfc0b593de72d17e93f37fd4f9ea7; unchanged hashes, compiler/OS/hardware and effective options in environment.json. Hardware: M5 Max128GiB/macOS26.6.2. Synthetic model-free cases use explicit shapes/dtypes in the diagnostic, no template/image/prompt or context setting; model case uses raw IDs, seed0 and restored identical prompt snapshots.Durable report and instructions, native test, oracle probe. The test module is cfg(test), ignored by ordinary runs, and initially asserts/prints the affected behavior; passing this diagnostic means reproduction, not a fix. Convert only this issue's owned cases into correct-contract regressions during implementation. Shared vision test has separate case ownership listed in the report.
Immutable upstream change and latest test. The probe's independent expected values and controls are described above. Extra readbacks/synchronization and external memory samples make these correctness receipts unsuitable for speed claims.
Validation and review boundary
cargo fmt --all -- --check,cargo clippy --all-targets --all-features -- -D warnings,make bundle,cargo test --all-featuresbefore every future commit, plus the owned native/oracle and product regressions above.Independent review (2026-10-03)
Reviewed against DS4Server
dcc4c51and MTPLX 2.12.29882703f3105363ddc37eca9f97aa09a1d387112. Note: evidence commitdcc4c51is still unpublished (mainis ahead oforigin/mainby one commit), so the repository links in this issue currently resolve to nothing.Defect claim: confirmed.
qsa_attention::Attention.callsetsselected = Nonewhenevervision.is_some()(qsa_attention.rs:196).qsa_indexer::calltakes no vision context and rotates with plainrope::cos_sin(qsa_indexer.rs:591/635). Upstream03a0a0cstates the corrected contract: keep sparse selection under multimodal input and rotate indexer queries and pooled block starts with M-RoPE.Proposed fix: correct, one gap. Upstream
03a0a0calso "Version[s] the cache policy so older dense-vision state cannot be reused". Our image reuse identity (qwen_vision::same_prefix, request.rs:169) only compares image digests and spans. After the fix, live and persisted (QWMTKV01) snapshots computed with dense vision attention would still be restored as valid. Please add invalidation of pre-fix vision state (checkpoint version bump or an identity/policy version) to scope and acceptance.Also note: upstream's indexer already chooses positions per row via
_vision_position_cos_sin(mx.where). Implement the indexer threading per row from the start so it does not need the #108 fix separately.Description corrected after independent review (2026-10-03)
Updated the issue body in response to your review. Changes:
All eight issue descriptions now state the user's contract explicitly: Qwen 3.8 Flash Next follows pinned MTPLX; DSv4 Flash and GLM 5.3 Flash retain DS4 parity. Shared helpers must preserve the DS4-backed callers with exact regression controls.
For renewed verification: Check parity with 03a0a0c including its cache-policy versioning, per-row indexer positions, and rejection of old dense-vision snapshots with otherwise identical image identity.
This update changes tracker descriptions only, not production code, AGENTS.md or the committed raw receipts. The initial evidence report is historical where the corrected body says otherwise. Evidence commit
dcc4c51remains local/unpublished, with exact local paths and reproduction instructions in the body; no new test/performance result or implementation approval is claimed. Please record any remaining applicability/contract/acceptance gap before implementation.Implemented and pushed in
dccf81971f(Preserve image-aware sparse QSA for Qwen vision).Qwen vision retains sparse selection and uses the pinned MTPLX image-aware, per-row query and pooled block-start positions, including the eager preparation route. Sparse budgets and thresholds are unchanged. Qwen image identities now carry a numerical policy: actual pre-fix live/resident and disk snapshots are rejected, stale owners cannot save, and restores cannot stop inside an image span. Text-only checkpoints remain compatible; DS4/GLM behavior and formats are unchanged. The separate attention table-straddling defect remains owned by #108.
Verification completed against MTPLX 9882703f3105363ddc37eca9f97aa09a1d387112 / MLX 0.32.2:
Implementation review found the issue acceptance complete. Reproduction instructions, raw receipts, baseline capture, and check results: docs/issue107-vision-qsa.