Files
DS4Server/docs/issue108-vision-boundary

Issue 108: row-wise attention positions

Qwen attention now implements pinned MTPLX's _vision_chunk_cos_sin route: covered rows use the original table slice and M-RoPE graph; uncovered rows use plain RoPE at index + delta; mixed chunks concatenate those two tables. The YaRN amplitude applies to both portions. With the shipped scaling of one, wholly inside/outside calls build the same numerical graphs as before. The text-only branch and indexer remain unchanged. Production changes are confined to two private Qwen modules, with no DS4/GLM callers or shared numerical changes.

This fixes a latent module mismatch. Review of vision::prepare, owner setup, model dispatch, suffix prefill and MTP history found no new product trigger: prepare builds a table over the complete rendered prompt; prefill stops within it, and decoding starts after it. The final post-image text positions are equal across axes at index + delta. A normal MTP window crossing that text boundary is therefore not a reproduction. No state rebase, prompt mutation or stale-table restore was added. No new DS4Server feature is involved.

Verification

tools/issue108-reference.py runs the pinned MTPLX revision 9882703f3105363ddc37eca9f97aa09a1d387112 in isolated MLX 0.32.2. It uses the oracle's actual frequency constructor and chunk helper, plus independent NumPy per-row checks. Its 56 cases cover small and shipped head geometry (dim256, 24 query heads, rotary64), F32/BF16, scaling 1/1.25, wholly inside/outside, unequal image-axis straddling, equal axes, absent/empty tables, and a full-prompt post-image text tail. Native cosine/sine and Q/K outputs match every oracle byte. The text-tail and equal-axis crossings also match plain shifted RoPE exactly.

The owned vision_positions case in issue103_vision_position_receipts now requires corrected cached keys. Actual Attention calls compare one mixed chunk with the same input split at the table end: outputs, valid K/V, offsets and the next call's output/state agree exactly. The unrelated sparse-QSA and attention Gate diagnostic cases retain their current separate ownership (#107/#109).

Existing whole-attention fixtures exercise the unchanged text/vision controls, cache hashes and execution routes in both TF32 modes, with lazy and prepared inputs. The mandatory Rust formatting, Clippy, bundle and complete ordinary test gates are recorded in checks.json; the adjacent JSONL receipts retain the owned test results. These are correctness checks, not performance claims.

Reproduction

local-eval-results/mtplx-reference-env-0.32.2/bin/python \
  tools/issue108-reference.py /path/to/pinned/mtplx /tmp/issue108-oracle
cargo test --all-features --bin ds4-server --no-run
cargo build --bin test-supervisor
export DS4_ISSUE108_ORACLE=/tmp/issue108-oracle
# Use the test executable printed by cargo, not cargo itself in the supervisor.
target/debug/test-supervisor 4096 60 120 --command TEST_EXECUTABLE \
  issue108_attention_chunk_positions --ignored --nocapture --test-threads=1
target/debug/test-supervisor 4096 60 120 --command TEST_EXECUTABLE \
  issue103_vision_position_receipts --ignored --nocapture --test-threads=1
# Run mtplx_qsa_attention_module_matches_reference the same way with
# MLX_ENABLE_TF32=1 and then MLX_ENABLE_TF32=0, serially.