58 lines
3.2 KiB
Markdown
58 lines
3.2 KiB
Markdown
# Issue 108: row-wise attention positions
|
|
|
|
Qwen attention now implements pinned MTPLX's `_vision_chunk_cos_sin` route:
|
|
covered rows use the original table slice and M-RoPE graph; uncovered rows use
|
|
plain RoPE at `index + delta`; mixed chunks concatenate those two tables. The
|
|
YaRN amplitude applies to both portions. With the shipped scaling of one, wholly
|
|
inside/outside calls build the same numerical graphs as before. The text-only
|
|
branch and indexer remain unchanged. Production changes are confined to two
|
|
private Qwen modules, with no DS4/GLM callers or shared numerical changes.
|
|
|
|
This fixes a **latent module mismatch**. Review of `vision::prepare`, owner setup,
|
|
model dispatch, suffix prefill and MTP history found no new product trigger:
|
|
`prepare` builds a table over the complete rendered prompt; prefill stops within
|
|
it, and decoding starts after it. The final post-image text positions are equal
|
|
across axes at `index + delta`. A normal MTP window crossing that text boundary
|
|
is therefore not a reproduction. No state rebase, prompt mutation or stale-table
|
|
restore was added. No new DS4Server feature is involved.
|
|
|
|
## Verification
|
|
|
|
`tools/issue108-reference.py` runs the pinned MTPLX revision
|
|
`9882703f3105363ddc37eca9f97aa09a1d387112` in isolated MLX 0.32.2. It uses the
|
|
oracle's actual frequency constructor and chunk helper, plus independent NumPy
|
|
per-row checks. Its 56 cases cover small and shipped head geometry (dim256,
|
|
24 query heads, rotary64), F32/BF16, scaling 1/1.25, wholly inside/outside,
|
|
unequal image-axis straddling, equal axes, absent/empty tables, and a full-prompt
|
|
post-image text tail. Native cosine/sine and Q/K outputs match every oracle byte.
|
|
The text-tail and equal-axis crossings also match plain shifted RoPE exactly.
|
|
|
|
The owned `vision_positions` case in `issue103_vision_position_receipts` now
|
|
requires corrected cached keys. Actual Attention calls compare one mixed chunk
|
|
with the same input split at the table end: outputs, valid K/V, offsets and the
|
|
next call's output/state agree exactly. The unrelated sparse-QSA and attention
|
|
Gate diagnostic cases retain their current separate ownership (#107/#109).
|
|
|
|
Existing whole-attention fixtures exercise the unchanged text/vision controls,
|
|
cache hashes and execution routes in both TF32 modes, with lazy and prepared
|
|
inputs. The mandatory Rust formatting, Clippy, bundle and complete ordinary test
|
|
gates are recorded in `checks.json`; the adjacent JSONL receipts retain the
|
|
owned test results. These are correctness checks, not performance claims.
|
|
|
|
## Reproduction
|
|
|
|
```sh
|
|
local-eval-results/mtplx-reference-env-0.32.2/bin/python \
|
|
tools/issue108-reference.py /path/to/pinned/mtplx /tmp/issue108-oracle
|
|
cargo test --all-features --bin ds4-server --no-run
|
|
cargo build --bin test-supervisor
|
|
export DS4_ISSUE108_ORACLE=/tmp/issue108-oracle
|
|
# Use the test executable printed by cargo, not cargo itself in the supervisor.
|
|
target/debug/test-supervisor 4096 60 120 --command TEST_EXECUTABLE \
|
|
issue108_attention_chunk_positions --ignored --nocapture --test-threads=1
|
|
target/debug/test-supervisor 4096 60 120 --command TEST_EXECUTABLE \
|
|
issue103_vision_position_receipts --ignored --nocapture --test-threads=1
|
|
# Run mtplx_qsa_attention_module_matches_reference the same way with
|
|
# MLX_ENABLE_TF32=1 and then MLX_ENABLE_TF32=0, serially.
|
|
```
|