Publication status: analysis commit dcc4c516f0e030566c67472f8f1be0baa21ab629 is durable locally; its push to origin/main awaits explicit approval after automatic review refused the default-branch push. Until published, linked repository evidence files are available locally under /Users/gb/Projects/DS4Server/docs/issue103-analysis/ and the committed test/tool paths, not yet at the remote file URLs.
Review status (updated 2026-10-03)
Confirmed defect or numerical parity difference; implementation still awaits verification. This description incorporates the independent review and the user's explicit model-specific oracle clarification. It supersedes the original issue scope and conflicting conclusions in the initial evidence report. Original raw receipts/diagnostics remain unchanged; no production fix or new runtime/performance result is claimed. The report and AGENTS.md have not been rewritten in this issue-only update.
Model-specific parity boundary
The user explicitly clarified the oracle assignment after independent review:
Qwen 3.8 Flash Next: behavioral/numerical parity with the pinned MTPLX 2.12.2 revision 9882703f3105363ddc37eca9f97aa09a1d387112, including token processing, RNG/sampling, context accounting, inference and cache/session state.
DSv4 Flash and GLM 5.3 Flash: preserve DS4 behavior. A Qwen fix must not change their token IDs, sampler/RNG sequences, numerical inference, context accounting or cache behavior.
Keep changes in the Qwen family/phase/dispatch where applicable. Before changing any shared helper, trace every caller; preserve DS4-backed callers exactly and provide DS4 regression controls. A helper shared by Qwen and GLM must not silently impose MTPLX semantics on GLM. Do not globally change the DS4 sampler, shared RNG, Unicode tables, sigmoid/fast-math rules, or non-Qwen checkpoint formats for this ticket.
This is the user's model-specific clarification of the blanket DS4-oracle wording in AGENTS.md, not permission to weaken DS4 parity. Production remains native Rust/Metal with the existing Objective-C bridge; MTPLX Python is an isolated oracle only.
Broken behavior and evidence
Confirmed Qwen verify-phase numerical parity difference, not a claim that the mathematical sigmoid formula is wrong. qsa_attention::Attention.finish evaluates standalone Unary::Sigmoid on BF16 then multiplies. MLX0.32.2 lowers standalone precompiled sigmoid using precise exp (-fno-fast-math), whereas fused JIT sigmoid+multiply resolves exp to the fast variant. Latest MTPLX makes eager decode_verify use the same compiled expression as its fixed-M4 verifier; prefill/plain decode keep standalone behavior. DS4Server currently has the eager verifier, not the complete fixed-M4 owner, but Qwen still owes the reference verify-phase bits.
The exact BF16 reference is compiled output * sigmoid(gate.astype(f32).astype(bf16)): the value-preserving round trip forces the fused lowering; sigmoid is not computed in F32 then rounded to BF16. That latter operation would be a third, incorrect contract. For F32, upstream compiles output * sigmoid(gate) separately. FP16 keeps the reference's unchanged behavior.
Gate -6.84375: eager BF16 sigmoid=.00106048583984375; compiled/reference decode_verify=.001068115234375. Actual output element0=.0003185272216796875 versus expected verify=.0003204345703125; all 8 probe elements differ. The native diagnostic correctly uses Replay::trace_compiled with the same round-trip expression. Full-model sampled token divergence and independent PLE/hyper/shared-expert failures were not reproduced. This corrected description replaces the initial erroneous F32-sigmoid explanation.
Native implementation scope
Apply the exact fused verify-phase attention-gate contract only in Qwen decode_verify, using the existing compiled-replay expression or an equivalent numerically verified fused Metal sigmoid*mul lowering. If F32 gates are reachable, cover the separate _f32_attention_gate law. Preserve prefill/plain decode, dtype behavior and all unrelated sigmoid sites. Do not change global Unary::Sigmoid, compiler fast-math flags, DS4/GLM kernels or other Qwen GDN/PLE/hyper/shared-expert gates merely because they share a primitive. Additional siblings enter scope only with a demonstrated matching phase/site failure. Do not install a full fixed-M4 verifier for this fix. Scalar constant formatting remains the separately documented dormant candidate.
Acceptance
Retain active_attention_gate and bf16_gate as exact tests of the compiled BF16 round-trip expression, including -6.84375 and neighboring BF16 values. Where reachable, compare F32 with _f32_attention_gate; check FP16 follows its unchanged reference law. Prove prefill/plain decode unchanged and verify-phase output matches the pinned MTPLX attention_math.py/tests. Compare actual verify rows/logits and warm-next-turn state; use exactness where required rather than tolerances hiding lowering changes. Any additional gate site requires its own active-call/phase receipt. DSv4/GLM numerics and general sigmoid behavior remain unchanged. Token-level consequences of this single defect are unmeasured.
Severity
Medium: reproduced last-bit verify parity mismatch; no independent mathematical-correctness or token-impact claim.
Shared evidence and reproduction
Analysis parent: #103. Affected DS4Server revision: cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983. Compared MTPLX old e652d55e2652137a4abcf1312357abbf3eb9d692 (2.10.2) with refreshed 9882703f3105363ddc37eca9f97aa09a1d387112 (2.12.2), on MLX exactly 0.32.2. Same Bare-Speed artifact revision 74559cdf34fbfc0b593de72d17e93f37fd4f9ea7; unchanged hashes, compiler/OS/hardware and effective options in environment.json. Hardware: M5 Max128GiB/macOS26.6.2. Synthetic model-free cases use explicit shapes/dtypes in the diagnostic, no template/image/prompt or context setting; model case uses raw IDs, seed0 and restored identical prompt snapshots.
Durable report and instructions, native test, oracle probe. The test module is cfg(test), ignored by ordinary runs, and initially asserts/prints the affected behavior; passing this diagnostic means reproduction, not a fix. Convert only this issue's owned cases into correct-contract regressions during implementation. Shared vision test has separate case ownership listed in the report.
cargo test --all-features --bin ds4-server issue103_vision_position_receipts --no-run
cargo build --bin test-supervisor
exportDS4_QWEN38_ARTIFACTS='/path/to/unchanged/qwen3.8-flash-next'exportDS4_ISSUE103_ORACLE="$PWD/docs/issue103-analysis/latest-oracle.jsonl"# Resolve the emitted test executable; do not pass cargo to the watchdog.
target/debug/test-supervisor 409660120 --command /path/to/ds4_server-test-binary issue103_vision_position_receipts --ignored --nocapture --test-threads=1
Immutable upstream change and latest test. Exact BF16/F32/phase law: attention_math.py. Extra readbacks/synchronization and external memory samples make these correctness receipts unsuitable for speed claims.
Validation and review boundary
Implement production behavior in Rust with existing Metal/Objective-C bridge only; Python remains an isolated oracle.
Run cargo fmt --all -- --check, cargo clippy --all-targets --all-features -- -D warnings, make bundle, cargo test --all-features before every future commit, plus the owned native/oracle and product regressions above.
Implementation awaits the user's independent-agent review and subsequent instruction. Creating this issue or closing #103 does not authorize a fix. Review the corrected model-specific oracle, active-route applicability and state ownership first; preserve DS4-backed families.
No arbitrary dependencies on sibling issues: coordinate shared source changes where needed; each issue owns its concrete failure and acceptance.
Publication status: analysis commit `dcc4c516f0e030566c67472f8f1be0baa21ab629` is durable locally; its push to origin/main awaits explicit approval after automatic review refused the default-branch push. Until published, linked repository evidence files are available locally under `/Users/gb/Projects/DS4Server/docs/issue103-analysis/` and the committed test/tool paths, not yet at the remote file URLs.
## Review status (updated 2026-10-03)
Confirmed defect or numerical parity difference; implementation still awaits verification. This description incorporates the [independent review](https://git.rfc1437.de/hugo/DS4Server/issues/109#issuecomment-3072) and the user's explicit model-specific oracle clarification. It supersedes the original issue scope and conflicting conclusions in the initial evidence report. Original raw receipts/diagnostics remain unchanged; no production fix or new runtime/performance result is claimed. The report and AGENTS.md have not been rewritten in this issue-only update.
## Model-specific parity boundary
The user explicitly clarified the oracle assignment after independent review:
- Qwen 3.8 Flash Next: behavioral/numerical parity with the pinned MTPLX 2.12.2 revision `9882703f3105363ddc37eca9f97aa09a1d387112`, including token processing, RNG/sampling, context accounting, inference and cache/session state.
- DSv4 Flash and GLM 5.3 Flash: preserve DS4 behavior. A Qwen fix must not change their token IDs, sampler/RNG sequences, numerical inference, context accounting or cache behavior.
- Keep changes in the Qwen family/phase/dispatch where applicable. Before changing any shared helper, trace every caller; preserve DS4-backed callers exactly and provide DS4 regression controls. A helper shared by Qwen and GLM must not silently impose MTPLX semantics on GLM. Do not globally change the DS4 sampler, shared RNG, Unicode tables, sigmoid/fast-math rules, or non-Qwen checkpoint formats for this ticket.
This is the user's model-specific clarification of the blanket DS4-oracle wording in AGENTS.md, not permission to weaken DS4 parity. Production remains native Rust/Metal with the existing Objective-C bridge; MTPLX Python is an isolated oracle only.
## Broken behavior and evidence
Confirmed Qwen verify-phase numerical parity difference, not a claim that the mathematical sigmoid formula is wrong. `qsa_attention::Attention.finish` evaluates standalone Unary::Sigmoid on BF16 then multiplies. MLX0.32.2 lowers standalone precompiled sigmoid using precise exp (-fno-fast-math), whereas fused JIT sigmoid+multiply resolves exp to the fast variant. Latest MTPLX makes eager decode_verify use the same compiled expression as its fixed-M4 verifier; prefill/plain decode keep standalone behavior. DS4Server currently has the eager verifier, not the complete fixed-M4 owner, but Qwen still owes the reference verify-phase bits.
The exact BF16 reference is compiled `output * sigmoid(gate.astype(f32).astype(bf16))`: the value-preserving round trip forces the fused lowering; sigmoid is not computed in F32 then rounded to BF16. That latter operation would be a third, incorrect contract. For F32, upstream compiles `output * sigmoid(gate)` separately. FP16 keeps the reference's unchanged behavior.
Gate -6.84375: eager BF16 sigmoid=.00106048583984375; compiled/reference decode_verify=.001068115234375. Actual output element0=.0003185272216796875 versus expected verify=.0003204345703125; all 8 probe elements differ. The native diagnostic correctly uses Replay::trace_compiled with the same round-trip expression. Full-model sampled token divergence and independent PLE/hyper/shared-expert failures were not reproduced. This corrected description replaces the initial erroneous F32-sigmoid explanation.
## Native implementation scope
Apply the exact fused verify-phase attention-gate contract only in Qwen decode_verify, using the existing compiled-replay expression or an equivalent numerically verified fused Metal sigmoid*mul lowering. If F32 gates are reachable, cover the separate _f32_attention_gate law. Preserve prefill/plain decode, dtype behavior and all unrelated sigmoid sites. Do not change global Unary::Sigmoid, compiler fast-math flags, DS4/GLM kernels or other Qwen GDN/PLE/hyper/shared-expert gates merely because they share a primitive. Additional siblings enter scope only with a demonstrated matching phase/site failure. Do not install a full fixed-M4 verifier for this fix. Scalar constant formatting remains the separately documented dormant candidate.
## Acceptance
Retain active_attention_gate and bf16_gate as exact tests of the compiled BF16 round-trip expression, including -6.84375 and neighboring BF16 values. Where reachable, compare F32 with _f32_attention_gate; check FP16 follows its unchanged reference law. Prove prefill/plain decode unchanged and verify-phase output matches the pinned MTPLX attention_math.py/tests. Compare actual verify rows/logits and warm-next-turn state; use exactness where required rather than tolerances hiding lowering changes. Any additional gate site requires its own active-call/phase receipt. DSv4/GLM numerics and general sigmoid behavior remain unchanged. Token-level consequences of this single defect are unmeasured.
## Severity
Medium: reproduced last-bit verify parity mismatch; no independent mathematical-correctness or token-impact claim.
## Shared evidence and reproduction
Analysis parent: [#103](https://git.rfc1437.de/hugo/DS4Server/issues/103). Affected DS4Server revision: `cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983`. Compared MTPLX old `e652d55e2652137a4abcf1312357abbf3eb9d692` (2.10.2) with refreshed `9882703f3105363ddc37eca9f97aa09a1d387112` (2.12.2), on MLX exactly 0.32.2. Same Bare-Speed artifact revision `74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`; unchanged hashes, compiler/OS/hardware and effective options in [environment.json](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/docs/issue103-analysis/environment.json). Hardware: M5 Max128GiB/macOS26.6.2. Synthetic model-free cases use explicit shapes/dtypes in the diagnostic, no template/image/prompt or context setting; model case uses raw IDs, seed0 and restored identical prompt snapshots.
Durable [report and instructions](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/docs/issue103-analysis/README.md), [native test](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/src/engine/metal/qwen_mtplx/issue103_analysis.rs), [oracle probe](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/tools/issue103-reference.py). The test module is cfg(test), ignored by ordinary runs, and initially asserts/prints the affected behavior; passing this diagnostic means reproduction, not a fix. Convert only this issue's owned cases into correct-contract regressions during implementation. Shared vision test has separate case ownership listed in the report.
```sh
cargo test --all-features --bin ds4-server issue103_vision_position_receipts --no-run
cargo build --bin test-supervisor
export DS4_QWEN38_ARTIFACTS='/path/to/unchanged/qwen3.8-flash-next'
export DS4_ISSUE103_ORACLE="$PWD/docs/issue103-analysis/latest-oracle.jsonl"
# Resolve the emitted test executable; do not pass cargo to the watchdog.
target/debug/test-supervisor 4096 60 120 --command /path/to/ds4_server-test-binary issue103_vision_position_receipts --ignored --nocapture --test-threads=1
```
Immutable upstream [change](https://github.com/youssofal/mtplx/commit/a075e78b5a685d974d64984e4a50267b35f75054) and [latest test](https://github.com/youssofal/mtplx/blob/9882703f3105363ddc37eca9f97aa09a1d387112/tests/test_attention_gate_exactness.py). Exact BF16/F32/phase law: [attention_math.py](https://github.com/youssofal/mtplx/blob/9882703f3105363ddc37eca9f97aa09a1d387112/mtplx/attention_math.py). Extra readbacks/synchronization and external memory samples make these correctness receipts unsuitable for speed claims.
## Validation and review boundary
- Implement production behavior in Rust with existing Metal/Objective-C bridge only; Python remains an isolated oracle.
- Run `cargo fmt --all -- --check`, `cargo clippy --all-targets --all-features -- -D warnings`, `make bundle`, `cargo test --all-features` before every future commit, plus the owned native/oracle and product regressions above.
- Implementation awaits the user's independent-agent review and subsequent instruction. Creating this issue or closing #103 does not authorize a fix. Review the corrected model-specific oracle, active-route applicability and state ownership first; preserve DS4-backed families.
- No arbitrary dependencies on sibling issues: coordinate shared source changes where needed; each issue owns its concrete failure and acceptance.
hugo
added the bug label 2026-10-03 09:20:32 +00:00
Reviewed against DS4Server dcc4c51 and MTPLX 2.12.2 9882703f3105363ddc37eca9f97aa09a1d387112. Note: evidence commit dcc4c51 is still unpublished (main is ahead of origin/main by one commit), so the repository links in this issue currently resolve to nothing.
Defect claim: confirmed as a parity difference. The description of the upstream law is wrong and must be corrected before implementation.
The issue says latest MTPLX "uses the compiled float32 gate law and casts to BF16 before the multiply". mtplx/attention_math.py at 9882703 says the opposite. The difference comes from MLX 0.32.2 lowering sigmoid differently in a fused JIT kernel (unqualified metal::exp resolves to fast exp) versus the precompiled metallib (precise exp, -fno-fast-math). _bf16_attention_gate is @mx.compile of output * sigmoid(gate.astype(f32).astype(bf16)). The round trip only forces the fused lowering, and the comment warns: "Computing sigmoid in fp32 and then casting would change that contract." An implementer following the issue text (f32 sigmoid, then cast) would produce a third, wrong result.
The diagnostic in issue103_analysis.rs is correct: it uses Replay::trace_compiled of the same expression. The fix should reuse that existing compiled-replay path (or an equivalent fast-exp fused sigmoid*mul kernel) for the verify-phase gate only.
Context to add: upstream's motivation (a075e78) is parity between its own compiled fixed-M4 verifier and its eager verifier. DS4Server has only the eager verifier (report: fixed-M4 owner not installed). The change aligns our verify bits with the reference's verify lowering; it is not a correctness fix in the mathematical sense. Medium severity and the explicit DS4-contract review note are appropriate. Also apply the F32 variant (_f32_attention_gate) if F32 gates are reachable.
## Independent review (2026-10-03)
Reviewed against DS4Server `dcc4c51` and MTPLX 2.12.2 `9882703f3105363ddc37eca9f97aa09a1d387112`. Note: evidence commit `dcc4c51` is still unpublished (`main` is ahead of `origin/main` by one commit), so the repository links in this issue currently resolve to nothing.
**Defect claim: confirmed as a parity difference. The description of the upstream law is wrong and must be corrected before implementation.**
The issue says latest MTPLX "uses the compiled float32 gate law and casts to BF16 before the multiply". `mtplx/attention_math.py` at `9882703` says the opposite. The difference comes from MLX 0.32.2 lowering `sigmoid` differently in a fused JIT kernel (unqualified `metal::exp` resolves to fast exp) versus the precompiled metallib (precise exp, `-fno-fast-math`). `_bf16_attention_gate` is `@mx.compile` of `output * sigmoid(gate.astype(f32).astype(bf16))`. The round trip only forces the fused lowering, and the comment warns: "Computing sigmoid in fp32 and then casting would change that contract." An implementer following the issue text (f32 sigmoid, then cast) would produce a third, wrong result.
The diagnostic in `issue103_analysis.rs` is correct: it uses `Replay::trace_compiled` of the same expression. The fix should reuse that existing compiled-replay path (or an equivalent fast-exp fused sigmoid*mul kernel) for the verify-phase gate only.
Context to add: upstream's motivation (`a075e78`) is parity between its own compiled fixed-M4 verifier and its eager verifier. DS4Server has only the eager verifier (report: fixed-M4 owner not installed). The change aligns our verify bits with the reference's verify lowering; it is not a correctness fix in the mathematical sense. Medium severity and the explicit DS4-contract review note are appropriate. Also apply the F32 variant (`_f32_attention_gate`) if F32 gates are reachable.
Description corrected after independent review (2026-10-03)
Updated the issue body in response to your review. Changes:
Replaced the wrong F32-sigmoid/cast description with the exact BF16 round-trip compiled expression.
Explained fast-exp fused JIT versus precise-exp standalone lowering and eager/fixed-M4 parity motivation.
Restricted the fix to Qwen verify-phase attention, with F32 coverage only if reachable; prefill/decode/FP16 follow their reference contracts.
Explicitly prohibited global sigmoid/fast-math changes and speculative changes to other gate sites.
All eight issue descriptions now state the user's contract explicitly: Qwen 3.8 Flash Next follows pinned MTPLX; DSv4 Flash and GLM 5.3 Flash retain DS4 parity. Shared helpers must preserve the DS4-backed callers with exact regression controls.
For renewed verification: Compare the BF16/F32 expressions and phase dispatch with immutable attention_math.py, verify the native diagnostic expression, and ensure the proposed implementation cannot change global or DS4-family numerical semantics.
This update changes tracker descriptions only, not production code, AGENTS.md or the committed raw receipts. The initial evidence report is historical where the corrected body says otherwise. Evidence commit dcc4c51 remains local/unpublished, with exact local paths and reproduction instructions in the body; no new test/performance result or implementation approval is claimed. Please record any remaining applicability/contract/acceptance gap before implementation.
## Description corrected after independent review (2026-10-03)
Updated the issue body in response to [your review](https://git.rfc1437.de/hugo/DS4Server/issues/109#issuecomment-3072). Changes:
- Replaced the wrong F32-sigmoid/cast description with the exact BF16 round-trip compiled expression.
- Explained fast-exp fused JIT versus precise-exp standalone lowering and eager/fixed-M4 parity motivation.
- Restricted the fix to Qwen verify-phase attention, with F32 coverage only if reachable; prefill/decode/FP16 follow their reference contracts.
- Explicitly prohibited global sigmoid/fast-math changes and speculative changes to other gate sites.
All eight issue descriptions now state the user's contract explicitly: Qwen 3.8 Flash Next follows pinned MTPLX; DSv4 Flash and GLM 5.3 Flash retain DS4 parity. Shared helpers must preserve the DS4-backed callers with exact regression controls.
For renewed verification: Compare the BF16/F32 expressions and phase dispatch with immutable attention_math.py, verify the native diagnostic expression, and ensure the proposed implementation cannot change global or DS4-family numerical semantics.
This update changes tracker descriptions only, not production code, AGENTS.md or the committed raw receipts. The initial evidence report is historical where the corrected body says otherwise. Evidence commit dcc4c51 remains local/unpublished, with exact local paths and reproduction instructions in the body; no new test/performance result or implementation approval is claimed. Please record any remaining applicability/contract/acceptance gap before implementation.
Qwen MTP target verification, repair and final-primary commits now use the exact compiled BF16 cast-round-trip gate, the separate F32 law and unchanged FP16 behavior. Prefill/ordinary AR retain their original law; phase is included in compiled layer replay keys. Two actual eager HC-injection calls reproduced the same verify-phase failure, so that explicitly permitted sibling site is corrected with its own active-call evidence. Other sigmoid sites and DS4/GLM remain unchanged. Old MTP gate-policy checkpoints are rejected before restoration; unaffected old AR files remain compatible.
Verified all finite BF16 and FP16 inputs, 65,536 F32 samples, all three phases, nine pinned upstream tests, original owned diagnostics, whole-attention controls in both TF32 modes, HC controls, greedy/sampled final-primary commits at depths 1–3, old/new checkpoint behavior and exact resident-versus-disk warm state. Real cold/warm requests match all eight verify calls and final logits/hidden byte-for-byte, tokens and cache values.
Review boundary: the real-model gate isolation uses MTPLX's explicit unchunked-SDPA control. Default head-chunked SDPA is a separate reproduced missing route tracked in #113. Full-cache comparison also exposed one prefill signed-zero bit difference, present before verification, tracked in #114; raw/value hashes and the exact coordinate are retained. Neither is represented as default-route or full raw-cache parity here.
All mandatory gates passed: formatting, Clippy with warnings denied, signed release app bundle, and cargo test --all-features (308 application + 8 tool tests). Native Metal/oracle tests ran serially. Complete scope review, commands and receipts: docs/issue109-verify-gate. This is a Qwen oracle correction, not a new DS4Server feature.
Implemented and pushed as [57b8ac28a7ec](https://git.rfc1437.de/hugo/DS4Server/commit/57b8ac28a7ec20b4c9e42f292e28604f87c22530).
Qwen MTP target verification, repair and final-primary commits now use the exact compiled BF16 cast-round-trip gate, the separate F32 law and unchanged FP16 behavior. Prefill/ordinary AR retain their original law; phase is included in compiled layer replay keys. Two actual eager HC-injection calls reproduced the same verify-phase failure, so that explicitly permitted sibling site is corrected with its own active-call evidence. Other sigmoid sites and DS4/GLM remain unchanged. Old MTP gate-policy checkpoints are rejected before restoration; unaffected old AR files remain compatible.
Verified all finite BF16 and FP16 inputs, 65,536 F32 samples, all three phases, nine pinned upstream tests, original owned diagnostics, whole-attention controls in both TF32 modes, HC controls, greedy/sampled final-primary commits at depths 1–3, old/new checkpoint behavior and exact resident-versus-disk warm state. Real cold/warm requests match all eight verify calls and final logits/hidden byte-for-byte, tokens and cache values.
Review boundary: the real-model gate isolation uses MTPLX's explicit unchunked-SDPA control. Default head-chunked SDPA is a separate reproduced missing route tracked in #113. Full-cache comparison also exposed one prefill signed-zero bit difference, present before verification, tracked in #114; raw/value hashes and the exact coordinate are retained. Neither is represented as default-route or full raw-cache parity here.
All mandatory gates passed: formatting, Clippy with warnings denied, signed release app bundle, and cargo test --all-features (308 application + 8 tool tests). Native Metal/oracle tests ran serially. Complete scope review, commands and receipts: [docs/issue109-verify-gate](https://git.rfc1437.de/hugo/DS4Server/src/commit/57b8ac28a7ec20b4c9e42f292e28604f87c22530/docs/issue109-verify-gate). This is a Qwen oracle correction, not a new DS4Server feature.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Publication status: analysis commit
dcc4c516f0e030566c67472f8f1be0baa21ab629is durable locally; its push to origin/main awaits explicit approval after automatic review refused the default-branch push. Until published, linked repository evidence files are available locally under/Users/gb/Projects/DS4Server/docs/issue103-analysis/and the committed test/tool paths, not yet at the remote file URLs.Review status (updated 2026-10-03)
Confirmed defect or numerical parity difference; implementation still awaits verification. This description incorporates the independent review and the user's explicit model-specific oracle clarification. It supersedes the original issue scope and conflicting conclusions in the initial evidence report. Original raw receipts/diagnostics remain unchanged; no production fix or new runtime/performance result is claimed. The report and AGENTS.md have not been rewritten in this issue-only update.
Model-specific parity boundary
The user explicitly clarified the oracle assignment after independent review:
9882703f3105363ddc37eca9f97aa09a1d387112, including token processing, RNG/sampling, context accounting, inference and cache/session state.This is the user's model-specific clarification of the blanket DS4-oracle wording in AGENTS.md, not permission to weaken DS4 parity. Production remains native Rust/Metal with the existing Objective-C bridge; MTPLX Python is an isolated oracle only.
Broken behavior and evidence
Confirmed Qwen verify-phase numerical parity difference, not a claim that the mathematical sigmoid formula is wrong.
qsa_attention::Attention.finishevaluates standalone Unary::Sigmoid on BF16 then multiplies. MLX0.32.2 lowers standalone precompiled sigmoid using precise exp (-fno-fast-math), whereas fused JIT sigmoid+multiply resolves exp to the fast variant. Latest MTPLX makes eager decode_verify use the same compiled expression as its fixed-M4 verifier; prefill/plain decode keep standalone behavior. DS4Server currently has the eager verifier, not the complete fixed-M4 owner, but Qwen still owes the reference verify-phase bits.The exact BF16 reference is compiled
output * sigmoid(gate.astype(f32).astype(bf16)): the value-preserving round trip forces the fused lowering; sigmoid is not computed in F32 then rounded to BF16. That latter operation would be a third, incorrect contract. For F32, upstream compilesoutput * sigmoid(gate)separately. FP16 keeps the reference's unchanged behavior.Gate -6.84375: eager BF16 sigmoid=.00106048583984375; compiled/reference decode_verify=.001068115234375. Actual output element0=.0003185272216796875 versus expected verify=.0003204345703125; all 8 probe elements differ. The native diagnostic correctly uses Replay::trace_compiled with the same round-trip expression. Full-model sampled token divergence and independent PLE/hyper/shared-expert failures were not reproduced. This corrected description replaces the initial erroneous F32-sigmoid explanation.
Native implementation scope
Apply the exact fused verify-phase attention-gate contract only in Qwen decode_verify, using the existing compiled-replay expression or an equivalent numerically verified fused Metal sigmoid*mul lowering. If F32 gates are reachable, cover the separate _f32_attention_gate law. Preserve prefill/plain decode, dtype behavior and all unrelated sigmoid sites. Do not change global Unary::Sigmoid, compiler fast-math flags, DS4/GLM kernels or other Qwen GDN/PLE/hyper/shared-expert gates merely because they share a primitive. Additional siblings enter scope only with a demonstrated matching phase/site failure. Do not install a full fixed-M4 verifier for this fix. Scalar constant formatting remains the separately documented dormant candidate.
Acceptance
Retain active_attention_gate and bf16_gate as exact tests of the compiled BF16 round-trip expression, including -6.84375 and neighboring BF16 values. Where reachable, compare F32 with _f32_attention_gate; check FP16 follows its unchanged reference law. Prove prefill/plain decode unchanged and verify-phase output matches the pinned MTPLX attention_math.py/tests. Compare actual verify rows/logits and warm-next-turn state; use exactness where required rather than tolerances hiding lowering changes. Any additional gate site requires its own active-call/phase receipt. DSv4/GLM numerics and general sigmoid behavior remain unchanged. Token-level consequences of this single defect are unmeasured.
Severity
Medium: reproduced last-bit verify parity mismatch; no independent mathematical-correctness or token-impact claim.
Shared evidence and reproduction
Analysis parent: #103. Affected DS4Server revision:
cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983. Compared MTPLX olde652d55e2652137a4abcf1312357abbf3eb9d692(2.10.2) with refreshed9882703f3105363ddc37eca9f97aa09a1d387112(2.12.2), on MLX exactly 0.32.2. Same Bare-Speed artifact revision74559cdf34fbfc0b593de72d17e93f37fd4f9ea7; unchanged hashes, compiler/OS/hardware and effective options in environment.json. Hardware: M5 Max128GiB/macOS26.6.2. Synthetic model-free cases use explicit shapes/dtypes in the diagnostic, no template/image/prompt or context setting; model case uses raw IDs, seed0 and restored identical prompt snapshots.Durable report and instructions, native test, oracle probe. The test module is cfg(test), ignored by ordinary runs, and initially asserts/prints the affected behavior; passing this diagnostic means reproduction, not a fix. Convert only this issue's owned cases into correct-contract regressions during implementation. Shared vision test has separate case ownership listed in the report.
Immutable upstream change and latest test. Exact BF16/F32/phase law: attention_math.py. Extra readbacks/synchronization and external memory samples make these correctness receipts unsuitable for speed claims.
Validation and review boundary
cargo fmt --all -- --check,cargo clippy --all-targets --all-features -- -D warnings,make bundle,cargo test --all-featuresbefore every future commit, plus the owned native/oracle and product regressions above.Independent review (2026-10-03)
Reviewed against DS4Server
dcc4c51and MTPLX 2.12.29882703f3105363ddc37eca9f97aa09a1d387112. Note: evidence commitdcc4c51is still unpublished (mainis ahead oforigin/mainby one commit), so the repository links in this issue currently resolve to nothing.Defect claim: confirmed as a parity difference. The description of the upstream law is wrong and must be corrected before implementation.
The issue says latest MTPLX "uses the compiled float32 gate law and casts to BF16 before the multiply".
mtplx/attention_math.pyat9882703says the opposite. The difference comes from MLX 0.32.2 loweringsigmoiddifferently in a fused JIT kernel (unqualifiedmetal::expresolves to fast exp) versus the precompiled metallib (precise exp,-fno-fast-math)._bf16_attention_gateis@mx.compileofoutput * sigmoid(gate.astype(f32).astype(bf16)). The round trip only forces the fused lowering, and the comment warns: "Computing sigmoid in fp32 and then casting would change that contract." An implementer following the issue text (f32 sigmoid, then cast) would produce a third, wrong result.The diagnostic in
issue103_analysis.rsis correct: it usesReplay::trace_compiledof the same expression. The fix should reuse that existing compiled-replay path (or an equivalent fast-exp fused sigmoid*mul kernel) for the verify-phase gate only.Context to add: upstream's motivation (
a075e78) is parity between its own compiled fixed-M4 verifier and its eager verifier. DS4Server has only the eager verifier (report: fixed-M4 owner not installed). The change aligns our verify bits with the reference's verify lowering; it is not a correctness fix in the mathematical sense. Medium severity and the explicit DS4-contract review note are appropriate. Also apply the F32 variant (_f32_attention_gate) if F32 gates are reachable.Description corrected after independent review (2026-10-03)
Updated the issue body in response to your review. Changes:
All eight issue descriptions now state the user's contract explicitly: Qwen 3.8 Flash Next follows pinned MTPLX; DSv4 Flash and GLM 5.3 Flash retain DS4 parity. Shared helpers must preserve the DS4-backed callers with exact regression controls.
For renewed verification: Compare the BF16/F32 expressions and phase dispatch with immutable attention_math.py, verify the native diagnostic expression, and ensure the proposed implementation cannot change global or DS4-family numerical semantics.
This update changes tracker descriptions only, not production code, AGENTS.md or the committed raw receipts. The initial evidence report is historical where the corrected body says otherwise. Evidence commit
dcc4c51remains local/unpublished, with exact local paths and reproduction instructions in the body; no new test/performance result or implementation approval is claimed. Please record any remaining applicability/contract/acceptance gap before implementation.Implemented and pushed as 57b8ac28a7ec.
Qwen MTP target verification, repair and final-primary commits now use the exact compiled BF16 cast-round-trip gate, the separate F32 law and unchanged FP16 behavior. Prefill/ordinary AR retain their original law; phase is included in compiled layer replay keys. Two actual eager HC-injection calls reproduced the same verify-phase failure, so that explicitly permitted sibling site is corrected with its own active-call evidence. Other sigmoid sites and DS4/GLM remain unchanged. Old MTP gate-policy checkpoints are rejected before restoration; unaffected old AR files remain compatible.
Verified all finite BF16 and FP16 inputs, 65,536 F32 samples, all three phases, nine pinned upstream tests, original owned diagnostics, whole-attention controls in both TF32 modes, HC controls, greedy/sampled final-primary commits at depths 1–3, old/new checkpoint behavior and exact resident-versus-disk warm state. Real cold/warm requests match all eight verify calls and final logits/hidden byte-for-byte, tokens and cache values.
Review boundary: the real-model gate isolation uses MTPLX's explicit unchunked-SDPA control. Default head-chunked SDPA is a separate reproduced missing route tracked in #113. Full-cache comparison also exposed one prefill signed-zero bit difference, present before verification, tracked in #114; raw/value hashes and the exact coordinate are retained. Neither is represented as default-route or full raw-cache parity here.
All mandatory gates passed: formatting, Clippy with warnings denied, signed release app bundle, and cargo test --all-features (308 application + 8 tool tests). Native Metal/oracle tests ran serially. Complete scope review, commands and receipts: docs/issue109-verify-gate. This is a Qwen oracle correction, not a new DS4Server feature.