Publication status: analysis commit dcc4c516f0e030566c67472f8f1be0baa21ab629 is durable locally; its push to origin/main awaits explicit approval after automatic review refused the default-branch push. Until published, linked repository evidence files are available locally under /Users/gb/Projects/DS4Server/docs/issue103-analysis/ and the committed test/tool paths, not yet at the remote file URLs.
Review status (updated 2026-10-03)
Confirmed defect or numerical parity difference; implementation still awaits verification. This description incorporates the independent review and the user's explicit model-specific oracle clarification. It supersedes the original issue scope and conflicting conclusions in the initial evidence report. Original raw receipts/diagnostics remain unchanged; no production fix or new runtime/performance result is claimed. The report and AGENTS.md have not been rewritten in this issue-only update.
Model-specific parity boundary
The user explicitly clarified the oracle assignment after independent review:
Qwen 3.8 Flash Next: behavioral/numerical parity with the pinned MTPLX 2.12.2 revision 9882703f3105363ddc37eca9f97aa09a1d387112, including token processing, RNG/sampling, context accounting, inference and cache/session state.
DSv4 Flash and GLM 5.3 Flash: preserve DS4 behavior. A Qwen fix must not change their token IDs, sampler/RNG sequences, numerical inference, context accounting or cache behavior.
Keep changes in the Qwen family/phase/dispatch where applicable. Before changing any shared helper, trace every caller; preserve DS4-backed callers exactly and provide DS4 regression controls. A helper shared by Qwen and GLM must not silently impose MTPLX semantics on GLM. Do not globally change the DS4 sampler, shared RNG, Unicode tables, sigmoid/fast-math rules, or non-Qwen checkpoint formats for this ticket.
This is the user's model-specific clarification of the blanket DS4-oracle wording in AGENTS.md, not permission to weaken DS4 parity. Production remains native Rust/Metal with the existing Objective-C bridge; MTPLX Python is an isolated oracle only.
Broken behavior and evidence
NaN, positive infinity and all-negative-infinity Qwen logits return apparently valid token IDs instead of explicit errors. sampling::sample_from_logits greedy argmax, host_sparse/SparseDistribution invalid-mass fallback, and the old lazy primitive lack the corrected validity contract. Caller: Qwen product generation → native AR or MTP target/draft sampling → response/session publication.
For 64-element rows ordinary sampling returns Ok(0) at temperatures 0/.6/1; the lazy primitive emits19/0/16 with key 7. Latest MTPLX ordinary sampling raises NonFiniteLogitsError; its off-decode-path lazy primitive returns checked sentinel-1. Legal [-inf,2,-inf,1,-inf,...] remains valid (token1 or3). The receipts reproduce the sampling boundary, not an HTTP/session fault-injection scenario.
MTPLX _non_finite_logits_failure deliberately clears every bank entry of the affected session: the poisoned state may already have been saved during prompt encoding. The earlier instruction to keep an allegedly independent prefill prefix was unsupported and is removed.
Native implementation scope
Validate greedy, ordinary and target/draft rows explicitly; NaN/+inf and all-masked rows fail, while negative-infinity masks with a finite winner remain legal. Reject before any fabricated ID is emitted/indexed/banked. Clear all reusable cached state of the affected Qwen session, including current/prefix snapshots, resident lease/bank entries and persisted checkpoint/selection for that session; prevent delayed persistence from republishing poisoned state. Do not retain the prefix that produced the invalid row and do not evict unrelated healthy sessions or globally change DS4-family failure handling. The server must stay usable. #104 removes the old lazy decode lane; validate any retained lazy primitive only at its real remaining callers, rather than reintroducing an obsolete decode path.
Acceptance
Keep all four synthetic rows for greedy/sampled and applicable target/draft/remaining lazy callers. Assert explicit error/no fabricated ID, legal masks, and checked sentinel if retained. Inject invalid rows through Qwen AR and MTP, streaming/non-streaming, and demonstrate removal of live/current/prefix/resident/persisted state of only that session, no stale checkpoint/bootstrap restore or delayed re-save, and a subsequent valid cold request. Cover prefix captured before failure and final-capture/persistence ownership. Unrelated sessions and DSv4/GLM error/state behavior remain unchanged. This end-to-end cleanup/retry test is required acceptance, not a completed analysis receipt.
Severity
High: non-finite output is silently accepted as generation. Spontaneous model-NaN frequency and complete session-cleanup failure were not measured.
Shared evidence and reproduction
Analysis parent: #103. Affected DS4Server revision: cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983. Compared MTPLX old e652d55e2652137a4abcf1312357abbf3eb9d692 (2.10.2) with refreshed 9882703f3105363ddc37eca9f97aa09a1d387112 (2.12.2), on MLX exactly 0.32.2. Same Bare-Speed artifact revision 74559cdf34fbfc0b593de72d17e93f37fd4f9ea7; unchanged hashes, compiler/OS/hardware and effective options in environment.json. Hardware: M5 Max128GiB/macOS26.6.2. Synthetic model-free cases use explicit shapes/dtypes in the diagnostic, no template/image/prompt or context setting; model case uses raw IDs, seed0 and restored identical prompt snapshots.
Durable report and instructions, native test, oracle probe. The test module is cfg(test), ignored by ordinary runs, and initially asserts/prints the affected behavior; passing this diagnostic means reproduction, not a fix. Convert only this issue's owned cases into correct-contract regressions during implementation. Shared vision test has separate case ownership listed in the report.
cargo test --all-features --bin ds4-server issue103_sampling_receipts --no-run
cargo build --bin test-supervisor
exportDS4_QWEN38_ARTIFACTS='/path/to/unchanged/qwen3.8-flash-next'exportDS4_ISSUE103_ORACLE="$PWD/docs/issue103-analysis/latest-oracle.jsonl"# Resolve the emitted test executable; do not pass cargo to the watchdog.
target/debug/test-supervisor 409660120 --command /path/to/ds4_server-test-binary issue103_sampling_receipts --ignored --nocapture --test-threads=1
Immutable upstream change and latest test. Cleanup reference: _non_finite_logits_failure. The primitive is off the latest AR decode path; #104 owns its retirement locally. Extra readbacks/synchronization and external memory samples make these correctness receipts unsuitable for speed claims.
Validation and review boundary
Implement production behavior in Rust with existing Metal/Objective-C bridge only; Python remains an isolated oracle.
Run cargo fmt --all -- --check, cargo clippy --all-targets --all-features -- -D warnings, make bundle, cargo test --all-features before every future commit, plus the owned native/oracle and product regressions above.
Implementation awaits the user's independent-agent review and subsequent instruction. Creating this issue or closing #103 does not authorize a fix. Review the corrected model-specific oracle, active-route applicability and state ownership first; preserve DS4-backed families.
No arbitrary dependencies on sibling issues: coordinate shared source changes where needed; each issue owns its concrete failure and acceptance.
Publication status: analysis commit `dcc4c516f0e030566c67472f8f1be0baa21ab629` is durable locally; its push to origin/main awaits explicit approval after automatic review refused the default-branch push. Until published, linked repository evidence files are available locally under `/Users/gb/Projects/DS4Server/docs/issue103-analysis/` and the committed test/tool paths, not yet at the remote file URLs.
## Review status (updated 2026-10-03)
Confirmed defect or numerical parity difference; implementation still awaits verification. This description incorporates the [independent review](https://git.rfc1437.de/hugo/DS4Server/issues/105#issuecomment-3066) and the user's explicit model-specific oracle clarification. It supersedes the original issue scope and conflicting conclusions in the initial evidence report. Original raw receipts/diagnostics remain unchanged; no production fix or new runtime/performance result is claimed. The report and AGENTS.md have not been rewritten in this issue-only update.
## Model-specific parity boundary
The user explicitly clarified the oracle assignment after independent review:
- Qwen 3.8 Flash Next: behavioral/numerical parity with the pinned MTPLX 2.12.2 revision `9882703f3105363ddc37eca9f97aa09a1d387112`, including token processing, RNG/sampling, context accounting, inference and cache/session state.
- DSv4 Flash and GLM 5.3 Flash: preserve DS4 behavior. A Qwen fix must not change their token IDs, sampler/RNG sequences, numerical inference, context accounting or cache behavior.
- Keep changes in the Qwen family/phase/dispatch where applicable. Before changing any shared helper, trace every caller; preserve DS4-backed callers exactly and provide DS4 regression controls. A helper shared by Qwen and GLM must not silently impose MTPLX semantics on GLM. Do not globally change the DS4 sampler, shared RNG, Unicode tables, sigmoid/fast-math rules, or non-Qwen checkpoint formats for this ticket.
This is the user's model-specific clarification of the blanket DS4-oracle wording in AGENTS.md, not permission to weaken DS4 parity. Production remains native Rust/Metal with the existing Objective-C bridge; MTPLX Python is an isolated oracle only.
## Broken behavior and evidence
NaN, positive infinity and all-negative-infinity Qwen logits return apparently valid token IDs instead of explicit errors. `sampling::sample_from_logits` greedy argmax, host_sparse/SparseDistribution invalid-mass fallback, and the old lazy primitive lack the corrected validity contract. Caller: Qwen product generation → native AR or MTP target/draft sampling → response/session publication.
For 64-element rows ordinary sampling returns `Ok(0)` at temperatures 0/.6/1; the lazy primitive emits19/0/16 with key 7. Latest MTPLX ordinary sampling raises NonFiniteLogitsError; its off-decode-path lazy primitive returns checked sentinel-1. Legal `[-inf,2,-inf,1,-inf,...]` remains valid (token1 or3). The receipts reproduce the sampling boundary, not an HTTP/session fault-injection scenario.
MTPLX `_non_finite_logits_failure` deliberately clears every bank entry of the affected session: the poisoned state may already have been saved during prompt encoding. The earlier instruction to keep an allegedly independent prefill prefix was unsupported and is removed.
## Native implementation scope
Validate greedy, ordinary and target/draft rows explicitly; NaN/+inf and all-masked rows fail, while negative-infinity masks with a finite winner remain legal. Reject before any fabricated ID is emitted/indexed/banked. Clear all reusable cached state of the affected Qwen session, including current/prefix snapshots, resident lease/bank entries and persisted checkpoint/selection for that session; prevent delayed persistence from republishing poisoned state. Do not retain the prefix that produced the invalid row and do not evict unrelated healthy sessions or globally change DS4-family failure handling. The server must stay usable. #104 removes the old lazy decode lane; validate any retained lazy primitive only at its real remaining callers, rather than reintroducing an obsolete decode path.
## Acceptance
Keep all four synthetic rows for greedy/sampled and applicable target/draft/remaining lazy callers. Assert explicit error/no fabricated ID, legal masks, and checked sentinel if retained. Inject invalid rows through Qwen AR and MTP, streaming/non-streaming, and demonstrate removal of live/current/prefix/resident/persisted state of only that session, no stale checkpoint/bootstrap restore or delayed re-save, and a subsequent valid cold request. Cover prefix captured before failure and final-capture/persistence ownership. Unrelated sessions and DSv4/GLM error/state behavior remain unchanged. This end-to-end cleanup/retry test is required acceptance, not a completed analysis receipt.
## Severity
High: non-finite output is silently accepted as generation. Spontaneous model-NaN frequency and complete session-cleanup failure were not measured.
## Shared evidence and reproduction
Analysis parent: [#103](https://git.rfc1437.de/hugo/DS4Server/issues/103). Affected DS4Server revision: `cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983`. Compared MTPLX old `e652d55e2652137a4abcf1312357abbf3eb9d692` (2.10.2) with refreshed `9882703f3105363ddc37eca9f97aa09a1d387112` (2.12.2), on MLX exactly 0.32.2. Same Bare-Speed artifact revision `74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`; unchanged hashes, compiler/OS/hardware and effective options in [environment.json](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/docs/issue103-analysis/environment.json). Hardware: M5 Max128GiB/macOS26.6.2. Synthetic model-free cases use explicit shapes/dtypes in the diagnostic, no template/image/prompt or context setting; model case uses raw IDs, seed0 and restored identical prompt snapshots.
Durable [report and instructions](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/docs/issue103-analysis/README.md), [native test](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/src/engine/metal/qwen_mtplx/issue103_analysis.rs), [oracle probe](https://git.rfc1437.de/hugo/DS4Server/src/commit/dcc4c516f0e030566c67472f8f1be0baa21ab629/tools/issue103-reference.py). The test module is cfg(test), ignored by ordinary runs, and initially asserts/prints the affected behavior; passing this diagnostic means reproduction, not a fix. Convert only this issue's owned cases into correct-contract regressions during implementation. Shared vision test has separate case ownership listed in the report.
```sh
cargo test --all-features --bin ds4-server issue103_sampling_receipts --no-run
cargo build --bin test-supervisor
export DS4_QWEN38_ARTIFACTS='/path/to/unchanged/qwen3.8-flash-next'
export DS4_ISSUE103_ORACLE="$PWD/docs/issue103-analysis/latest-oracle.jsonl"
# Resolve the emitted test executable; do not pass cargo to the watchdog.
target/debug/test-supervisor 4096 60 120 --command /path/to/ds4_server-test-binary issue103_sampling_receipts --ignored --nocapture --test-threads=1
```
Immutable upstream [change](https://github.com/youssofal/mtplx/commit/a2d0b4a493923f9c3b13178713e9f78413a1d2d7) and [latest test](https://github.com/youssofal/mtplx/blob/9882703f3105363ddc37eca9f97aa09a1d387112/tests/test_non_finite_logits_and_lazy_sampler_law.py). Cleanup reference: [_non_finite_logits_failure](https://github.com/youssofal/mtplx/blob/9882703f3105363ddc37eca9f97aa09a1d387112/mtplx/server/openai.py). The primitive is off the latest AR decode path; #104 owns its retirement locally. Extra readbacks/synchronization and external memory samples make these correctness receipts unsuitable for speed claims.
## Validation and review boundary
- Implement production behavior in Rust with existing Metal/Objective-C bridge only; Python remains an isolated oracle.
- Run `cargo fmt --all -- --check`, `cargo clippy --all-targets --all-features -- -D warnings`, `make bundle`, `cargo test --all-features` before every future commit, plus the owned native/oracle and product regressions above.
- Implementation awaits the user's independent-agent review and subsequent instruction. Creating this issue or closing #103 does not authorize a fix. Review the corrected model-specific oracle, active-route applicability and state ownership first; preserve DS4-backed families.
- No arbitrary dependencies on sibling issues: coordinate shared source changes where needed; each issue owns its concrete failure and acceptance.
hugo
added the bug label 2026-10-03 09:20:31 +00:00
Reviewed against DS4Server dcc4c51 and MTPLX 2.12.2 9882703f3105363ddc37eca9f97aa09a1d387112. Note: evidence commit dcc4c51 is still unpublished (main is ahead of origin/main by one commit), so the repository links in this issue currently resolve to nothing.
Defect claim: confirmed.host_sparse falls back to SparseDistribution([0],[1.0]) on invalid mass. The greedy branch of sample_from_logits runs argmax with no NaN/+inf check. lazy_sample has no validity sentinel. All of these match the upstream a2d0b4a description ("token 0 is !").
Proposed fix: one point contradicts upstream. The scope says to "discard any falsely complete reusable state while retaining an independently valid prefill prefix". Upstream _non_finite_logits_failure (mtplx/server/openai.py) deliberately clears all bank entries of the session: "The state that produced it may already sit in the session bank from the prompt encode ... a warm restore of a poisoned prefix would only reproduce the fault." The prefix that produced the non-finite row cannot be shown valid. Please change the scope to drop the session's cached state (live snapshot, prefix snapshot, persisted checkpoint for that session), then keep serving.
Everything else matches upstream: greedy check any(isnan) || isinf(max), -inf masks with a finite winner remain legal, the lazy -1 sentinel checked by the reader, and an explicit error. If #104 is fixed by removing the lazy lane from the decode path, the lazy part of this issue shrinks to the remaining callers (if any).
## Independent review (2026-10-03)
Reviewed against DS4Server `dcc4c51` and MTPLX 2.12.2 `9882703f3105363ddc37eca9f97aa09a1d387112`. Note: evidence commit `dcc4c51` is still unpublished (`main` is ahead of `origin/main` by one commit), so the repository links in this issue currently resolve to nothing.
**Defect claim: confirmed.** `host_sparse` falls back to `SparseDistribution([0],[1.0])` on invalid mass. The greedy branch of `sample_from_logits` runs `argmax` with no NaN/+inf check. `lazy_sample` has no validity sentinel. All of these match the upstream a2d0b4a description ("token 0 is `!`").
**Proposed fix: one point contradicts upstream.** The scope says to "discard any falsely complete reusable state while retaining an independently valid prefill prefix". Upstream `_non_finite_logits_failure` (mtplx/server/openai.py) deliberately clears **all** bank entries of the session: "The state that produced it may already sit in the session bank from the prompt encode ... a warm restore of a poisoned prefix would only reproduce the fault." The prefix that produced the non-finite row cannot be shown valid. Please change the scope to drop the session's cached state (live snapshot, prefix snapshot, persisted checkpoint for that session), then keep serving.
Everything else matches upstream: greedy check `any(isnan) || isinf(max)`, -inf masks with a finite winner remain legal, the lazy -1 sentinel checked by the reader, and an explicit error. If #104 is fixed by removing the lazy lane from the decode path, the lazy part of this issue shrinks to the remaining callers (if any).
Description corrected after independent review (2026-10-03)
Updated the issue body in response to your review. Changes:
Removed retention of an allegedly valid prefill prefix.
Required eviction of all reusable state belonging to the affected Qwen session, including disk/bootstrap selection and delayed persistence ownership.
Aligned lazy scope with #104: only actual remaining primitive callers after the decode lane is retired.
Added unaffected-session/DS4-family controls and retained the legal -inf mask distinction.
All eight issue descriptions now state the user's contract explicitly: Qwen 3.8 Flash Next follows pinned MTPLX; DSv4 Flash and GLM 5.3 Flash retain DS4 parity. Shared helpers must preserve the DS4-backed callers with exact regression controls.
For renewed verification: Compare cleanup with pinned MTPLX _non_finite_logits_failure and trace live, prefix, resident, persisted and pending-save ownership for one Qwen session. Confirm no global cache flush or DS4-family behavior change.
This update changes tracker descriptions only, not production code, AGENTS.md or the committed raw receipts. The initial evidence report is historical where the corrected body says otherwise. Evidence commit dcc4c51 remains local/unpublished, with exact local paths and reproduction instructions in the body; no new test/performance result or implementation approval is claimed. Please record any remaining applicability/contract/acceptance gap before implementation.
## Description corrected after independent review (2026-10-03)
Updated the issue body in response to [your review](https://git.rfc1437.de/hugo/DS4Server/issues/105#issuecomment-3066). Changes:
- Removed retention of an allegedly valid prefill prefix.
- Required eviction of all reusable state belonging to the affected Qwen session, including disk/bootstrap selection and delayed persistence ownership.
- Aligned lazy scope with #104: only actual remaining primitive callers after the decode lane is retired.
- Added unaffected-session/DS4-family controls and retained the legal -inf mask distinction.
All eight issue descriptions now state the user's contract explicitly: Qwen 3.8 Flash Next follows pinned MTPLX; DSv4 Flash and GLM 5.3 Flash retain DS4 parity. Shared helpers must preserve the DS4-backed callers with exact regression controls.
For renewed verification: Compare cleanup with pinned MTPLX _non_finite_logits_failure and trace live, prefix, resident, persisted and pending-save ownership for one Qwen session. Confirm no global cache flush or DS4-family behavior change.
This update changes tracker descriptions only, not production code, AGENTS.md or the committed raw receipts. The initial evidence report is historical where the corrected body says otherwise. Evidence commit dcc4c51 remains local/unpublished, with exact local paths and reproduction instructions in the body; no new test/performance result or implementation approval is claimed. Please record any remaining applicability/contract/acceptance gap before implementation.
Implemented in 5ecf8ab4f31aeaef7a06b4f03fbb8d658f28c486 and pushed to main.
Qwen now rejects NaN, +infinity and all-masked logits before consuming fabricated IDs. Greedy checks validity alongside argmax; sampled and sparse invalid-mass fallbacks raise explicit errors. Legal negative-infinity masks remain valid. Batched target checks operate per row, and the retained test-only lazy diagnostic returns -1 for invalid rows.
Numerical failure clears the affected live current/prefix, execution caches, resident aliases, checkpoint selection, persisted checkpoints/indexes/staging files, and bootstrap/conversation ancestry. A later save cannot republish state. Unrelated healthy sessions remain byte-identical, resident and restorable; DS4/GLM sampler/RNG and failure semantics are unchanged.
Oracle review: ordinary behavior matches MTPLX 9882703f3105363ddc37eca9f97aa09a1d387112 / MLX 0.32.2. Some specialized greedy MTP paths still fabricate token 0 upstream. The user explicitly approved stricter invalid-row rejection throughout Qwen MTP on 2026-10-04; this exception is documented rather than claimed as parity.
Validation: 1,080 fresh ordinary-sampler fixtures (three steps, exact tokens/probabilities/RNG/evaluation boundaries); 150 fallback, 653 batched, 100 mixed-distribution and 387 historical RNG/lazy cases; exact four-row oracle census/sentinel checks; mixed-validity batched target regression. Installed-model product tests passed 24 AR and 32 MTP cases through streaming/non-streaming HTTP adapters, including real prefill snapshots, target/draft/final-capture failures, bootstrap/disk/resident cleanup, refusal of later persistence, healthy-session retention and valid cold retries. Target tests exercise both context-copy and serial verification; draft tests force the actual draft head through the supported penalty policy.
All commit gates passed: cargo fmt, Clippy all targets/features with warnings denied, signed make bundle, cargo test --all-features (308 application tests passed, 214 ignored; 4+4 tool tests). Required ignored GPU/product tests above were run explicitly under the memory supervisor.
Implemented in `5ecf8ab4f31aeaef7a06b4f03fbb8d658f28c486` and pushed to main.
Qwen now rejects NaN, +infinity and all-masked logits before consuming fabricated IDs. Greedy checks validity alongside argmax; sampled and sparse invalid-mass fallbacks raise explicit errors. Legal negative-infinity masks remain valid. Batched target checks operate per row, and the retained test-only lazy diagnostic returns -1 for invalid rows.
Numerical failure clears the affected live current/prefix, execution caches, resident aliases, checkpoint selection, persisted checkpoints/indexes/staging files, and bootstrap/conversation ancestry. A later save cannot republish state. Unrelated healthy sessions remain byte-identical, resident and restorable; DS4/GLM sampler/RNG and failure semantics are unchanged.
Oracle review: ordinary behavior matches MTPLX 9882703f3105363ddc37eca9f97aa09a1d387112 / MLX 0.32.2. Some specialized greedy MTP paths still fabricate token 0 upstream. The user explicitly approved stricter invalid-row rejection throughout Qwen MTP on 2026-10-04; this exception is documented rather than claimed as parity.
Validation: 1,080 fresh ordinary-sampler fixtures (three steps, exact tokens/probabilities/RNG/evaluation boundaries); 150 fallback, 653 batched, 100 mixed-distribution and 387 historical RNG/lazy cases; exact four-row oracle census/sentinel checks; mixed-validity batched target regression. Installed-model product tests passed 24 AR and 32 MTP cases through streaming/non-streaming HTTP adapters, including real prefill snapshots, target/draft/final-capture failures, bootstrap/disk/resident cleanup, refusal of later persistence, healthy-session retention and valid cold retries. Target tests exercise both context-copy and serial verification; draft tests force the actual draft head through the supported penalty policy.
All commit gates passed: cargo fmt, Clippy all targets/features with warnings denied, signed make bundle, cargo test --all-features (308 application tests passed, 214 ignored; 4+4 tool tests). Required ignored GPU/product tests above were run explicitly under the memory supervisor.
[Implementation review and reproducible receipts](https://git.rfc1437.de/hugo/DS4Server/src/commit/5ecf8ab4f31aeaef7a06b4f03fbb8d658f28c486/docs/issue105-invalid-logits/README.md).
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Publication status: analysis commit
dcc4c516f0e030566c67472f8f1be0baa21ab629is durable locally; its push to origin/main awaits explicit approval after automatic review refused the default-branch push. Until published, linked repository evidence files are available locally under/Users/gb/Projects/DS4Server/docs/issue103-analysis/and the committed test/tool paths, not yet at the remote file URLs.Review status (updated 2026-10-03)
Confirmed defect or numerical parity difference; implementation still awaits verification. This description incorporates the independent review and the user's explicit model-specific oracle clarification. It supersedes the original issue scope and conflicting conclusions in the initial evidence report. Original raw receipts/diagnostics remain unchanged; no production fix or new runtime/performance result is claimed. The report and AGENTS.md have not been rewritten in this issue-only update.
Model-specific parity boundary
The user explicitly clarified the oracle assignment after independent review:
9882703f3105363ddc37eca9f97aa09a1d387112, including token processing, RNG/sampling, context accounting, inference and cache/session state.This is the user's model-specific clarification of the blanket DS4-oracle wording in AGENTS.md, not permission to weaken DS4 parity. Production remains native Rust/Metal with the existing Objective-C bridge; MTPLX Python is an isolated oracle only.
Broken behavior and evidence
NaN, positive infinity and all-negative-infinity Qwen logits return apparently valid token IDs instead of explicit errors.
sampling::sample_from_logitsgreedy argmax, host_sparse/SparseDistribution invalid-mass fallback, and the old lazy primitive lack the corrected validity contract. Caller: Qwen product generation → native AR or MTP target/draft sampling → response/session publication.For 64-element rows ordinary sampling returns
Ok(0)at temperatures 0/.6/1; the lazy primitive emits19/0/16 with key 7. Latest MTPLX ordinary sampling raises NonFiniteLogitsError; its off-decode-path lazy primitive returns checked sentinel-1. Legal[-inf,2,-inf,1,-inf,...]remains valid (token1 or3). The receipts reproduce the sampling boundary, not an HTTP/session fault-injection scenario.MTPLX
_non_finite_logits_failuredeliberately clears every bank entry of the affected session: the poisoned state may already have been saved during prompt encoding. The earlier instruction to keep an allegedly independent prefill prefix was unsupported and is removed.Native implementation scope
Validate greedy, ordinary and target/draft rows explicitly; NaN/+inf and all-masked rows fail, while negative-infinity masks with a finite winner remain legal. Reject before any fabricated ID is emitted/indexed/banked. Clear all reusable cached state of the affected Qwen session, including current/prefix snapshots, resident lease/bank entries and persisted checkpoint/selection for that session; prevent delayed persistence from republishing poisoned state. Do not retain the prefix that produced the invalid row and do not evict unrelated healthy sessions or globally change DS4-family failure handling. The server must stay usable. #104 removes the old lazy decode lane; validate any retained lazy primitive only at its real remaining callers, rather than reintroducing an obsolete decode path.
Acceptance
Keep all four synthetic rows for greedy/sampled and applicable target/draft/remaining lazy callers. Assert explicit error/no fabricated ID, legal masks, and checked sentinel if retained. Inject invalid rows through Qwen AR and MTP, streaming/non-streaming, and demonstrate removal of live/current/prefix/resident/persisted state of only that session, no stale checkpoint/bootstrap restore or delayed re-save, and a subsequent valid cold request. Cover prefix captured before failure and final-capture/persistence ownership. Unrelated sessions and DSv4/GLM error/state behavior remain unchanged. This end-to-end cleanup/retry test is required acceptance, not a completed analysis receipt.
Severity
High: non-finite output is silently accepted as generation. Spontaneous model-NaN frequency and complete session-cleanup failure were not measured.
Shared evidence and reproduction
Analysis parent: #103. Affected DS4Server revision:
cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983. Compared MTPLX olde652d55e2652137a4abcf1312357abbf3eb9d692(2.10.2) with refreshed9882703f3105363ddc37eca9f97aa09a1d387112(2.12.2), on MLX exactly 0.32.2. Same Bare-Speed artifact revision74559cdf34fbfc0b593de72d17e93f37fd4f9ea7; unchanged hashes, compiler/OS/hardware and effective options in environment.json. Hardware: M5 Max128GiB/macOS26.6.2. Synthetic model-free cases use explicit shapes/dtypes in the diagnostic, no template/image/prompt or context setting; model case uses raw IDs, seed0 and restored identical prompt snapshots.Durable report and instructions, native test, oracle probe. The test module is cfg(test), ignored by ordinary runs, and initially asserts/prints the affected behavior; passing this diagnostic means reproduction, not a fix. Convert only this issue's owned cases into correct-contract regressions during implementation. Shared vision test has separate case ownership listed in the report.
Immutable upstream change and latest test. Cleanup reference: _non_finite_logits_failure. The primitive is off the latest AR decode path; #104 owns its retirement locally. Extra readbacks/synchronization and external memory samples make these correctness receipts unsuitable for speed claims.
Validation and review boundary
cargo fmt --all -- --check,cargo clippy --all-targets --all-features -- -D warnings,make bundle,cargo test --all-featuresbefore every future commit, plus the owned native/oracle and product regressions above.Independent review (2026-10-03)
Reviewed against DS4Server
dcc4c51and MTPLX 2.12.29882703f3105363ddc37eca9f97aa09a1d387112. Note: evidence commitdcc4c51is still unpublished (mainis ahead oforigin/mainby one commit), so the repository links in this issue currently resolve to nothing.Defect claim: confirmed.
host_sparsefalls back toSparseDistribution([0],[1.0])on invalid mass. The greedy branch ofsample_from_logitsrunsargmaxwith no NaN/+inf check.lazy_samplehas no validity sentinel. All of these match the upstream a2d0b4a description ("token 0 is!").Proposed fix: one point contradicts upstream. The scope says to "discard any falsely complete reusable state while retaining an independently valid prefill prefix". Upstream
_non_finite_logits_failure(mtplx/server/openai.py) deliberately clears all bank entries of the session: "The state that produced it may already sit in the session bank from the prompt encode ... a warm restore of a poisoned prefix would only reproduce the fault." The prefix that produced the non-finite row cannot be shown valid. Please change the scope to drop the session's cached state (live snapshot, prefix snapshot, persisted checkpoint for that session), then keep serving.Everything else matches upstream: greedy check
any(isnan) || isinf(max), -inf masks with a finite winner remain legal, the lazy -1 sentinel checked by the reader, and an explicit error. If #104 is fixed by removing the lazy lane from the decode path, the lazy part of this issue shrinks to the remaining callers (if any).Description corrected after independent review (2026-10-03)
Updated the issue body in response to your review. Changes:
All eight issue descriptions now state the user's contract explicitly: Qwen 3.8 Flash Next follows pinned MTPLX; DSv4 Flash and GLM 5.3 Flash retain DS4 parity. Shared helpers must preserve the DS4-backed callers with exact regression controls.
For renewed verification: Compare cleanup with pinned MTPLX _non_finite_logits_failure and trace live, prefix, resident, persisted and pending-save ownership for one Qwen session. Confirm no global cache flush or DS4-family behavior change.
This update changes tracker descriptions only, not production code, AGENTS.md or the committed raw receipts. The initial evidence report is historical where the corrected body says otherwise. Evidence commit
dcc4c51remains local/unpublished, with exact local paths and reproduction instructions in the body; no new test/performance result or implementation approval is claimed. Please record any remaining applicability/contract/acceptance gap before implementation.Implemented in
5ecf8ab4f31aeaef7a06b4f03fbb8d658f28c486and pushed to main.Qwen now rejects NaN, +infinity and all-masked logits before consuming fabricated IDs. Greedy checks validity alongside argmax; sampled and sparse invalid-mass fallbacks raise explicit errors. Legal negative-infinity masks remain valid. Batched target checks operate per row, and the retained test-only lazy diagnostic returns -1 for invalid rows.
Numerical failure clears the affected live current/prefix, execution caches, resident aliases, checkpoint selection, persisted checkpoints/indexes/staging files, and bootstrap/conversation ancestry. A later save cannot republish state. Unrelated healthy sessions remain byte-identical, resident and restorable; DS4/GLM sampler/RNG and failure semantics are unchanged.
Oracle review: ordinary behavior matches MTPLX 9882703f3105363ddc37eca9f97aa09a1d387112 / MLX 0.32.2. Some specialized greedy MTP paths still fabricate token 0 upstream. The user explicitly approved stricter invalid-row rejection throughout Qwen MTP on 2026-10-04; this exception is documented rather than claimed as parity.
Validation: 1,080 fresh ordinary-sampler fixtures (three steps, exact tokens/probabilities/RNG/evaluation boundaries); 150 fallback, 653 batched, 100 mixed-distribution and 387 historical RNG/lazy cases; exact four-row oracle census/sentinel checks; mixed-validity batched target regression. Installed-model product tests passed 24 AR and 32 MTP cases through streaming/non-streaming HTTP adapters, including real prefill snapshots, target/draft/final-capture failures, bootstrap/disk/resident cleanup, refusal of later persistence, healthy-session retention and valid cold retries. Target tests exercise both context-copy and serial verification; draft tests force the actual draft head through the supported penalty policy.
All commit gates passed: cargo fmt, Clippy all targets/features with warnings denied, signed make bundle, cargo test --all-features (308 application tests passed, 214 ignored; 4+4 tool tests). Required ignored GPU/product tests above were run explicitly under the memory supervisor.
Implementation review and reproducible receipts.