Integrate Qwen3.8 Flash Next loading and tokenizer behavior #94

New Issue

hugo · 2026-09-01T18:24:14Z

hugo commented

2026-09-01 18:24:14 +00:00

Area

Integrate the frozen Qwen3.8 Flash Next artifacts into DS4Server's model catalog, download/verification flow, tokenizer, chat formatting, and strict loader. This issue owns loading behavior completely; it does not implement transformer execution.

Sequence

Step 2 of 7. Previous: #93. Next: #95. Start only after #93 has frozen artifact hashes; #95 starts only after this loader contract is complete.

Implementation

Add the exact Bare Speed artifact set from #93 to the existing model catalog and native UI, including byte sizes, hashes, download progress, verification, removal, and recovery from partial downloads.
Recognize the model from validated qwen4exp metadata rather than filenames. Validate all fixed architecture values from the official config before allocating model memory: 48 layers in a 3 GDN + 1 full-attention cycle, hidden size 2560, GDN and QSA head geometry, rank-320 four-stream hyper-connections, 512/top-10 MoE, layer-2 PLE, MTP layout, vocabulary, and native context.
Parse and bind the resident core, PLE, and MTP artifacts separately. Reject missing PLE, unexpected tensor precision/layout, wrong hashes, stale manifests, and MTP advertised without its required tensors.
Implement the official tokenizer and chat contract through the existing Rust tokenizer path, including thinking/direct modes, tools, multi-turn reasoning history, special tokens, EOS/reset behavior, and official sampling defaults of temperature 1.0, top-p 0.95, and top-k 20.
Extend memory admission to report core bytes, mapped PLE bytes, optional-at-runtime MTP bytes, KV/recurrent state, and prefill transients separately. Loading must refuse an impossible configuration before GPU allocation.
Expose a clear backend-not-yet-available result until #95 lands; do not fall through to another model executor.

Verification

Add valid and invalid metadata/tensor fixtures for every required rejection path.
Match tokenizer and rendered-chat golden vectors against the pinned Bare Speed files for thinking, direct, tools, multi-turn history, EOS, and PLE reset boundaries.
Exercise download interruption, hash mismatch, missing sidecars, cleanup, and re-verification.
Prove model selection survives restart and never loads through filename heuristics.

Completion requirements

The model can be selected, downloaded, verified, removed, and reloaded through existing DS4Server flows.
Loader validation fails closed before inference on every incompatible artifact condition.
Token IDs and chat prompts match the pinned model contract.
Memory admission accounts for all three artifact classes and runtime state.
Existing model catalog and tokenizer behavior remain unchanged.

Non-goals

Transformer kernels, PLE gathering, sparse QSA execution, MTP decoding, vision, or speed work.

## Area Integrate the frozen Qwen3.8 Flash Next artifacts into DS4Server's model catalog, download/verification flow, tokenizer, chat formatting, and strict loader. This issue owns loading behavior completely; it does not implement transformer execution. ## Sequence Step 2 of 7. Previous: #93. Next: #95. Start only after #93 has frozen artifact hashes; #95 starts only after this loader contract is complete. ## Implementation - Add the exact Bare Speed artifact set from #93 to the existing model catalog and native UI, including byte sizes, hashes, download progress, verification, removal, and recovery from partial downloads. - Recognize the model from validated `qwen4exp` metadata rather than filenames. Validate all fixed architecture values from the official config before allocating model memory: 48 layers in a 3 GDN + 1 full-attention cycle, hidden size 2560, GDN and QSA head geometry, rank-320 four-stream hyper-connections, 512/top-10 MoE, layer-2 PLE, MTP layout, vocabulary, and native context. - Parse and bind the resident core, PLE, and MTP artifacts separately. Reject missing PLE, unexpected tensor precision/layout, wrong hashes, stale manifests, and MTP advertised without its required tensors. - Implement the official tokenizer and chat contract through the existing Rust tokenizer path, including thinking/direct modes, tools, multi-turn reasoning history, special tokens, EOS/reset behavior, and official sampling defaults of temperature 1.0, top-p 0.95, and top-k 20. - Extend memory admission to report core bytes, mapped PLE bytes, optional-at-runtime MTP bytes, KV/recurrent state, and prefill transients separately. Loading must refuse an impossible configuration before GPU allocation. - Expose a clear backend-not-yet-available result until #95 lands; do not fall through to another model executor. ## Verification - Add valid and invalid metadata/tensor fixtures for every required rejection path. - Match tokenizer and rendered-chat golden vectors against the pinned Bare Speed files for thinking, direct, tools, multi-turn history, EOS, and PLE reset boundaries. - Exercise download interruption, hash mismatch, missing sidecars, cleanup, and re-verification. - Prove model selection survives restart and never loads through filename heuristics. ## Completion requirements - The model can be selected, downloaded, verified, removed, and reloaded through existing DS4Server flows. - Loader validation fails closed before inference on every incompatible artifact condition. - Token IDs and chat prompts match the pinned model contract. - Memory admission accounts for all three artifact classes and runtime state. - Existing model catalog and tokenizer behavior remain unchanged. ## Non-goals - Transformer kernels, PLE gathering, sparse QSA execution, MTP decoding, vision, or speed work.

hugo added the idea label 2026-09-01 18:24:14 +00:00

hugo referenced this issue

2026-09-01 18:27:01 +00:00

Prepare exact Qwen3.8 Flash Next Bare Speed artifacts #93

hugo changed title from ~~Define Qwen3.8 Flash Next artifacts and model contract~~ to Integrate Qwen3.8 Flash Next loading and tokenizer behavior

2026-09-01 18:36:51 +00:00

hugo referenced this issue

2026-09-01 18:36:51 +00:00

Implement the Qwen3.8 Flash Next Rust and Metal text core #95

hugo added enhancement and removed idea labels 2026-09-01 18:41:51 +00:00

hugo commented

2026-09-03 17:57:00 +00:00

Implemented and verified in commit 3773cfd (Integrate Qwen3.8 model intake).\n\nThe native model catalog and Model Manager now treat the pinned 29-file, 105,438,842,430-byte Bare Speed set as one resumable managed artifact with immutable revision URLs, per-file SHA-256 verification, partial recovery, strict promotion, re-verification, and complete removal. The Rust loader validates every frozen config/runtime pointer and all 2,527 tensor records, binds the 2,466 core, 3 PLE, and 58 MTP tensors separately, reports core/PLE/MTP/KV-recurrent/prefill memory classes, enforces the 262,144-token ceiling before allocation, and returns an explicit backend-unavailable result for #95.\n\nThe existing Rust tokenizer path now loads the pinned Qwen byte-level BPE, preserves exact special-token IDs and EOS/PLE reset boundaries, renders the official thinking/direct/tool/multi-turn ChatML contract, exposes only low/medium/xhigh/none, uses temperature 1.0, top-p 0.95, and top-k 20, and persists model/mode selection across restart. Qwen tool calls use the official nested function/parameter transport. Existing models retain their prior behavior.\n\nVerification: cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (233 passed, 20 ignored, plus 4/4 artifact-tool tests). The pinned-source loader/chat golden test passed against /private/tmp/qwen38-bare-source. The isolated full managed-artifact test hashed all 105,438,842,430 bytes, rejected missing MTP before hashing, validated and reloaded the complete set, and removed it cleanly (682.42s). DS4 contains no Qwen3.8 implementation, so this is a new DS4Server feature. All implementation code is Rust; no C or Python code was introduced.

Implemented and verified in commit 3773cfd (Integrate Qwen3.8 model intake).\n\nThe native model catalog and Model Manager now treat the pinned 29-file, 105,438,842,430-byte Bare Speed set as one resumable managed artifact with immutable revision URLs, per-file SHA-256 verification, partial recovery, strict promotion, re-verification, and complete removal. The Rust loader validates every frozen config/runtime pointer and all 2,527 tensor records, binds the 2,466 core, 3 PLE, and 58 MTP tensors separately, reports core/PLE/MTP/KV-recurrent/prefill memory classes, enforces the 262,144-token ceiling before allocation, and returns an explicit backend-unavailable result for #95.\n\nThe existing Rust tokenizer path now loads the pinned Qwen byte-level BPE, preserves exact special-token IDs and EOS/PLE reset boundaries, renders the official thinking/direct/tool/multi-turn ChatML contract, exposes only low/medium/xhigh/none, uses temperature 1.0, top-p 0.95, and top-k 20, and persists model/mode selection across restart. Qwen tool calls use the official nested function/parameter transport. Existing models retain their prior behavior.\n\nVerification: cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (233 passed, 20 ignored, plus 4/4 artifact-tool tests). The pinned-source loader/chat golden test passed against /private/tmp/qwen38-bare-source. The isolated full managed-artifact test hashed all 105,438,842,430 bytes, rejected missing MTP before hashing, validated and reloaded the complete set, and removed it cleanly (682.42s). DS4 contains no Qwen3.8 implementation, so this is a new DS4Server feature. All implementation code is Rust; no C or Python code was introduced.

hugo closed this issue

2026-09-03 17:58:16 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/DS4Server#94