Implement the Qwen3.8 Flash Next transformer core in Rust and Metal: embeddings, hyper-connections, Gated DeltaNet layers, dense full-attention behavior within the sparse budget, MoE, final projection, and recurrent state ownership.
Sequence
Step 3 of 7. Previous: #94. Next: #96. Start only after the exact loader contract in #94 is complete; #96 builds on this executor.
Implementation
Add a dedicated qwen4exp Rust executor and dispatch it only after #94 validation succeeds.
Reuse the GLM 5.3 BF16 bridge, recurrent/conv cache ownership, checkpoint plumbing, command-buffer reuse, MoE scheduling, and prefill/small-batch infrastructure where semantics match.
Implement four-stream rank-320 low-rank hyper-connection mix/combine, zero-centered norms, and the absence of a conventional output norm exactly as specified.
Implement 512-expert top-10 routing plus shared expert with the Bare Speed tensor formats. Reuse existing quantized matmul paths before adding kernels.
Implement Qwen RoPE and an exact dense 24Q/2KV full-attention path for histories where the 2,048-token sparse budget covers all available tokens. Sparse selection beyond that budget belongs to #97.
Persist every GDN convolution and recurrent component in session checkpoints with versioned validation.
Keep orchestration in Rust and native work in .metal plus the existing small Objective-C bridge.
Verification
Add primitive vectors for zero-centered norms, hyper-connection mix/combine, GDN projection/conv/recurrence/output gate, router/top-10 selection, shared+routed MoE, RoPE, and dense attention.
Compare layer-boundary hidden states and logits against pinned reference vectors.
Prove prefill/decode equivalence across chunk sizes and checkpoint/resume equivalence for recurrent state.
Run existing DeepSeek and GLM regression vectors to prove unchanged output.
Completion requirements
Every transformer-core primitive matches reference vectors within the agreed dtype tolerance.
Dense short-context forward passes produce the expected logits with explicit errors for the still-missing PLE and sparse-QSA stages.
GDN state survives save/restore without drift.
No other model path changes behavior.
Non-goals
PLE lookup, sparse attention beyond the dense budget, MTP, vision, or final optimization.
## Area
Implement the Qwen3.8 Flash Next transformer core in Rust and Metal: embeddings, hyper-connections, Gated DeltaNet layers, dense full-attention behavior within the sparse budget, MoE, final projection, and recurrent state ownership.
## Sequence
Step 3 of 7. Previous: #94. Next: #96. Start only after the exact loader contract in #94 is complete; #96 builds on this executor.
## Implementation
- Add a dedicated `qwen4exp` Rust executor and dispatch it only after #94 validation succeeds.
- Reuse the GLM 5.3 BF16 bridge, recurrent/conv cache ownership, checkpoint plumbing, command-buffer reuse, MoE scheduling, and prefill/small-batch infrastructure where semantics match.
- Implement the Qwen-specific 48-layer 3:1 schedule, 16-key/48-value-head GDN geometry, convolution width 4, fp32 recurrent state, sigmoid output gate, and exact L2-normalization epsilon placement.
- Implement four-stream rank-320 low-rank hyper-connection mix/combine, zero-centered norms, and the absence of a conventional output norm exactly as specified.
- Implement 512-expert top-10 routing plus shared expert with the Bare Speed tensor formats. Reuse existing quantized matmul paths before adding kernels.
- Implement Qwen RoPE and an exact dense 24Q/2KV full-attention path for histories where the 2,048-token sparse budget covers all available tokens. Sparse selection beyond that budget belongs to #97.
- Persist every GDN convolution and recurrent component in session checkpoints with versioned validation.
- Keep orchestration in Rust and native work in `.metal` plus the existing small Objective-C bridge.
## Verification
- Add primitive vectors for zero-centered norms, hyper-connection mix/combine, GDN projection/conv/recurrence/output gate, router/top-10 selection, shared+routed MoE, RoPE, and dense attention.
- Compare layer-boundary hidden states and logits against pinned reference vectors.
- Prove prefill/decode equivalence across chunk sizes and checkpoint/resume equivalence for recurrent state.
- Run existing DeepSeek and GLM regression vectors to prove unchanged output.
## Completion requirements
- Every transformer-core primitive matches reference vectors within the agreed dtype tolerance.
- Dense short-context forward passes produce the expected logits with explicit errors for the still-missing PLE and sparse-QSA stages.
- GDN state survives save/restore without drift.
- No other model path changes behavior.
## Non-goals
- PLE lookup, sparse attention beyond the dense budget, MTP, vision, or final optimization.
hugo
added the idea label 2026-09-01 18:24:30 +00:00
Implemented in c414640 (Implement Qwen text core).
The server now opens the frozen Qwen safetensors set through a dedicated Rust model/executor path and runs the 48-layer 3:1 hybrid text graph with MLX affine Q2/Q4/Q8 and BF16 projection kernels, four-stream rank-320 hyper-connections, BF16 width-4 convolution history, fp32 Gated DeltaNet recurrence, 24Q/2KV dense causal attention through the 2,048-token budget, top-10-of-512 routed experts plus the shared expert, Qwen RoPE, final hyper mixer, and LM projection. Rust owns model maps, graph scheduling, session state, resident-state swaps, and versioned checkpoints; native additions are limited to Metal kernels and the existing Objective-C dispatch bridge. Missing layer-2 PLE and over-budget sparse QSA fail before unsafe or partial execution and remain explicitly assigned to #96 and #97.
Verification completed in isolation on Apple M5 Max against the pinned 105 GB artifact set:
The pinned layer-0 boundary and projected logits match frozen values.
Direct and prefill entry points return the identical missing-PLE boundary without mutating session state.
A computed recurrent continuation produces identical logits, BF16 convolution bytes, and fp32 recurrent bytes after checkpoint/resume.
The sparse boundary returns the explicit #97 error.
The installed MLX runtime was used only as an external packing oracle; no Python or C implementation code was added.
Required gates passed: cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (233 passed, 22 ignored, plus 4/4 artifact-tool tests). The initial full test run encountered the unrelated randomized dev_brain YAML fixture flake; its isolated rerun and the complete rerun both passed. DS4 has no Qwen3.8 path, so this is a new DS4Server feature while preserving existing DeepSeek and GLM behavior.
Implemented in c414640 (Implement Qwen text core).
The server now opens the frozen Qwen safetensors set through a dedicated Rust model/executor path and runs the 48-layer 3:1 hybrid text graph with MLX affine Q2/Q4/Q8 and BF16 projection kernels, four-stream rank-320 hyper-connections, BF16 width-4 convolution history, fp32 Gated DeltaNet recurrence, 24Q/2KV dense causal attention through the 2,048-token budget, top-10-of-512 routed experts plus the shared expert, Qwen RoPE, final hyper mixer, and LM projection. Rust owns model maps, graph scheduling, session state, resident-state swaps, and versioned checkpoints; native additions are limited to Metal kernels and the existing Objective-C dispatch bridge. Missing layer-2 PLE and over-budget sparse QSA fail before unsafe or partial execution and remain explicitly assigned to #96 and #97.
Verification completed in isolation on Apple M5 Max against the pinned 105 GB artifact set:
- Metal primitive vectors cover affine dequantization, zero-centered grouped RMS, hyper mix/combine, convolution, fp32 delta recurrence, sigmoid output gate, routed/shared MoE, top-10 normalization, Qwen RoPE, BF16 KV storage, and dense causal attention.
- The pinned layer-0 boundary and projected logits match frozen values.
- Direct and prefill entry points return the identical missing-PLE boundary without mutating session state.
- A computed recurrent continuation produces identical logits, BF16 convolution bytes, and fp32 recurrent bytes after checkpoint/resume.
- The sparse boundary returns the explicit #97 error.
- The installed MLX runtime was used only as an external packing oracle; no Python or C implementation code was added.
Required gates passed: cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (233 passed, 22 ignored, plus 4/4 artifact-tool tests). The initial full test run encountered the unrelated randomized dev_brain YAML fixture flake; its isolated rerun and the complete rerun both passed. DS4 has no Qwen3.8 path, so this is a new DS4Server feature while preserving existing DeepSeek and GLM behavior.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Area
Implement the Qwen3.8 Flash Next transformer core in Rust and Metal: embeddings, hyper-connections, Gated DeltaNet layers, dense full-attention behavior within the sparse budget, MoE, final projection, and recurrent state ownership.
Sequence
Step 3 of 7. Previous: #94. Next: #96. Start only after the exact loader contract in #94 is complete; #96 builds on this executor.
Implementation
qwen4expRust executor and dispatch it only after #94 validation succeeds..metalplus the existing small Objective-C bridge.Verification
Completion requirements
Non-goals
hugo referenced this issue2026-09-01 18:27:01 +00:00
Implemented in
c414640(Implement Qwen text core).The server now opens the frozen Qwen safetensors set through a dedicated Rust model/executor path and runs the 48-layer 3:1 hybrid text graph with MLX affine Q2/Q4/Q8 and BF16 projection kernels, four-stream rank-320 hyper-connections, BF16 width-4 convolution history, fp32 Gated DeltaNet recurrence, 24Q/2KV dense causal attention through the 2,048-token budget, top-10-of-512 routed experts plus the shared expert, Qwen RoPE, final hyper mixer, and LM projection. Rust owns model maps, graph scheduling, session state, resident-state swaps, and versioned checkpoints; native additions are limited to Metal kernels and the existing Objective-C dispatch bridge. Missing layer-2 PLE and over-budget sparse QSA fail before unsafe or partial execution and remain explicitly assigned to #96 and #97.
Verification completed in isolation on Apple M5 Max against the pinned 105 GB artifact set:
Required gates passed: cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (233 passed, 22 ignored, plus 4/4 artifact-tool tests). The initial full test run encountered the unrelated randomized dev_brain YAML fixture flake; its isolated rerun and the complete rerun both passed. DS4 has no Qwen3.8 path, so this is a new DS4Server feature while preserving existing DeepSeek and GLM behavior.