Bring DS4Server's Metal runtime and supported model catalog to parity with the latest official antirez/ds4main. The audited upstream pin is 8db89fe (2026-08-28); re-check and record the current main pin when implementation starts.
DeepSeek V4 Flash 0731 becomes the default. Keep the preview checkpoint functional for existing profiles, but label it deprecated and do not select it for new profiles.
Findings
DS4Server currently defaults to Flash preview. Its 0731 entry only adds a third-party main GGUF and explicitly disables MTP/DSpark; Pro still points at the undated preview artifact, and GLM is at the initial 5.2 port.
Official DS4 now pairs Flash 0731 with the checkpoint-specific DeepSeek-V4-Flash-DSpark-support-0731.gguf; it must not be mixed with preview. DSpark replaces, rather than stacks with, legacy one-stage MTP for that run (artifact transition).
The official 0731 Q2 artifact is in antirez/deepseek-v4-gguf with SHA-256 ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0; its DSpark support artifact has SHA-256 7e319924541db3f7a163ed7e11d7532a70d48228ab59d36cb81e1d4511885360.
Upstream added streamed-prefill correctness, faster indexed and routed-MoE Metal prefill, long-context RoPE fixes, direct verifier-state commit, pipelined DFlash verification, and current greedy/stochastic DSpark semantics. DS4Server's July Metal/Rust port predates these changes.
Current upstream main has runnable support for Flash 0731, DeepSeek V4 Pro 0813, and GLM 5.2. It only added GLM 5.3 Flash download targets; runtime/QA remains off main, so GLM 5.3 must not be advertised as supported yet.
Implementation plan
Pin and audit upstream. Diff the implementation pin against DS4Server using the existing parity tooling. Cover backend-neutral behavior plus ds4.c, ds4.h, ds4_metal.m, ds4_gpu.h, metal/*.metal, model validation, and upstream QA. Port semantics into the Rust engine and the existing Metal/Objective-C bridge; do not execute or embed upstream C.
Update models and migration. Make 0731 the Rust/config default while preserving saved preview profiles. Mark preview deprecated. Replace the third-party 0731 artifact with the official Q2 GGUF and add its official DSpark artifact. Move Pro to the dated 0813 GGUF and oracle. Keep GLM 5.2 on an upstream-tested layout and catch its runtime up to the pin.
Encode checkpoint capabilities. Replace the shared Flash boolean with an explicit compatibility matrix: preview uses only its preview support artifacts; 0731 uses only DSpark-0731 and never legacy MTP; Pro has no DSpark; GLM uses embedded MTP. Validate both model identity and support-GGUF identity before allocation or execution.
Port current execution behavior. Bring Rust orchestration, context/KV accounting, SSD streaming, prefill, decode, and sampling to the pinned DS4 behavior. Sync the final (not intermediate/reverted) Metal changes, including mapped-expert prefill correctness, dual-head indexed prefill, routed-MoE TensorOps paths, long-context RoPE, DFlash verification/accepted-state handling, strict greedy identity, and opportunistic/exact stochastic sampling.
Add parity evidence. Extend existing unit/fixture tests for defaults, profile migration, artifact hashes, invalid cross-checkpoint pairings, and shape validation. Add installed-model acceptance runs for preview, 0731 target-only, 0731 DSpark (normal/strict greedy, partial acceptance, seeded stochastic/exact), Pro 0813, and GLM 5.2 including embedded MTP. Cover resident and SSD-streamed short/long prefills, compare continuations and state/KV invariants with the pinned DS4 oracle, and record prefill/decode benchmarks before and after.
Acceptance criteria
New installs/profiles default to Flash 0731; existing preview profiles still load and clearly show deprecation.
Managed artifacts use the official checkpointed files and reject every incompatible main/support pairing before engine startup.
Flash 0731, Pro 0813, and GLM 5.2 match the pinned DS4 Metal behavior for the tested resident/streamed, prefill/decode, and speculative paths; the upstream prefill improvements are present without a measured regression.
Any additional model is added only if executable runtime support and QA are on upstream main at the implementation pin; catalog/download-only entries do not count as support.
No JavaScript runtime, external DS4 process, CUDA/ROCm port, or generic workflow work is introduced. All repository commit gates pass.
## Goal
Bring DS4Server's Metal runtime and supported model catalog to parity with the latest official [`antirez/ds4`](https://github.com/antirez/ds4) `main`. The audited upstream pin is [`8db89fe`](https://github.com/antirez/ds4/commit/8db89fe083ae4d17c9a2428ccd29803d3ae8f577) (2026-08-28); re-check and record the current `main` pin when implementation starts.
DeepSeek V4 Flash 0731 becomes the default. Keep the preview checkpoint functional for existing profiles, but label it deprecated and do not select it for new profiles.
## Findings
- DS4Server currently defaults to Flash preview. Its 0731 entry only adds a third-party main GGUF and explicitly disables MTP/DSpark; Pro still points at the undated preview artifact, and GLM is at the initial 5.2 port.
- Official DS4 now pairs Flash 0731 with the checkpoint-specific `DeepSeek-V4-Flash-DSpark-support-0731.gguf`; it must not be mixed with preview. DSpark replaces, rather than stacks with, legacy one-stage MTP for that run ([artifact transition](https://github.com/antirez/ds4/commit/ae504c963e975f8b76352a9bd5895f39909042f9)).
- The official 0731 Q2 artifact is in `antirez/deepseek-v4-gguf` with SHA-256 `ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0`; its DSpark support artifact has SHA-256 `7e319924541db3f7a163ed7e11d7532a70d48228ab59d36cb81e1d4511885360`.
- Upstream added streamed-prefill correctness, faster indexed and routed-MoE Metal prefill, long-context RoPE fixes, direct verifier-state commit, pipelined DFlash verification, and current greedy/stochastic DSpark semantics. DS4Server's July Metal/Rust port predates these changes.
- Current upstream `main` has runnable support for Flash 0731, DeepSeek V4 Pro 0813, and GLM 5.2. It only added GLM 5.3 Flash download targets; runtime/QA remains off `main`, so GLM 5.3 must not be advertised as supported yet.
## Implementation plan
1. **Pin and audit upstream.** Diff the implementation pin against DS4Server using the existing parity tooling. Cover backend-neutral behavior plus `ds4.c`, `ds4.h`, `ds4_metal.m`, `ds4_gpu.h`, `metal/*.metal`, model validation, and upstream QA. Port semantics into the Rust engine and the existing Metal/Objective-C bridge; do not execute or embed upstream C.
2. **Update models and migration.** Make 0731 the Rust/config default while preserving saved preview profiles. Mark preview deprecated. Replace the third-party 0731 artifact with the official Q2 GGUF and add its official DSpark artifact. Move Pro to the dated 0813 GGUF and oracle. Keep GLM 5.2 on an upstream-tested layout and catch its runtime up to the pin.
3. **Encode checkpoint capabilities.** Replace the shared Flash boolean with an explicit compatibility matrix: preview uses only its preview support artifacts; 0731 uses only DSpark-0731 and never legacy MTP; Pro has no DSpark; GLM uses embedded MTP. Validate both model identity and support-GGUF identity before allocation or execution.
4. **Port current execution behavior.** Bring Rust orchestration, context/KV accounting, SSD streaming, prefill, decode, and sampling to the pinned DS4 behavior. Sync the final (not intermediate/reverted) Metal changes, including mapped-expert prefill correctness, dual-head indexed prefill, routed-MoE TensorOps paths, long-context RoPE, DFlash verification/accepted-state handling, strict greedy identity, and opportunistic/exact stochastic sampling.
5. **Add parity evidence.** Extend existing unit/fixture tests for defaults, profile migration, artifact hashes, invalid cross-checkpoint pairings, and shape validation. Add installed-model acceptance runs for preview, 0731 target-only, 0731 DSpark (normal/strict greedy, partial acceptance, seeded stochastic/exact), Pro 0813, and GLM 5.2 including embedded MTP. Cover resident and SSD-streamed short/long prefills, compare continuations and state/KV invariants with the pinned DS4 oracle, and record prefill/decode benchmarks before and after.
## Acceptance criteria
- New installs/profiles default to Flash 0731; existing preview profiles still load and clearly show deprecation.
- Managed artifacts use the official checkpointed files and reject every incompatible main/support pairing before engine startup.
- Flash 0731, Pro 0813, and GLM 5.2 match the pinned DS4 Metal behavior for the tested resident/streamed, prefill/decode, and speculative paths; the upstream prefill improvements are present without a measured regression.
- Any additional model is added only if executable runtime support and QA are on upstream `main` at the implementation pin; catalog/download-only entries do not count as support.
- No JavaScript runtime, external DS4 process, CUDA/ROCm port, or generic workflow work is introduced. All repository commit gates pass.
hugo
added the bug label 2026-08-07 11:43:40 +00:00
Implemented in f1c177b. DeepSeek V4 Flash 0731 is now the default managed checkpoint with the official target and checkpoint-specific DSpark artifacts; preview Flash remains available and is marked deprecated, Pro tracks the dated 0813 artifact, and incompatible target/support pairs are rejected by exact filename, size, and verified checksum identity before startup. Synced the Rust/Metal execution path to audited DS4 main 8db89fe, including current prefill/SSD kernels and long-context behavior, per-layer routed prefill mapping, direct partial verifier-state commits, opportunistic sampled DSpark, and exact stochastic DSpark acceptance/rejection sampling. Added capability, identity, sampling, preferences, and installed-model acceptance coverage. Verified in isolation with cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (171 passed, 14 fixture/hardware-gated). The official 0731 target/support files are not installed on this machine, so that explicit 93 GB hardware acceptance test remains ignored rather than running against the older third-party 0731 file.
Implemented in f1c177b. DeepSeek V4 Flash 0731 is now the default managed checkpoint with the official target and checkpoint-specific DSpark artifacts; preview Flash remains available and is marked deprecated, Pro tracks the dated 0813 artifact, and incompatible target/support pairs are rejected by exact filename, size, and verified checksum identity before startup. Synced the Rust/Metal execution path to audited DS4 main 8db89fe, including current prefill/SSD kernels and long-context behavior, per-layer routed prefill mapping, direct partial verifier-state commits, opportunistic sampled DSpark, and exact stochastic DSpark acceptance/rejection sampling. Added capability, identity, sampling, preferences, and installed-model acceptance coverage. Verified in isolation with cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (171 passed, 14 fixture/hardware-gated). The official 0731 target/support files are not installed on this machine, so that explicit 93 GB hardware acceptance test remains ignored rather than running against the older third-party 0731 file.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Goal
Bring DS4Server's Metal runtime and supported model catalog to parity with the latest official
antirez/ds4main. The audited upstream pin is8db89fe(2026-08-28); re-check and record the currentmainpin when implementation starts.DeepSeek V4 Flash 0731 becomes the default. Keep the preview checkpoint functional for existing profiles, but label it deprecated and do not select it for new profiles.
Findings
DeepSeek-V4-Flash-DSpark-support-0731.gguf; it must not be mixed with preview. DSpark replaces, rather than stacks with, legacy one-stage MTP for that run (artifact transition).antirez/deepseek-v4-ggufwith SHA-256ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0; its DSpark support artifact has SHA-2567e319924541db3f7a163ed7e11d7532a70d48228ab59d36cb81e1d4511885360.mainhas runnable support for Flash 0731, DeepSeek V4 Pro 0813, and GLM 5.2. It only added GLM 5.3 Flash download targets; runtime/QA remains offmain, so GLM 5.3 must not be advertised as supported yet.Implementation plan
ds4.c,ds4.h,ds4_metal.m,ds4_gpu.h,metal/*.metal, model validation, and upstream QA. Port semantics into the Rust engine and the existing Metal/Objective-C bridge; do not execute or embed upstream C.Acceptance criteria
mainat the implementation pin; catalog/download-only entries do not count as support.ds4 now has support for 9731to ds4 now has support for 0731Implemented in
f1c177b. DeepSeek V4 Flash 0731 is now the default managed checkpoint with the official target and checkpoint-specific DSpark artifacts; preview Flash remains available and is marked deprecated, Pro tracks the dated 0813 artifact, and incompatible target/support pairs are rejected by exact filename, size, and verified checksum identity before startup. Synced the Rust/Metal execution path to audited DS4 main 8db89fe, including current prefill/SSD kernels and long-context behavior, per-layer routed prefill mapping, direct partial verifier-state commits, opportunistic sampled DSpark, and exact stochastic DSpark acceptance/rejection sampling. Added capability, identity, sampling, preferences, and installed-model acceptance coverage. Verified in isolation with cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (171 passed, 14 fixture/hardware-gated). The official 0731 target/support files are not installed on this machine, so that explicit 93 GB hardware acceptance test remains ignored rather than running against the older third-party 0731 file.