Implement the complete DeepSeek Mojo-Metal backend with performance parity #87

New Issue

hugo · 2026-08-31T12:11:30Z

hugo commented

2026-08-31 12:11:30 +00:00

Goal

Implement the complete DeepSeek Mojo backend on Apple Metal while preserving exact DS4 semantics and meeting or exceeding the existing DS4Server Metal backend.

Blocked by #84 through #86. CUDA support is a later optional issue and is not part of this definition of done.

Scope

Implement load, prefill, decode, and backend-neutral checkpoint state for the primary DeepSeek Flash model, then the supported Flash, Pro, and V4 variants.
Cover resident weights, embedding, norm, activation, RoPE, matrix and fused projections, HC, QKV, KV compression and storage, raw, compressed and indexed attention, router, top-k, routed experts, and shared experts.
Support every quantization and mixed expert layout used by shipped DeepSeek artifacts, including DS4-specific formats.
Add long context, compressed or sparse attention, resident and SSD expert streaming, checkpoint cross-load, steering, and MTP or speculative decode sequentially. Each mode needs an isolated correctness and performance row.
Keep common operations under ops and genuine DeepSeek composition or fusion under models/deepseek.
Keep native Metal selectable for tests throughout migration. No benchmark may mix native and Mojo kernels.
Preserve target-neutral data contracts where this costs nothing, but do not add CUDA branches, Linux packaging, or NVIDIA tuning.

Correctness gates

DS4 token processing, context accounting, cache and index behavior, checkpoint semantics, logits, argmax, sampled tokens, and complete greedy output remain unchanged. Prefill plus decode and incremental decode produce equivalent state. Checkpoints cross-load where state remains backend neutral. Every operation closes its #85 coverage entry with real shapes and quantized bytes.

Performance gates

Mojo-Metal must meet or exceed DS4Server native Metal on the same Apple hardware for critical operations, complete layers, prefill, time to first token, stable decode, p95 latency, memory, transfers, and relevant cold start. Long context, streaming, steering, and MTP are measured independently and cannot inherit a green baseline.

Acceptance criteria

Complete end-to-end A/B tests pass on Apple hardware for short and long contexts.
At least 128 generated greedy tokens match, with decisive logits or top-k values checked after every step.
Optional modes are enabled only after their individual gates pass.
Coverage, fixtures, raw reports, binary closure, and licenses are current.
Mojo remains opt-in until the final Metal release issue is complete.
The later CUDA issue can reuse model and operation semantics without changing the proven Metal path.

Binding outcome from #84 (2026-08-31)

#84 stopped product integration on Mojo 1.0.0 because the open compiler/standard-library environment has no host-side Metal DeviceContext. The working max.gpu.host path requires MAX plus AsyncRT/KGEN dylibs whose component-specific redistribution permission could not be established. mojo build --emit shared-lib also links @rpath/libKGENCompilerRTShared.dylib for a trivial export.

The DeepSeek port cannot begin on MAX-backed GPU APIs. Preserve native Metal as the oracle and production implementation until the shared foundation is unblocked and accepted.

These constraints are acceptance criteria:

keep Mojo pinned to 1.0.0 through the committed uv.lock; do not use nightly or an older release;
do not add max, max-core, max-mojo-libs, AsyncRT, or KGEN to the product/build/package closure without an authoritative redistribution grant naming every shipped component;
do not treat Apache-licensed MAX Mojo source as permission to redistribute separately delivered runtime binaries;
target Apple silicon and macOS 15 or newer, while preserving native Metal as production until the runtime gate passes;
before implementation resumes, re-run the #84 closure, clean-machine packaging, code-signing, and same-workload parity gates documented in docs/MOJO_SPIKE_84.md;
CUDA remains a separate Linux/NVIDIA toolchain, runtime-closure, redistribution, and hardware-validation decision.

If the runtime grant or open host API changes, update #84 evidence first so this milestone has one source of truth.

## Goal Implement the complete DeepSeek Mojo backend on Apple Metal while preserving exact DS4 semantics and meeting or exceeding the existing DS4Server Metal backend. Blocked by #84 through #86. CUDA support is a later optional issue and is not part of this definition of done. ## Scope - Implement load, prefill, decode, and backend-neutral checkpoint state for the primary DeepSeek Flash model, then the supported Flash, Pro, and V4 variants. - Cover resident weights, embedding, norm, activation, RoPE, matrix and fused projections, HC, QKV, KV compression and storage, raw, compressed and indexed attention, router, top-k, routed experts, and shared experts. - Support every quantization and mixed expert layout used by shipped DeepSeek artifacts, including DS4-specific formats. - Add long context, compressed or sparse attention, resident and SSD expert streaming, checkpoint cross-load, steering, and MTP or speculative decode sequentially. Each mode needs an isolated correctness and performance row. - Keep common operations under ops and genuine DeepSeek composition or fusion under models/deepseek. - Keep native Metal selectable for tests throughout migration. No benchmark may mix native and Mojo kernels. - Preserve target-neutral data contracts where this costs nothing, but do not add CUDA branches, Linux packaging, or NVIDIA tuning. ## Correctness gates DS4 token processing, context accounting, cache and index behavior, checkpoint semantics, logits, argmax, sampled tokens, and complete greedy output remain unchanged. Prefill plus decode and incremental decode produce equivalent state. Checkpoints cross-load where state remains backend neutral. Every operation closes its #85 coverage entry with real shapes and quantized bytes. ## Performance gates Mojo-Metal must meet or exceed DS4Server native Metal on the same Apple hardware for critical operations, complete layers, prefill, time to first token, stable decode, p95 latency, memory, transfers, and relevant cold start. Long context, streaming, steering, and MTP are measured independently and cannot inherit a green baseline. ## Acceptance criteria - Complete end-to-end A/B tests pass on Apple hardware for short and long contexts. - At least 128 generated greedy tokens match, with decisive logits or top-k values checked after every step. - Optional modes are enabled only after their individual gates pass. - Coverage, fixtures, raw reports, binary closure, and licenses are current. - Mojo remains opt-in until the final Metal release issue is complete. - The later CUDA issue can reuse model and operation semantics without changing the proven Metal path. ## Binding outcome from #84 (2026-08-31) #84 stopped product integration on Mojo 1.0.0 because the open compiler/standard-library environment has no host-side Metal `DeviceContext`. The working `max.gpu.host` path requires MAX plus AsyncRT/KGEN dylibs whose component-specific redistribution permission could not be established. `mojo build --emit shared-lib` also links `@rpath/libKGENCompilerRTShared.dylib` for a trivial export. The DeepSeek port cannot begin on MAX-backed GPU APIs. Preserve native Metal as the oracle and production implementation until the shared foundation is unblocked and accepted. These constraints are acceptance criteria: - keep Mojo pinned to 1.0.0 through the committed `uv.lock`; do not use nightly or an older release; - do not add `max`, `max-core`, `max-mojo-libs`, AsyncRT, or KGEN to the product/build/package closure without an authoritative redistribution grant naming every shipped component; - do not treat Apache-licensed MAX Mojo source as permission to redistribute separately delivered runtime binaries; - target Apple silicon and macOS 15 or newer, while preserving native Metal as production until the runtime gate passes; - before implementation resumes, re-run the #84 closure, clean-machine packaging, code-signing, and same-workload parity gates documented in `docs/MOJO_SPIKE_84.md`; - CUDA remains a separate Linux/NVIDIA toolchain, runtime-closure, redistribution, and hardware-validation decision. If the runtime grant or open host API changes, update #84 evidence first so this milestone has one source of truth.

hugo added the idea label 2026-08-31 12:11:30 +00:00

hugo referenced this issue

2026-08-31 12:13:27 +00:00

Add a performance-parity Qwen3.8-27B Q4 and Q8 text backend on Metal #88

hugo added this to the Mojo GPU Backend Transition milestone 2026-08-31 12:14:25 +00:00

hugo changed title from ~~Implement the complete DeepSeek Mojo backend with native performance parity~~ to Implement the complete DeepSeek Mojo-Metal backend with performance parity

2026-08-31 12:30:21 +00:00

hugo commented

2026-08-31 14:01:23 +00:00

Based on findings in #84, all work on MOJO migration is stopped. Licene-hell.

hugo closed this issue

2026-08-31 14:01:23 +00:00

hugo commented

2026-08-31 15:31:51 +00:00

Project decision: the Mojo backend effort is canceled. All committed spike tooling, dependencies, lockfiles, and documentation were removed in 9a33c61; generated probe state was deleted as well. The independent Apple-silicon/macOS-15 product floor remains. This issue is no longer actionable.

Project decision: the Mojo backend effort is canceled. All committed spike tooling, dependencies, lockfiles, and documentation were removed in `9a33c61`; generated probe state was deleted as well. The independent Apple-silicon/macOS-15 product floor remains. This issue is no longer actionable.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/DS4Server#87