Implement the shared Mojo foundation and a quantized Metal vertical slice #86

New Issue

hugo · 2026-08-31T12:11:30Z

hugo commented

2026-08-31 12:11:30 +00:00

Goal

Build the smallest production-shaped Mojo foundation and prove the hardest representative quantized routed-MoE path on Apple Metal before translating the complete engine.

Blocked by #84 and #85. CUDA is deferred and must not block this Metal gate.

Architecture

Keep the dependency direction:

api -> models -> ops -> runtime and targets -> Mojo or approved dependencies

Rust retains GGUF parsing, token and model flow, sessions, context accounting, sampling, checkpoint format, backend selection, and end-to-end telemetry.
Mojo owns device and tensor memory, packed weights, GPU operations and fusion, command execution, device cache state, prefill, decode, optional verify, and device telemetry.
Runtime knows no model semantics. Target policies contain no complete models. Models share code only through real reusable operations.
Do not create a graph IR, plugin framework, second GGUF parser, or per-kernel Rust FFI surface.

Implement the Apple policy and only the shared seams demonstrably needed for a future target. Do not implement NVIDIA policies, CUDA build support, or CUDA kernels in this issue.

Scope

Implement only the runtime, Apple target policy, tensor and memory facilities, errors, telemetry, and shared operations required by the slice.
Follow #84 component decisions. A MAX kernel is eligible only if both license and Metal performance gates pass; otherwise implement a DS4Server-owned Mojo kernel based on the native Metal oracle.
Implement a complete representative layer containing a difficult production quantization format, routed experts, attention, and KV state for prefill and decode using real model shapes and identical fixture bytes.
Keep tensors and temporary storage resident and cross the Rust-Mojo boundary only through coarse calls.
Add only measured Apple tile, MMA, vector width, and launch policies needed for parity.
Mirror public operations with a data-driven correctness test and representative benchmark entry.

Gates

Results match native Metal under the #85 correctness contract.
Mojo-Metal meets or exceeds native Metal for the full slice, including operation timing, layer wall-clock time, memory, transfers, and command count.
No extra token-path transfer or silent fallback.
Shared quantization, matrix, norm, attention, cache, and MoE code is not duplicated in model or target directories.
Every Apple specialization has measured evidence.
Binary and license audits are closed for the produced macOS artifact.

If parity cannot be reached after profiling, fusion, and justified Apple policies, stop the broad port and update the remaining issues rather than translating easier kernels first.

Upstream candidates to reassess

Binding outcome from #84 (2026-08-31)

#84 stopped product integration on Mojo 1.0.0 because the open compiler/standard-library environment has no host-side Metal DeviceContext. The working max.gpu.host path requires MAX plus AsyncRT/KGEN dylibs whose component-specific redistribution permission could not be established. mojo build --emit shared-lib also links @rpath/libKGENCompilerRTShared.dylib for a trivial export.

Do not create the shared Mojo GPU foundation or quantized vertical slice while the host/runtime closure is blocked. Re-run #84 first; the first slice must include an approved package closure and clean-machine app proof.

These constraints are acceptance criteria:

keep Mojo pinned to 1.0.0 through the committed uv.lock; do not use nightly or an older release;
do not add max, max-core, max-mojo-libs, AsyncRT, or KGEN to the product/build/package closure without an authoritative redistribution grant naming every shipped component;
do not treat Apache-licensed MAX Mojo source as permission to redistribute separately delivered runtime binaries;
target Apple silicon and macOS 15 or newer, while preserving native Metal as production until the runtime gate passes;
before implementation resumes, re-run the #84 closure, clean-machine packaging, code-signing, and same-workload parity gates documented in docs/MOJO_SPIKE_84.md;
CUDA remains a separate Linux/NVIDIA toolchain, runtime-closure, redistribution, and hardware-validation decision.

If the runtime grant or open host API changes, update #84 evidence first so this milestone has one source of truth.

## Goal Build the smallest production-shaped Mojo foundation and prove the hardest representative quantized routed-MoE path on Apple Metal before translating the complete engine. Blocked by #84 and #85. CUDA is deferred and must not block this Metal gate. ## Architecture Keep the dependency direction: api -> models -> ops -> runtime and targets -> Mojo or approved dependencies - Rust retains GGUF parsing, token and model flow, sessions, context accounting, sampling, checkpoint format, backend selection, and end-to-end telemetry. - Mojo owns device and tensor memory, packed weights, GPU operations and fusion, command execution, device cache state, prefill, decode, optional verify, and device telemetry. - Runtime knows no model semantics. Target policies contain no complete models. Models share code only through real reusable operations. - Do not create a graph IR, plugin framework, second GGUF parser, or per-kernel Rust FFI surface. Implement the Apple policy and only the shared seams demonstrably needed for a future target. Do not implement NVIDIA policies, CUDA build support, or CUDA kernels in this issue. ## Scope - Implement only the runtime, Apple target policy, tensor and memory facilities, errors, telemetry, and shared operations required by the slice. - Follow #84 component decisions. A MAX kernel is eligible only if both license and Metal performance gates pass; otherwise implement a DS4Server-owned Mojo kernel based on the native Metal oracle. - Implement a complete representative layer containing a difficult production quantization format, routed experts, attention, and KV state for prefill and decode using real model shapes and identical fixture bytes. - Keep tensors and temporary storage resident and cross the Rust-Mojo boundary only through coarse calls. - Add only measured Apple tile, MMA, vector width, and launch policies needed for parity. - Mirror public operations with a data-driven correctness test and representative benchmark entry. ## Gates - Results match native Metal under the #85 correctness contract. - Mojo-Metal meets or exceeds native Metal for the full slice, including operation timing, layer wall-clock time, memory, transfers, and command count. - No extra token-path transfer or silent fallback. - Shared quantization, matrix, norm, attention, cache, and MoE code is not duplicated in model or target directories. - Every Apple specialization has measured evidence. - Binary and license audits are closed for the produced macOS artifact. If parity cannot be reached after profiling, fusion, and justified Apple policies, stop the broad port and update the remaining issues rather than translating easier kernels first. ## Upstream candidates to reassess - https://docs.modular.com/stable/api/mojo/ - https://docs.modular.com/api/mojo/max/gpu/compute/arch/ - https://docs.modular.com/mojo/kernels/nn/flash_attention/ - https://docs.modular.com/stable/api/mojo/kv_cache/ - https://github.com/modular/modular/tree/main/max/kernels/src/quantization ## Binding outcome from #84 (2026-08-31) #84 stopped product integration on Mojo 1.0.0 because the open compiler/standard-library environment has no host-side Metal `DeviceContext`. The working `max.gpu.host` path requires MAX plus AsyncRT/KGEN dylibs whose component-specific redistribution permission could not be established. `mojo build --emit shared-lib` also links `@rpath/libKGENCompilerRTShared.dylib` for a trivial export. Do not create the shared Mojo GPU foundation or quantized vertical slice while the host/runtime closure is blocked. Re-run #84 first; the first slice must include an approved package closure and clean-machine app proof. These constraints are acceptance criteria: - keep Mojo pinned to 1.0.0 through the committed `uv.lock`; do not use nightly or an older release; - do not add `max`, `max-core`, `max-mojo-libs`, AsyncRT, or KGEN to the product/build/package closure without an authoritative redistribution grant naming every shipped component; - do not treat Apache-licensed MAX Mojo source as permission to redistribute separately delivered runtime binaries; - target Apple silicon and macOS 15 or newer, while preserving native Metal as production until the runtime gate passes; - before implementation resumes, re-run the #84 closure, clean-machine packaging, code-signing, and same-workload parity gates documented in `docs/MOJO_SPIKE_84.md`; - CUDA remains a separate Linux/NVIDIA toolchain, runtime-closure, redistribution, and hardware-validation decision. If the runtime grant or open host API changes, update #84 evidence first so this milestone has one source of truth.

hugo added the idea label 2026-08-31 12:11:30 +00:00

hugo referenced this issue

2026-08-31 12:11:30 +00:00

Implement the complete DeepSeek Mojo-Metal backend with performance parity #87

hugo added this to the Mojo GPU Backend Transition milestone 2026-08-31 12:14:25 +00:00

hugo changed title from ~~Implement the shared Mojo foundation and a quantized routed-MoE vertical slice~~ to Implement the shared Mojo foundation and a quantized Metal vertical slice

2026-08-31 12:30:21 +00:00

hugo commented

2026-08-31 14:01:14 +00:00

Based on findings in #84, all work on MOJO migration is stopped. Licene-hell.

hugo closed this issue

2026-08-31 14:01:14 +00:00

hugo commented

2026-08-31 15:31:50 +00:00

Project decision: the Mojo backend effort is canceled. All committed spike tooling, dependencies, lockfiles, and documentation were removed in 9a33c61; generated probe state was deleted as well. The independent Apple-silicon/macOS-15 product floor remains. This issue is no longer actionable.

Project decision: the Mojo backend effort is canceled. All committed spike tooling, dependencies, lockfiles, and documentation were removed in `9a33c61`; generated probe state was deleted as well. The independent Apple-silicon/macOS-15 product floor remains. This issue is no longer actionable.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/DS4Server#86