Implement the GLM 5.2 Mojo-Metal backend with isolated sparse-path gates #90

New Issue

hugo · 2026-08-31T12:13:28Z

hugo commented

2026-08-31 12:13:28 +00:00

Goal

Implement GLM 5.2 on the shared Mojo backend for Apple Metal while preserving DS4 behavior and meeting or exceeding DS4Server native Metal.

Blocked by the shared foundation, harness, and DeepSeek work. CUDA is deferred to the optional follow-up.

Scope

Compare pinned MAX GLM 5.2, Sparse MLA, index sharing, and sparse-attention sources with the current DS4 GLM path.
Reuse only operations whose layouts, indexing, precision, cache semantics, license closure, binary closure, and Metal performance pass #84 and #85. Implement DS4Server-owned Mojo kernels for all failures.
Add GLM config and weight binding, composition, RoPE, Compact KV, indexer, sparse or indexed attention, router and MoE, load, prefill, decode, checkpoint state, and telemetry.
Add MTP only after the baseline passes and measure it independently.
Reuse shared quantization, matrix, norm, attention, cache, routing, expert, and verify facilities. Keep genuine GLM composition or fusion under models/glm.
Add measured Apple tuning only. Do not implement CUDA, NVIDIA policies, or Linux packaging.

Correctness gates

DS4 remains authoritative for token flow, logits, context accounting, Compact KV, indexer decisions, sparse attention, routing, cache transitions, MTP, and checkpoints. Verify Compact KV, indexer, sparse attention, MoE, MTP, and checkpoint transfer separately before the complete model gate.

Performance gates

Compare Mojo-Metal only with DS4Server native Metal on the same Apple hardware. Isolate Compact KV, indexer, sparse attention, routing and experts, complete layers, prefill, first token, stable decode, long context, MTP, memory, and transfers. Every critical component and the complete model must satisfy #85 with no fallback.

Acceptance criteria

GLM 5.2 passes the complete Metal correctness and performance matrix.
Every operation closes its coverage entry with real shapes and fixture bytes.
MAX-derived source and runtime use has explicit version, license, redistribution, binary closure, and Metal benchmark evidence.
Binary, fixture, benchmark, and license inventories are current.
The later CUDA issue can reuse the proven semantic operations without changing Metal behavior.

Upstream material to reassess

Upstream implementations remain candidates, never replacements for the DS4 oracle.

Binding outcome from #84 (2026-08-31)

#84 stopped product integration on Mojo 1.0.0 because the open compiler/standard-library environment has no host-side Metal DeviceContext. The working max.gpu.host path requires MAX plus AsyncRT/KGEN dylibs whose component-specific redistribution permission could not be established. mojo build --emit shared-lib also links @rpath/libKGENCompilerRTShared.dylib for a trivial export.

The GLM port cannot begin on MAX-backed GPU APIs. Sparse/indexed operation inventory may proceed, but implementation depends on the approved shared host/runtime path.

These constraints are acceptance criteria:

keep Mojo pinned to 1.0.0 through the committed uv.lock; do not use nightly or an older release;
do not add max, max-core, max-mojo-libs, AsyncRT, or KGEN to the product/build/package closure without an authoritative redistribution grant naming every shipped component;
do not treat Apache-licensed MAX Mojo source as permission to redistribute separately delivered runtime binaries;
target Apple silicon and macOS 15 or newer, while preserving native Metal as production until the runtime gate passes;
before implementation resumes, re-run the #84 closure, clean-machine packaging, code-signing, and same-workload parity gates documented in docs/MOJO_SPIKE_84.md;
CUDA remains a separate Linux/NVIDIA toolchain, runtime-closure, redistribution, and hardware-validation decision.

If the runtime grant or open host API changes, update #84 evidence first so this milestone has one source of truth.

## Goal Implement GLM 5.2 on the shared Mojo backend for Apple Metal while preserving DS4 behavior and meeting or exceeding DS4Server native Metal. Blocked by the shared foundation, harness, and DeepSeek work. CUDA is deferred to the optional follow-up. ## Scope - Compare pinned MAX GLM 5.2, Sparse MLA, index sharing, and sparse-attention sources with the current DS4 GLM path. - Reuse only operations whose layouts, indexing, precision, cache semantics, license closure, binary closure, and Metal performance pass #84 and #85. Implement DS4Server-owned Mojo kernels for all failures. - Add GLM config and weight binding, composition, RoPE, Compact KV, indexer, sparse or indexed attention, router and MoE, load, prefill, decode, checkpoint state, and telemetry. - Add MTP only after the baseline passes and measure it independently. - Reuse shared quantization, matrix, norm, attention, cache, routing, expert, and verify facilities. Keep genuine GLM composition or fusion under models/glm. - Add measured Apple tuning only. Do not implement CUDA, NVIDIA policies, or Linux packaging. ## Correctness gates DS4 remains authoritative for token flow, logits, context accounting, Compact KV, indexer decisions, sparse attention, routing, cache transitions, MTP, and checkpoints. Verify Compact KV, indexer, sparse attention, MoE, MTP, and checkpoint transfer separately before the complete model gate. ## Performance gates Compare Mojo-Metal only with DS4Server native Metal on the same Apple hardware. Isolate Compact KV, indexer, sparse attention, routing and experts, complete layers, prefill, first token, stable decode, long context, MTP, memory, and transfers. Every critical component and the complete model must satisfy #85 with no fallback. ## Acceptance criteria - GLM 5.2 passes the complete Metal correctness and performance matrix. - Every operation closes its coverage entry with real shapes and fixture bytes. - MAX-derived source and runtime use has explicit version, license, redistribution, binary closure, and Metal benchmark evidence. - Binary, fixture, benchmark, and license inventories are current. - The later CUDA issue can reuse the proven semantic operations without changing Metal behavior. ## Upstream material to reassess - https://docs.modular.com/releases/v26.5/ - https://docs.modular.com/models/ Upstream implementations remain candidates, never replacements for the DS4 oracle. ## Binding outcome from #84 (2026-08-31) #84 stopped product integration on Mojo 1.0.0 because the open compiler/standard-library environment has no host-side Metal `DeviceContext`. The working `max.gpu.host` path requires MAX plus AsyncRT/KGEN dylibs whose component-specific redistribution permission could not be established. `mojo build --emit shared-lib` also links `@rpath/libKGENCompilerRTShared.dylib` for a trivial export. The GLM port cannot begin on MAX-backed GPU APIs. Sparse/indexed operation inventory may proceed, but implementation depends on the approved shared host/runtime path. These constraints are acceptance criteria: - keep Mojo pinned to 1.0.0 through the committed `uv.lock`; do not use nightly or an older release; - do not add `max`, `max-core`, `max-mojo-libs`, AsyncRT, or KGEN to the product/build/package closure without an authoritative redistribution grant naming every shipped component; - do not treat Apache-licensed MAX Mojo source as permission to redistribute separately delivered runtime binaries; - target Apple silicon and macOS 15 or newer, while preserving native Metal as production until the runtime gate passes; - before implementation resumes, re-run the #84 closure, clean-machine packaging, code-signing, and same-workload parity gates documented in `docs/MOJO_SPIKE_84.md`; - CUDA remains a separate Linux/NVIDIA toolchain, runtime-closure, redistribution, and hardware-validation decision. If the runtime grant or open host API changes, update #84 evidence first so this milestone has one source of truth.

hugo added the idea label 2026-08-31 12:13:28 +00:00

hugo added this to the Mojo GPU Backend Transition milestone 2026-08-31 12:14:26 +00:00

hugo changed title from ~~Implement the GLM 5.2 Mojo backend with isolated sparse-path parity gates~~ to Implement the GLM 5.2 Mojo-Metal backend with isolated sparse-path gates

2026-08-31 12:29:54 +00:00

hugo commented

2026-08-31 14:01:48 +00:00

Based on findings in #84, all work on MOJO migration is stopped. Licene-hell.

hugo closed this issue

2026-08-31 14:01:48 +00:00

hugo commented

2026-08-31 15:31:51 +00:00

Project decision: the Mojo backend effort is canceled. All committed spike tooling, dependencies, lockfiles, and documentation were removed in 9a33c61; generated probe state was deleted as well. The independent Apple-silicon/macOS-15 product floor remains. This issue is no longer actionable.

Project decision: the Mojo backend effort is canceled. All committed spike tooling, dependencies, lockfiles, and documentation were removed in `9a33c61`; generated probe state was deleted as well. The independent Apple-silicon/macOS-15 product floor remains. This issue is no longer actionable.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/DS4Server#90