Complete Qwen3.8-27B on Apple Metal by adding MTP and image or video inference as separately measurable extensions after #88.
CUDA is deferred and does not block this issue.
Scope and order
Implement and accept these steps sequentially:
Qwen-specific MTP heads and shared verify scheduling.
Image preprocessing and visual tokenization in Rust.
Vision encoder and projector in Mojo.
Combined visual-token and text prefill through the existing text model.
Video only if it belongs to the DS4Server product scope when implemented.
Keep model weights and heads under models/qwen3_8. Move a vision operation into shared ops only after a second real consumer needs identical semantics. Do not add speculative cross-platform or universal vision abstractions.
Correctness gates
MTP candidate generation, verification, cache updates, rollback, accepted tokens, and output match the pinned Qwen Metal reference.
Image preprocessing, visual-token order, projection, context accounting, checkpoints, and multimodal output match within operation-specific tolerances.
Text-only requests use the existing text path and retain the same tokens and state when MTP or visual input is absent.
Do not describe Qwen as fully multimodal until vision passes end-to-end.
Performance gates
Measure baseline decode, MTP draft and verify work, accepted-token throughput, memory, image preprocessing, vision encoder, projector, combined prefill, transfers, and cold start separately on the same Mac. Mojo-Metal must meet the native Metal reference for each enabled feature. A faster text backend cannot hide a slower MTP or vision stage, and absent visual input may not add text-path work.
Acceptance criteria
Qwen MTP passes Metal correctness and performance parity.
Representative image and image-text prompts pass Metal correctness, memory, and prefill parity.
Video is either covered by equivalent Metal gates or explicitly unsupported.
Coverage, fixture provenance, raw results, binary closure, and licenses are updated.
No native fallback or mixed backend execution appears in a claimed Mojo result.
Data contracts remain suitable for the later optional CUDA issue without adding CUDA work here.
#84 stopped product integration on Mojo 1.0.0 because the open compiler/standard-library environment has no host-side Metal DeviceContext. The working max.gpu.host path requires MAX plus AsyncRT/KGEN dylibs whose component-specific redistribution permission could not be established. mojo build --emit shared-lib also links @rpath/libKGENCompilerRTShared.dylib for a trivial export.
MTP, multimodal, and Gated DeltaNet design work must retain the same backend boundary. Do not introduce MAX to obtain missing host-side GPU functionality.
These constraints are acceptance criteria:
keep Mojo pinned to 1.0.0 through the committed uv.lock; do not use nightly or an older release;
do not add max, max-core, max-mojo-libs, AsyncRT, or KGEN to the product/build/package closure without an authoritative redistribution grant naming every shipped component;
do not treat Apache-licensed MAX Mojo source as permission to redistribute separately delivered runtime binaries;
target Apple silicon and macOS 15 or newer, while preserving native Metal as production until the runtime gate passes;
before implementation resumes, re-run the #84 closure, clean-machine packaging, code-signing, and same-workload parity gates documented in docs/MOJO_SPIKE_84.md;
CUDA remains a separate Linux/NVIDIA toolchain, runtime-closure, redistribution, and hardware-validation decision.
If the runtime grant or open host API changes, update #84 evidence first so this milestone has one source of truth.
## Goal
Complete Qwen3.8-27B on Apple Metal by adding MTP and image or video inference as separately measurable extensions after #88.
CUDA is deferred and does not block this issue.
## Scope and order
Implement and accept these steps sequentially:
1. Qwen-specific MTP heads and shared verify scheduling.
2. Image preprocessing and visual tokenization in Rust.
3. Vision encoder and projector in Mojo.
4. Combined visual-token and text prefill through the existing text model.
5. Video only if it belongs to the DS4Server product scope when implemented.
Keep model weights and heads under models/qwen3_8. Move a vision operation into shared ops only after a second real consumer needs identical semantics. Do not add speculative cross-platform or universal vision abstractions.
## Correctness gates
- MTP candidate generation, verification, cache updates, rollback, accepted tokens, and output match the pinned Qwen Metal reference.
- Image preprocessing, visual-token order, projection, context accounting, checkpoints, and multimodal output match within operation-specific tolerances.
- Text-only requests use the existing text path and retain the same tokens and state when MTP or visual input is absent.
- Do not describe Qwen as fully multimodal until vision passes end-to-end.
## Performance gates
Measure baseline decode, MTP draft and verify work, accepted-token throughput, memory, image preprocessing, vision encoder, projector, combined prefill, transfers, and cold start separately on the same Mac. Mojo-Metal must meet the native Metal reference for each enabled feature. A faster text backend cannot hide a slower MTP or vision stage, and absent visual input may not add text-path work.
## Acceptance criteria
- Qwen MTP passes Metal correctness and performance parity.
- Representative image and image-text prompts pass Metal correctness, memory, and prefill parity.
- Video is either covered by equivalent Metal gates or explicitly unsupported.
- Coverage, fixture provenance, raw results, binary closure, and licenses are updated.
- No native fallback or mixed backend execution appears in a claimed Mojo result.
- Data contracts remain suitable for the later optional CUDA issue without adding CUDA work here.
## Binding outcome from #84 (2026-08-31)
#84 stopped product integration on Mojo 1.0.0 because the open compiler/standard-library environment has no host-side Metal `DeviceContext`. The working `max.gpu.host` path requires MAX plus AsyncRT/KGEN dylibs whose component-specific redistribution permission could not be established. `mojo build --emit shared-lib` also links `@rpath/libKGENCompilerRTShared.dylib` for a trivial export.
MTP, multimodal, and Gated DeltaNet design work must retain the same backend boundary. Do not introduce MAX to obtain missing host-side GPU functionality.
These constraints are acceptance criteria:
- keep Mojo pinned to 1.0.0 through the committed `uv.lock`; do not use nightly or an older release;
- do not add `max`, `max-core`, `max-mojo-libs`, AsyncRT, or KGEN to the product/build/package closure without an authoritative redistribution grant naming every shipped component;
- do not treat Apache-licensed MAX Mojo source as permission to redistribute separately delivered runtime binaries;
- target Apple silicon and macOS 15 or newer, while preserving native Metal as production until the runtime gate passes;
- before implementation resumes, re-run the #84 closure, clean-machine packaging, code-signing, and same-workload parity gates documented in `docs/MOJO_SPIKE_84.md`;
- CUDA remains a separate Linux/NVIDIA toolchain, runtime-closure, redistribution, and hardware-validation decision.
If the runtime grant or open host API changes, update #84 evidence first so this milestone has one source of truth.
hugo
added the idea label 2026-08-31 12:13:28 +00:00
hugo
added this to the Mojo GPU Backend Transition milestone 2026-08-31 12:14:26 +00:00
hugo
changed title from Add Qwen3.8 MTP and multimodal inference without regressing the text path to Add Qwen3.8 MTP and multimodal inference on Metal without regressing text2026-08-31 12:29:54 +00:00
Project decision: the Mojo backend effort is canceled. All committed spike tooling, dependencies, lockfiles, and documentation were removed in 9a33c61; generated probe state was deleted as well. The independent Apple-silicon/macOS-15 product floor remains. This issue is no longer actionable.
Project decision: the Mojo backend effort is canceled. All committed spike tooling, dependencies, lockfiles, and documentation were removed in `9a33c61`; generated probe state was deleted as well. The independent Apple-silicon/macOS-15 product floor remains. This issue is no longer actionable.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Goal
Complete Qwen3.8-27B on Apple Metal by adding MTP and image or video inference as separately measurable extensions after #88.
CUDA is deferred and does not block this issue.
Scope and order
Implement and accept these steps sequentially:
Keep model weights and heads under models/qwen3_8. Move a vision operation into shared ops only after a second real consumer needs identical semantics. Do not add speculative cross-platform or universal vision abstractions.
Correctness gates
Performance gates
Measure baseline decode, MTP draft and verify work, accepted-token throughput, memory, image preprocessing, vision encoder, projector, combined prefill, transfers, and cold start separately on the same Mac. Mojo-Metal must meet the native Metal reference for each enabled feature. A faster text backend cannot hide a slower MTP or vision stage, and absent visual input may not add text-path work.
Acceptance criteria
Binding outcome from #84 (2026-08-31)
#84 stopped product integration on Mojo 1.0.0 because the open compiler/standard-library environment has no host-side Metal
DeviceContext. The workingmax.gpu.hostpath requires MAX plus AsyncRT/KGEN dylibs whose component-specific redistribution permission could not be established.mojo build --emit shared-libalso links@rpath/libKGENCompilerRTShared.dylibfor a trivial export.MTP, multimodal, and Gated DeltaNet design work must retain the same backend boundary. Do not introduce MAX to obtain missing host-side GPU functionality.
These constraints are acceptance criteria:
uv.lock; do not use nightly or an older release;max,max-core,max-mojo-libs, AsyncRT, or KGEN to the product/build/package closure without an authoritative redistribution grant naming every shipped component;docs/MOJO_SPIKE_84.md;If the runtime grant or open host API changes, update #84 evidence first so this milestone has one source of truth.
Add Qwen3.8 MTP and multimodal inference without regressing the text pathto Add Qwen3.8 MTP and multimodal inference on Metal without regressing textBased on findings in #84, all work on MOJO migration is stopped. Licene-hell.
Project decision: the Mojo backend effort is canceled. All committed spike tooling, dependencies, lockfiles, and documentation were removed in
9a33c61; generated probe state was deleted as well. The independent Apple-silicon/macOS-15 product floor remains. This issue is no longer actionable.