hugo/DS4Server
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

Establish the Mojo 1.0 integration gate

Browse Source
This commit is contained in:
Georg Bauer
2026-08-31 15:25:04 +02:00
parent 2c2c3702b0
commit 0dcf181dbe
11 changed files with 287 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.cargo/config.toml Normal file
View File

@@ -0,0 +1,2 @@
[env]
MACOSX_DEPLOYMENT_TARGET = "15.0"

1
.gitignore vendored
View File

@@ -1,2 +1,3 @@
/target
/.venv
.DS_Store

2
Cargo.toml
View File

@@ -48,4 +48,4 @@ icons = ["assets/DS4Server.icns", "assets/app-icon.png"]
resources = ["metal", "assets/dev-brain"]
[package.metadata.packager.macos]
minimum-system-version = "13.0"
minimum-system-version = "15.0"

2
README.md
View File

@@ -96,7 +96,7 @@ intentionally separate from `cargo test`.
## Build and run
DS4Server targets macOS 13 or newer and requires Rust 1.97 plus
DS4Server targets Apple silicon with macOS 15 or newer and requires Rust 1.97 plus
`cargo-packager` 0.11.8.
```sh

153
docs/MOJO_SPIKE_84.md Normal file
View File

@@ -0,0 +1,153 @@
# Mojo 1.0 Metal integration spike (#84)
## Decision
**STOP** the product integration at the license/runtime gate. DS4Server can pin
and invoke the Mojo 1.0 compiler through `uv`, but Mojo 1.0's open standard
library does not provide the host-side `DeviceContext` needed to load and run a
Metal kernel. That API is delivered by MAX and links to separately shipped
AsyncRT and KGEN runtime libraries whose redistribution status is not explicit.
This is a mandatory stop under issue #84, not a performance conclusion. The
existing Objective-C Metal backend remains the production path. DS4Server now
targets Apple silicon and macOS 15 or newer, matching Mojo 1.0's supported host
floor. No MAX wheel, runtime library, virtual environment, or Modular source
checkout is bundled.
Run the committed proof from the repository root:
```sh
tools/mojo-spike/check.sh
```
The command syncs the locked `uv` environment, checks the exact compiler build,
emits a C-ABI object and dynamic library, records their closure under
`target/mojo-spike`, and verifies that the compiler-only environment cannot
import the GPU host API. Once the lock has been synced, the same proof runs
without network access as `UV_OFFLINE=1 tools/mojo-spike/check.sh`.
## Pinned environment
| Component | Pin / observation |
| --- | --- |
| Mojo compiler | `mojo-compiler==1.0.0` |
| Compiler identity | `Mojo 1.0.0 (ed45d567)` |
| Dependency manager | `uv.lock` revision 3; hashes recorded for every wheel |
| Mojo source tag | `mojo/v1.0.0`, commit `b4497b7ce9ba96331c72c637ad41b44bab374f33` |
| MAX probe | `max==26.5.0`, isolated outside the project environment |
| Host | Apple M5 Max, arm64, macOS 26.6.2 (25G83) |
| Xcode / Metal | Xcode 26.6 (17F113), Metal toolchain 17F109 |
Only `mojo-compiler` is a project build dependency. The broader `mojo` package
and MAX are intentionally absent because they add tools and runtimes that the
committed proof does not need or may not redistribute.
## Build and ABI findings
`@export` produces a callable C symbol and `mojo build --emit object` produces a
Mach-O object that can be linked by the platform linker. This is the viable ABI
boundary if the GPU runtime gate is resolved: opaque handles, fixed-width C
scalars, caller-owned buffers, integer status codes, and an explicit last-error
function. Rust unwinding and Mojo exceptions must never cross it.
`mojo build --emit shared-lib` is not self-contained. Even the trivial
`ds4_mojo_probe` export has this non-system dependency:
```text
@rpath/libKGENCompilerRTShared.dylib
```
The object form avoids that dependency for the trivial CPU export, but it does
not provide a way to construct a Metal device context or launch a GPU kernel.
Importing `DeviceContext` from the open `std.gpu.host` package fails because it
is not exported there. Importing the working API from `max.gpu.host` fails in
the compiler-only environment because MAX is not installed.
Compiling the Apache-licensed `max.gpu.host` Mojo sources directly did not make
the program self-contained. The link still required these external symbols:
```text
_AsyncRT_DeviceContext_create
_AsyncRT_DeviceContext_deviceName
_AsyncRT_DeviceContext_release
_AsyncRT_DeviceContext_strfree
```
An isolated `max==26.5.0` probe could enumerate the Apple M5 Max, proving that
the API is technically viable when the proprietary runtime is present. Its
executable closure included:
```text
@rpath/libKGENCompilerRTShared.dylib
@rpath/libAsyncRTMojoBindings.dylib
/usr/lib/libSystem.B.dylib
```
That probe is deliberately not part of the repository dependency graph.
## License and redistribution inventory
| Component | Source / package evidence | Product decision |
| --- | --- | --- |
| Mojo language, compiler sources, and standard library | Apache-2.0 with LLVM exceptions in the `modular/modular` repository | Approved for build use, subject to normal notices |
| `mojo-compiler` and `mojo-compiler-mojo-libs` wheels | Pinned by hash; wheel metadata identifies the MAX Platform Software License | Build-only; do not redistribute the wheels |
| MAX Mojo source modules | Repository source is Apache-2.0 with LLVM exceptions, while Modular separately states MAX usage and distribution remain under the Community License | Source license alone does not approve the runtime closure |
| `max`, `max-core`, and `max-mojo-libs` wheels | Wheel metadata: `LicenseRef-MAX-Platform-Software-License` | Not a product dependency; do not bundle |
| `libAsyncRTMojoBindings.dylib` and related AsyncRT libraries | Supplied only by the MAX wheel in this probe; no component-specific redistribution grant found | Blocked |
| `libKGENCompilerRTShared.dylib` | Required by compiler-emitted shared libraries and the MAX GPU executable; no component-specific redistribution grant found | Blocked |
| Apple Metal frameworks and system libraries | Supplied by macOS | Approved system dependency |
| Existing DS4Server Objective-C Metal backend | Existing project code and Apple system frameworks | Remains the production implementation |
The governing sources are the [Mojo repository license](https://github.com/modular/modular/blob/mojo/v1.0.0/LICENSE),
the [Modular repository notice separating MAX licensing](https://github.com/modular/modular#license),
and the [MAX and Mojo Community License](https://www.modular.com/legal/max-mojo-license).
The Community License permits redistribution only for components Modular
expressly identifies as redistributable and subject to its conditions. No
authoritative component list was found that names the two required dylibs.
Silence is not permission, so packaging is blocked.
## Platform and packaging result
Mojo 1.0 officially requires Apple silicon, macOS 15 or newer, Xcode or Command
Line Tools 16 or newer, and the Metal toolchain. DS4Server now declares that
same architecture and OS floor in its build environment, package metadata, and
README. The native backend remains necessary because the license/runtime gate,
not the platform floor, blocks the Mojo backend.
The required package proof cannot pass today:
- a clean machine would need non-system MAX runtime dylibs;
- their exact redistributable status and required notices are unresolved;
- an ad-hoc signed `.app` cannot be accepted while those dylibs are absent or
copied without permission;
- CUDA viability cannot be inferred from this Metal probe and remains a
separate Linux/NVIDIA build, runtime, and license audit.
## Performance result
No parity number is reported. The approved compiler-only environment cannot
launch a Metal operation, so there is no license-safe Mojo candidate to compare
with native Metal. Timing a MAX-backed or CPU-only substitute would answer a
different question and create a misleading baseline. The parity harness and raw
GPU/wall-clock measurements begin only after an approved host/runtime path can
build, link, and package.
## Conditions to resume
All of the following are required before issue #85 or later implementation may
replace native code:
1. Modular publishes an authoritative redistribution grant naming every
required runtime library, or Mojo exposes a host-side Metal launch path whose
complete closure is under an approved open-source/system license.
2. The locked environment still builds through `uv` and works offline after
dependency installation.
3. `otool -L` and `nm -u` show only system libraries and explicitly approved,
packaged runtime components.
4. A clean-machine ad-hoc signed app launch proves install-name, rpath, code-sign,
and notice handling.
5. The parity harness records identical workloads, correctness, GPU execution
time, wall time, and repeated samples against the native Metal oracle.
Until then, later milestone issues are design and inventory work only; they may
not introduce MAX as a Cargo, `uv`, build-script, or application dependency.

13
native/metal/ds4_metal.m
View File

@@ -13111,12 +13111,7 @@ static int ds4_gpu_stream_full_expert_addr_table_requested(void) {
static uint64_t ds4_gpu_buffer_address(id<MTLBuffer> buffer, NSUInteger inner) {
if (!buffer) return 0;
#if TARGET_OS_OSX
if (@available(macOS 13.0, *)) {
return (uint64_t)[buffer gpuAddress] + (uint64_t)inner;
}
#endif
return 0;
}
static int ds4_gpu_stream_compact_addr_ensure_buffers(uint32_t layer) {
@@ -17286,8 +17281,6 @@ static DS4MetalQ4ExpertTable *ds4_gpu_q4_expert_address_table(
n_total_expert];
if (!expert_buffers) return nil;
#if TARGET_OS_OSX
if (@available(macOS 13.0, *)) {
for (uint32_t first = 0; first < n_total_expert; first += table_group_size) {
const uint32_t remaining = n_total_expert - first;
const uint32_t group_n =
@@ -17325,12 +17318,6 @@ static DS4MetalQ4ExpertTable *ds4_gpu_q4_expert_address_table(
}
[expert_buffers addObject:group_buf];
}
} else
#endif
{
fprintf(stderr, "ds4: Metal GPU addresses require macOS 13 or newer\n");
return nil;
}
[address_buffer didModifyRange:NSMakeRange(0,
(NSUInteger)n_total_expert * sizeof(uint64_t))];

8
pyproject.toml Normal file
View File

@@ -0,0 +1,8 @@
[project]
name = "ds4server-mojo"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = ["mojo-compiler==1.0.0"]
[tool.uv]
package = false

3
tools/mojo-spike/abi.mojo Normal file
View File

@@ -0,0 +1,3 @@
@export("ds4_mojo_probe")
def ds4_mojo_probe(value: Int32) abi("C") -> Int32:
return value

41
tools/mojo-spike/check.sh Executable file
View File

@@ -0,0 +1,41 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/../.."
output_dir=target/mojo-spike
mkdir -p "$output_dir"
uv sync --frozen
version=$(uv run --frozen --no-sync mojo --version)
test "$version" = "Mojo 1.0.0 (ed45d567)"
uv run --frozen --no-sync mojo build --emit object \
-o "$output_dir/abi.o" tools/mojo-spike/abi.mojo
uv run --frozen --no-sync mojo build --emit shared-lib \
-o "$output_dir/libds4_mojo_probe.dylib" tools/mojo-spike/abi.mojo
otool -L "$output_dir/libds4_mojo_probe.dylib" \
> "$output_dir/abi-closure.txt"
nm -u "$output_dir/libds4_mojo_probe.dylib" \
> "$output_dir/abi-undefined.txt"
grep -F '@rpath/libKGENCompilerRTShared.dylib' \
"$output_dir/abi-closure.txt" >/dev/null
if uv run --frozen --no-sync mojo build \
-o "$output_dir/gpu-host" tools/mojo-spike/gpu_host.mojo \
> "$output_dir/gpu-host-build.txt" 2>&1; then
echo "unexpectedly found a GPU host runtime in the compiler-only environment" >&2
exit 1
fi
grep -F "unable to locate module 'max'" \
"$output_dir/gpu-host-build.txt" >/dev/null
printf '%s\n' \
"Mojo 1.0 compiler and C ABI: verified" \
"Open Metal GPU host path: unavailable" \
"Shared-library runtime closure: @rpath/libKGENCompilerRTShared.dylib" \
"Decision: STOP until the runtime closure is explicitly redistributable"

6
tools/mojo-spike/gpu_host.mojo Normal file
View File

@@ -0,0 +1,6 @@
from max.gpu.host import DeviceContext
def main() raises:
var context = DeviceContext()
print(context.name())

35
uv.lock generated Normal file
View File

@@ -0,0 +1,35 @@
version = 1
revision = 3
requires-python = ">=3.10"
[[package]]
name = "ds4server-mojo"
version = "0.1.0"
source = { virtual = "." }
dependencies = [
{ name = "mojo-compiler" },
]
[package.metadata]
requires-dist = [{ name = "mojo-compiler", specifier = "==1.0.0" }]
[[package]]
name = "mojo-compiler"
version = "1.0.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "mojo-compiler-mojo-libs" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/60/59/c11cf3129d4bc77b1604ac3a84bee51ee1a7cd8a3a8fa25c14528f4bd51e/mojo_compiler-1.0.0-py3-none-macosx_13_0_arm64.whl", hash = "sha256:5077ee27c0dfbbdf138a11ec12838b0d3f10148a874cf95040c9ca2332ae2622", size = 70979164, upload-time = "2026-08-11T14:34:51.281Z" },
{ url = "https://files.pythonhosted.org/packages/a5/f9/cbfe2bf947d0926ad57599513d898f1e02ee0c60af1253b779ecaa810235/mojo_compiler-1.0.0-py3-none-manylinux_2_34_aarch64.whl", hash = "sha256:3ae3eb0c58a8956f542e324f736866df0ae2f9f875e8a279cbeea961ec4e9ee8", size = 76295262, upload-time = "2026-08-11T14:34:22.131Z" },
{ url = "https://files.pythonhosted.org/packages/f0/5f/f38fefe327d1c81e28def69c4a52ae4f75e389cb6e613a2c04ca8d68d582/mojo_compiler-1.0.0-py3-none-manylinux_2_34_x86_64.whl", hash = "sha256:e9e60f9638e69ca0f4be7292468523fc98a143f58dbf9024f60ed68b874a867e", size = 79142588, upload-time = "2026-08-11T14:34:05.301Z" },
]
[[package]]
name = "mojo-compiler-mojo-libs"
version = "1.0.0"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/54/99/ea401ff1db56a4af8607283b95627e01b986fc67f510715b07f100118105/mojo_compiler_mojo_libs-1.0.0-py3-none-any.whl", hash = "sha256:20a92e37ecbd19e2dbb1a525612a8de4c9f266f5536def55c5e2b0786320e8b3", size = 3187048, upload-time = "2026-08-11T14:33:53.171Z" },
]