# Apple Metal parity for issue #78

## GLM 5.3 Flash parity target

- Previous DS4 oracle: `8db89fe083ae4d17c9a2428ccd29803d3ae8f577`
- Current DS4 oracle: `ec7642cdd9ec81d01ad4b1fd8f8a3d1511533748`
- Main artifact: `GLM-5.3-Flash-Q2.gguf`, SHA-256
  `e81fd6241c6e55a64e1e14e47a3eab61a173fa8d7e4b5c1d1848827119705b32`
- Vision artifact: `GLM-5.3-Flash-Vision-Encoder.gguf`, SHA-256
  `ae23e14c6979e889051b2e4a39351abcdafb161e18e606fae4d8c40095a4bf3a`
- Machine: MacBook Pro `Mac17,7`, Apple M5 Max, 128 GiB unified memory,
  resident Q2 weights

The applicable commits after the previous oracle are:

| DS4 commit(s) | Scope carried into DS4Server |
| --- | --- |
| `147109a` | GLM 5.3 shape, GGUF binding, KDA/DSA attention, hyperconnections, routed MoE, BF16 tensors, output head, integrated MTP, and Metal kernels |
| `4771329`, `9f95d9f` | Vision encoder, image preprocessing, prompt overlays, and compact-prefill correctness |
| `b0c31af` | Pooled indexer cache, bounded attention memory, and batched prefill |
| `747eafc`, `1df8ff0` | Fixed-image encoder oracle, replay/logit checks, and multimodal quality gates |
| `8adb221` | Directional attention and FFN steering at the DS4 residual points |
| `639d4eb` | Continued-prefill cache and position behavior |
| `6cf658a` | BF16 decode projection schedule and Metal kernel specialization |
| `01f3316`, `81c8c39`, `5da3206` | Current Metal compact-cache ABI, Q2/full-checkpoint binding, and GLM streaming compatibility |

CUDA- and ROCm-only commits in the range are excluded. The Rust release gates
compare three-run medians against this exact DS4 build, require identical
greedy token hashes, and fail only when Rust is more than 2% slower than DS4;
there is no upper bound when Rust is faster. Text measures an
8192-token frontier plus steady decode, both with and without integrated MTP.
Vision compares the fixed encoder embedding (cosine at least 0.995, mean
absolute error at most 0.001, maximum error at most 0.06), encoder latency,
multimodal prefill, steady decode, image-conditioned logits, and replay.

### GLM 5.3 Flash acceptance results

The text fixture is `ds4.c`, rendered as one no-system chat turn (`-sys ''` in
DS4), truncated to exactly 8192 tokens, with a 32768-token allocation and 64
greedy output tokens. Same-process three-run medians are used because both DS4
and DS4Server exhibit a load-order/thermal artifact with this 96 GB model.

| Capability | Metric | DS4 | DS4Server | Result |
| --- | --- | ---: | ---: | --- |
| Text | Prefill tok/s | 288.91 | 295.21 | Rust 2.18% faster |
| Text | Decode tok/s | 22.58 | 22.78 | Rust 0.89% faster |
| Integrated MTP | Decode tok/s | 26.55 | 26.80 | Rust 0.93% faster |
| Vision encoder | Fixed-image seconds | 2.6255 | 2.4354 | Rust 7.24% faster |
| Vision prompt | Prefill tok/s | 185.81 | 201.51 | Rust 8.45% faster |
| Vision prompt | Decode tok/s | 26.40 | 27.21 | Rust 3.09% faster |

Text and MTP produced the same token SHA-256,
`0b61e613708d8367fc87f5a32fbfd4ba098b40ee920645b1e38381e867131108`.
The 1094-token fixed-image prompt produced the same vision token SHA-256,
`424fdadc10c0a2bd95b405a3985151e84aa844a209d2a2ca72cd01c39dfa53db`.
The Rust vision embedding compared with the DS4 oracle at cosine
`0.999958726`, mean absolute error `0.000036702`, and maximum error
`0.006153620`.

The checkpoint advertises a 1048576-token maximum. On this 128 GiB machine,
32768 is the default manageable context and 50000 is the extended option: the
50000-token run used 3.51 GiB of live runtime tensors in addition to 90.61 GiB
of mapped model wrappers and retained the exact output hash. Larger allocations
leave too little operating-system and application headroom for a reliable
default.

## Frozen comparison

- DS4 oracle: `8db89fe083ae4d17c9a2428ccd29803d3ae8f577`
- DS4Server pre-issue baseline: `79468c65b68c28e1539acf0a2a14bd42e000e365`
- DeepSeek V4 Flash 0731 IQ2XXS/w2Q2K model SHA-256:
  `ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0`
- DSpark support-model SHA-256:
  `7e319924541db3f7a163ed7e11d7532a70d48228ab59d36cb81e1d4511885360`
- Machine: MacBook Pro `Mac17,7`, Apple M5 Max, 128 GiB unified memory,
  macOS 26.6.2, power 100%
- Context and prefill allocation: 4096 tokens; quality, steering, speculative
  decoding, and DSpark disabled for the throughput gate
- SSD mode: 4096 cached experts, 4096 popularity-preloaded experts, ordinary
  (not cold) streaming
- Prompt: `Count from one to two hundred, spelling out every number.` rendered
  to 16 tokens; 128 greedy non-EOS tokens are measured

The longer prompt is deliberate: all 128 generated tokens are reachable before
EOS and are identical between DS4 and DS4Server in each mode. A short-response
fixture reached EOS early and made a forced post-EOS speed run compare
unreachable, mode-dependent states.

## DS4 performance-change matrix

The audited range is
`efdadd41e20134af4f3381e1ed90e96fe4faef6f..8db89fe083ae4d17c9a2428ccd29803d3ae8f577`.
Documentation-only, parser/server-only, CUDA, ROCm, tensor-parallel, model
download, and merge commits do not affect the Rust-owned single-session Metal
graph. The applicable performance/runtime commits are accounted for below.

| DS4 commit(s) | DS4 call site or behavior | DS4Server result |
| --- | --- | --- |
| `3ce6777`, `8e0f997` | Native streamed-expert cache pinning and safe fallback | Active in the byte-matched native Metal cache; Rust preserves the cache budget and fallback. |
| `df04887` | `ds4_gpu_attention_indexed_mixed_batch_heads_tensor` | Active through `encode_layer_with_cache_rows`; the long-context fixture crosses the indexed boundary. |
| `74e8f11`, `4591cb1`, `a51e6ec`, `f284165`, `af80694`, `7fb2830`, `0e89a0e`, `769a8ba`, `8d683d6` | DSpark block commit, deterministic scheduling, acceptance replay/direct commit, greedy identity, and exact/opportunistic sampling | Active in `Dspark::propose` and the verifier commit paths; greedy and exact stochastic fixtures cover the frozen support model. |
| `96c3ba4`, `d69a017`, `81f8534`, `ac5cb2c`, `48392e3`, `222b2cb`, `532ec8b` | Metal 4 Q4 projection, compact routed-MoE, aligned Q8, indexed-attention, and routed prefill kernels | Active through the existing `encode_batch_layer` Q4/indexed/routed entry points; no second prefill graph was added. |
| `24903f3` | `ds4_gpu_head_rms_norm_rope_tail_tensor` | Existing Rust Q-head norm/RoPE fusion is retained and used. |
| `4893e0c`, `d14ce35` | Mapped-prefill expert seeding and consistent expert/non-expert maps | Rust seeds with `ds4_gpu_stream_expert_cache_seed_experts_gpu_copy`, falls back to CPU/pread, and restores one cached static decode map with the same non-routed spans. |
| `0dd0d36` | Release speed regression gate | Rust has an ignored release gate with retained raw CSV, DS4 reference threshold, baseline threshold, p50/p95, cache metrics, and token hashes. |
| `4e401a1`, `0e125e6` | Automatic pre-M5/M5 admission for exact decode fusions | Rust binds both device predicates and mirrors per-generation rollback variables without a UI preference. |
| `71d8c2a` | Q/KV pair plus ratio-4/128 compressor projection/store | `try_qkv_pair_compressor_fusion` calls the native combined entry point only for DS4's resident full-decode shapes. |
| `b64d3e0` | Emit-path compressor finalizer | `update_compression_pair` defers both updates only when the resident finalizer is eligible, then requires exact success. |
| `3826f52`, `ad4d05c` | Exact compressor softmax/product and ratio-4 pool | Native kernels are active; Rust's compressor ABI now passes the upstream `decode_one_token` and `defer_finalize` flags exactly. |
| `92d83e6` | HC norm/mix producer clustering | `decode_hc_mix` uses split-norm producer fusion, with the established norm/mix/split sequence as fallback. |
| `1be41c8` | Packed exact split-K attention reduction | Active in the vendored native attention kernel reached by Rust's decode attention calls. |
| `0e8fb54`, `e6a6ab5` | Router/select, shared gate/up, QKV norm/KV RoPE/store, parallel full FFN, inverse-attention RoPE, and pre-M5 ports | All applicable entry points are called with DS4 shape, tensor, quality, steering, profile, SSD, and device guards. Parallel FFN aborts on partial failure; inverse RoPE falls back when the attention kernel does not consume it. |
| `0658dac` | Long-context prefill/decode correctness | The 4096-crossing ignored fixture covers compressed/indexed state and the next decode token. |
| `023614e`, `0ad494e`, `42033ee` | Remove stream-512 experiment; make exact paths automatic; verify DFlash pipeline | No stream-512 or opt-in experiment remains. Feature rollbacks retain identical token hashes. |
| `24fa85e` | Keep successfully loaded disk KV checkpoints | Existing Rust checkpoint load/round-trip behavior is verified; only invalid checkpoints follow deletion policy. |
| `7694112` | Reuse a shorter exact GLM live prefix | `live_prefix_rewind_target` rewinds to `prompt_len - 1`, clears pending speculation, and reevaluates the last prompt token. |
| `5186e2c` | Keep Metal decode changes portable | Device checks and unavailable returns preserve non-M5 and unfused execution. |

The Metal MXFP4 commits (`725b084`, `1f862bd`, `7bec128`, `bcd1e57`,
`f75a065`, and their merge/format work) are intentionally excluded: MXFP4 is
issue #79. `ds4_gpu_set_decode_pipeline_fast_lookup` is reset to disabled,
matching DS4 because that cache is enabled only for the pre-M5 MXFP4 path.

## Results

Three implementations were run sequentially, three times per mode, with a
five-second cooldown. Medians are from
[`metal-parity-issue-78.csv`](metal-parity-issue-78.csv).

| Implementation | Mode | Prefill tok/s | TTFT ms | First decode ms | Stable tok/s | p50 ms | p95 ms |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| DS4Server baseline | resident | 69.63 | 256.07 | 26.37 | 39.53 | 24.88 | 26.61 |
| DS4 | resident | 56.49 | 305.94 | 22.71 | 45.60 | 21.91 | 22.32 |
| DS4Server issue #78 | resident | 69.76 | 252.01 | 22.66 | 45.60 | 21.94 | 22.33 |
| DS4Server baseline | SSD | 6.81 | 2476.94 | 125.30 | 8.63 | 114.93 | 124.28 |
| DS4 | SSD | 11.88 | 1393.54 | 46.75 | 26.11 | 35.62 | 46.86 |
| DS4Server issue #78 | SSD | 11.49 | 1440.79 | 48.84 | 25.07 | 37.70 | 48.85 |

TTFT is the per-run prompt prefill duration plus first decode latency; its
median is reported separately from the first decode kernel latency.

Resident stable decode improves 15.4% over the baseline and matches DS4.
SSD stable decode improves 190.3% over the baseline and is 4.0% below
DS4, inside the required 5% boundary. SSD cache accounting is identical to
DS4 for the measured token sequence: 4096 entries, 32,958 hits, 4,194 misses,
and 29,684,662,272 pread bytes. The resident and SSD token SHA-256 values are,
respectively, `ed5461476a82824c3e0440e943a0b2f6b8a8f19dc26a550a1bc255003851ab8d`
and `2bc33a04bb88f9f3c56f8bae0124b7bf25fd2f32aee81a1bdaf6e979448da052`;
DS4 emitted the same 128-token sequences in the corresponding modes.

Representative `/usr/bin/time -l` measurements report:

| Implementation | Mode | Maximum RSS bytes | Peak footprint bytes | Process swaps |
| --- | --- | ---: | ---: | ---: |
| DS4 | resident | 78,790,656 | 4,746,511,440 | 0 |
| DS4Server issue #78 | resident | 129,024,000 | 104,071,816 | 0 |
| DS4 | SSD | 29,066,280,960 | 29,595,620,664 | 0 |
| DS4Server issue #78 | SSD | 29,098,393,600 | 104,350,344 | 0 |

The post-run system memory-pressure reading was 92% free. Every measured
process reported zero swaps; the post-run host counters were 0 swap-ins and 20
historical swap-outs.

## Reproduction

DS4 uses the exact 16 Rust-rendered prompt token IDs with its speed-benchmark
format. DS4Server uses the installed-model fixture:

```sh
DS4_REFERENCE_TPS=45.60 \
DS4SERVER_BASELINE_TPS=39.526849 \
cargo test --release --all-features flash_0731_m5_decode_performance_gate \
  -- --ignored --nocapture --test-threads=1

DS4SERVER_BENCH_SSD=1 \
DS4_REFERENCE_TPS=26.11 \
DS4SERVER_BASELINE_TPS=8.634659 \
cargo test --release --all-features flash_0731_m5_decode_performance_gate \
  -- --ignored --nocapture --test-threads=1
```

The gate fails unless throughput improves over the frozen DS4Server baseline
and remains at least 95% of DS4. `DS4SERVER_BENCH_TOKENS=1` prints the prompt
and generated sequences for direct DS4 comparison.