Add GLM 5.3 Flash support
This commit is contained in:
@@ -1,5 +1,69 @@
|
||||
# Apple Metal parity for issue #78
|
||||
|
||||
## GLM 5.3 Flash parity target
|
||||
|
||||
- Previous DS4 oracle: `8db89fe083ae4d17c9a2428ccd29803d3ae8f577`
|
||||
- Current DS4 oracle: `ec7642cdd9ec81d01ad4b1fd8f8a3d1511533748`
|
||||
- Main artifact: `GLM-5.3-Flash-Q2.gguf`, SHA-256
|
||||
`e81fd6241c6e55a64e1e14e47a3eab61a173fa8d7e4b5c1d1848827119705b32`
|
||||
- Vision artifact: `GLM-5.3-Flash-Vision-Encoder.gguf`, SHA-256
|
||||
`ae23e14c6979e889051b2e4a39351abcdafb161e18e606fae4d8c40095a4bf3a`
|
||||
- Machine: MacBook Pro `Mac17,7`, Apple M5 Max, 128 GiB unified memory,
|
||||
resident Q2 weights
|
||||
|
||||
The applicable commits after the previous oracle are:
|
||||
|
||||
| DS4 commit(s) | Scope carried into DS4Server |
|
||||
| --- | --- |
|
||||
| `147109a` | GLM 5.3 shape, GGUF binding, KDA/DSA attention, hyperconnections, routed MoE, BF16 tensors, output head, integrated MTP, and Metal kernels |
|
||||
| `4771329`, `9f95d9f` | Vision encoder, image preprocessing, prompt overlays, and compact-prefill correctness |
|
||||
| `b0c31af` | Pooled indexer cache, bounded attention memory, and batched prefill |
|
||||
| `747eafc`, `1df8ff0` | Fixed-image encoder oracle, replay/logit checks, and multimodal quality gates |
|
||||
| `8adb221` | Directional attention and FFN steering at the DS4 residual points |
|
||||
| `639d4eb` | Continued-prefill cache and position behavior |
|
||||
| `6cf658a` | BF16 decode projection schedule and Metal kernel specialization |
|
||||
| `01f3316`, `81c8c39`, `5da3206` | Current Metal compact-cache ABI, Q2/full-checkpoint binding, and GLM streaming compatibility |
|
||||
|
||||
CUDA- and ROCm-only commits in the range are excluded. The Rust release gates
|
||||
compare three-run medians against this exact DS4 build, require identical
|
||||
greedy token hashes, and fail only when Rust is more than 2% slower than DS4;
|
||||
there is no upper bound when Rust is faster. Text measures an
|
||||
8192-token frontier plus steady decode, both with and without integrated MTP.
|
||||
Vision compares the fixed encoder embedding (cosine at least 0.995, mean
|
||||
absolute error at most 0.001, maximum error at most 0.06), encoder latency,
|
||||
multimodal prefill, steady decode, image-conditioned logits, and replay.
|
||||
|
||||
### GLM 5.3 Flash acceptance results
|
||||
|
||||
The text fixture is `ds4.c`, rendered as one no-system chat turn (`-sys ''` in
|
||||
DS4), truncated to exactly 8192 tokens, with a 32768-token allocation and 64
|
||||
greedy output tokens. Same-process three-run medians are used because both DS4
|
||||
and DS4Server exhibit a load-order/thermal artifact with this 96 GB model.
|
||||
|
||||
| Capability | Metric | DS4 | DS4Server | Result |
|
||||
| --- | --- | ---: | ---: | --- |
|
||||
| Text | Prefill tok/s | 288.91 | 295.21 | Rust 2.18% faster |
|
||||
| Text | Decode tok/s | 22.58 | 22.78 | Rust 0.89% faster |
|
||||
| Integrated MTP | Decode tok/s | 26.55 | 26.80 | Rust 0.93% faster |
|
||||
| Vision encoder | Fixed-image seconds | 2.6255 | 2.4354 | Rust 7.24% faster |
|
||||
| Vision prompt | Prefill tok/s | 185.81 | 201.51 | Rust 8.45% faster |
|
||||
| Vision prompt | Decode tok/s | 26.40 | 27.21 | Rust 3.09% faster |
|
||||
|
||||
Text and MTP produced the same token SHA-256,
|
||||
`0b61e613708d8367fc87f5a32fbfd4ba098b40ee920645b1e38381e867131108`.
|
||||
The 1094-token fixed-image prompt produced the same vision token SHA-256,
|
||||
`424fdadc10c0a2bd95b405a3985151e84aa844a209d2a2ca72cd01c39dfa53db`.
|
||||
The Rust vision embedding compared with the DS4 oracle at cosine
|
||||
`0.999958726`, mean absolute error `0.000036702`, and maximum error
|
||||
`0.006153620`.
|
||||
|
||||
The checkpoint advertises a 1048576-token maximum. On this 128 GiB machine,
|
||||
32768 is the default manageable context and 50000 is the extended option: the
|
||||
50000-token run used 3.51 GiB of live runtime tensors in addition to 90.61 GiB
|
||||
of mapped model wrappers and retained the exact output hash. Larger allocations
|
||||
leave too little operating-system and application headroom for a reliable
|
||||
default.
|
||||
|
||||
## Frozen comparison
|
||||
|
||||
- DS4 oracle: `8db89fe083ae4d17c9a2428ccd29803d3ae8f577`
|
||||
|
||||
Reference in New Issue
Block a user