DS4Server/docs/glm-scheduling-followup-20260911.md

# GLM execution and responsiveness follow-up — 2026-09-11

## User acceptance and scope

- Qwen is confirmed good in normal interactive use.
- GLM decode is now also confirmed good interactively. GLM prefill remains
  usable, but feels less smooth than Qwen and affects other applications.
  This is not a claim of a complete freeze or a new confirmed beachball.
- The user authorized resuming the outstanding work and explicitly authorized
  compiling/running antirez/ds4 as a standalone, supervised reference benchmark.
  No DS4 C objects are linked into DS4Server or its application bundle.
- Qwen's golden master remains MTPLX; DeepSeek/GLM remain antirez/ds4.
  The accepted 2.6% Qwen Summary AR exception is not a general tolerance.

## Implemented execution changes

Both scalar GLM loops now flush periodically every four completed layers,
excluding the final layer and SSD expert streaming. Previously they flushed
only once at layer four. This follows the active indexed DS4 graph, including
scalar MTP fallback/rejection replay. The reference's dynamic per-layer mapping
fallback must not be confused with Rust's static non-expert decode mapping:
`glm_streaming_model_spans` retains non-expert tensors, the configured resident
expert prefix, and incompatible expert layouts; selected experts are loaded
through the existing native cache. No new per-layer SSD waits were introduced.
Low-memory dynamic mapping fallback parity is not established by this patch.

GLM 5.3 prefill progress now advances at existing completed GPU drains and after
the final output evaluation, not after every submitted layer. Chunk selection,
prefill flush/drain placement, Metal kernels, sampling and power policy are
unchanged. This corrects progress accounting; it does not by itself fix the
remaining prefill smoothness issue.

Targeted checks passed: periodic/final/SSD decode boundary test; existing
prefill boundary test; live two-row verifier acceptance, rejection, rewind,
scalar fallback, recurrent state, unused HC workspace guards and lifetime
counters. The live test additionally verifies completed-prefill progress points.

## Standalone reference and instrumentation

`tools/ds4-session-reference.rs` is a separate Rust benchmark driver for the
unchanged public DS4 engine/session interface. It is deliberately not a Cargo
target and is never included in the app. Its build script verifies reference
commit `ec7642cdd9ec81d01ad4b1fd8f8a3d1511533748`, the pinned header hash,
unchanged tracked engine sources and current reference objects. The arm64 ABI
layout is checked against Clang's record layout (engine options 280 bytes,
distributed offset 152, TP offset 216).

The public DS4 CLI/API maps `low` to `high`. The driver instead constructs the
Low system prefix through the public chat API and passes tokens to the original
session implementation. It also reproduces the UI's separate system-prefix
prefill (9 tokens for GLM), and retains generated token history in one session.
`glm53_reference_prompt_tokens_match_shared_runtime` checks every token of all
four prompt streams, including continued turns, against the production Rust
tokenizer. The first complete matched-bootstrap AR reference passed this check.

Build from the DS4Server checkout, with already-built reference objects:

```sh
bash tools/build-ds4-session-reference.sh /Users/gb/Projects/ds4 /absolute/path/reference
```

Run the resulting binary from the reference checkout under `test-supervisor`:

```text
test-supervisor 114688 30 45 --command /absolute/path/reference MODEL_GGUF glm on README_PATH
```

The driver uses power100, Low, context32768, temperature0.6, top-p0.95,
top-k0, min-p0, seed42, SSD streaming off and graph-selected GLM chunks.
Warmup is a separate session (up to32 tokens), followed by README Summary,
lighthouse Story, and Python `is_prime` in one ongoing chat, to natural EOS.
There is no total-runtime watchdog. Missing models fail; no downloads occur.
DeepSeek is supported by the driver's `deepseek` family argument, with the
installed DSpark support GGUF required for acceleration-on; it has not yet been
validated by this follow-up's GLM runs.

Optional `DS4_REFERENCE_CANARY=/absolute/path/ds4-server` starts the **same native
probe and monitor implementation** as the UI/harness in a separate process.
The `gpu-canary` CLI accepts phase labels on stdin and ends on EOF. A readiness
handshake waits for the first successful probe before model loading. Its
readiness/stall/sample reports are not model-progress watchdog heartbeats.
Clean throughput runs leave this variable unset. An external observer and an
in-process observer must not be treated as identical OS scheduling conditions.

`DS4_REFERENCE_IN_PROCESS_CANARY=1` instead enables a native probe thread inside
the reference process. It links the same `native/metal/ds4_canary.m` used by
DS4Server, with the same4096-byte blit, separate queue and100ms cadence, rather
than duplicating a Metal implementation. This bridge contains no model code.
Its Rust monitor logs per-sample phases/timing but is not the UI event loop.
The optional external observer can also be enabled simultaneously. Neither is
enabled for clean reference throughput. `reference --canary-self-test` exercises
readiness, two phases and clean shutdown without loading a model.

The initial external-probe integration test exposed a startup race: a phase
could end before the executable initialized. It was not worked around by
loosening the assertion; the adapter now requires a readiness handshake. The
initial diagnostic without that handshake is retained, not a full-startup proof.

## Measurements and limitations

All raw receipts, outputs and failed attempts are retained under
`local-eval-results/glm-scheduling-20260911.7VSaeW/` (ignored, local evidence).
The before binary is the secured `02db096` implementation, SHA256
`ab289f067a7a01c22113eec76aa896638d83e392242192e1440d14ed11524d5c`.

Fresh complete AR runs (after, then before), same output/reasoning/tokens/EOS:

| Turn | Tokens | Before decode t/s | After decode t/s | Before prefill ms | After prefill ms |
| --- | ---: | ---: | ---: | ---: | ---: |
| Summary |640|24.467|25.403|7561|6034|
| Story |1102|23.703|24.135|262|255|
| Python |200|24.726|25.401|305|302|

These are single sequential pairs, not drift-controlled medians. The large
summary-prefill difference cannot be attributed to a decode-only flush change.
No hard decode regression was observed; full performance parity is not proven.

The matched-bootstrap original DS4 AR reference completed naturally at
26.618/25.123/25.825 decode t/s, with779/1047/198 output tokens. Its generated
text differs from Rust despite matching initial prompt tokens/settings; later
contexts therefore also differ. This is not an exact-output performance pair.
Reference prefill timers measure session sync; Rust's current GLM `prefill_ms`
still includes the observed UI phase. Do not silently equate those intervals.

The new Rust MTP run retains the previous625/1026/196 completion tokens and
identical output/reasoning. Its draft acceptance fractions are241/385 (62.6%),
342/685 (49.9%) and95/102 (93.1%). Python therefore has the expected higher
acceptance; MTP's benefit is workload-dependent, not uniformly absent.

The fresh MTP before/after pair also preserves every output/reasoning token
and natural completion:

| Turn | Tokens | Before decode t/s | After decode t/s | Before prefill ms | After prefill ms |
| --- | ---: | ---: | ---: | ---: | ---: |
| Summary |625|19.988|23.154|9303|6190|
| Story |1026|17.728|19.295|306|272|
| Python |196|27.843|29.506|347|322|

The same sequential-run/drift limitation applies. This establishes no observed
hard regression, not a controlled causal speedup or reference-parity acceptance.

### Canary placement and timestamp attribution

The full `after-mtp-dual-canary` run had simultaneous internal/external probes.
The internal probe recorded867 successful samples, with a prefill maximum
of490.740ms and decode maximum3.180ms. The external probe was ready before model
launch and continued until after termination:3004 successful samples, overall
maximum3.331ms (startup), and1.572ms while labelled `preparing` across the model
lifetime. That external label is deliberately not turn/phase attribution.
Both probes stopped cleanly; no sample failed or reached2s. This is diagnostic
evidence, not a clean throughput run or a compositor-frame test.

Thus the previous `completed_ms` cannot be interpreted as a measured systemwide
GPU blockade. It includes host-side waiting and completion delivery. The optional
shared native probe now also records commit-to-GPU-start (`gpu_wait_ms`),
GPU-start-to-end (`gpu_interval_ms`), and GPU-end-to-host-return (`host_return_ms`).
Metal's GPU timestamps use system mach time; the probe uses `mach_absolute_time`
and the native timebase for those differences, not `CLOCK_MONOTONIC`. Missing or
inconsistent timestamps remain null, not zero. The GPU interval includes possible
GPU scheduling/preemption, not exclusively active blit execution. See Apple's
[GPUStartTime documentation](https://developer.apple.com/documentation/metal/mtlcommandbuffer/gpustarttime).

The model-free Metal integration check verifies phase coverage, valid nonnegative
intervals and their bounds against wall completion; the synthetic unit check
retains null timing when unavailable. The existing UI uses the same enhanced
native probe, but its stats panel still displays the existing wall latency fields.
Neither model work nor disabled-canary execution invokes the new timestamp work.

The first full Rust timestamp run (`after-mtp-timeline`) preserved every MTP
output/reasoning token and EOS. All902 samples had valid Metal timestamps and
none failed. Its worst prefill sample was300.003ms:299.892ms before GPU start,
0.001917ms GPU interval, and0.105958ms after GPU end. The decode maximum was
3.733ms. This directly rules out delayed host return as the dominant cause of
that prefill sample; the queued probe waits for GPU execution. It does not show
that a different application's rendering queue is delayed by the same amount.

The subsequent extraction into the shared native object changes no probe work:
direct `[cb commit]` replaces the wrapper whose model-queue-only hook never
applied to this separate canary queue. Both native bindings pass their model-free
checks after extraction. Final current product binary SHA256:
`50b4b4abbbdc45ff600c1f46d0bec611879249ac8e4d8291d22d656b9c6e9a5d`;
standalone reference binary:
`ccc7a8a774cb1c202add6dba60b04dffe3597822b15a34e22c7e4a5574b50adf`;
shared probe source:
`dd3abc34088ee27ba0759f01a291b9b714114420295252d63e85fd6f326fddab`.

Answer correctness is checked separately from natural termination. Rust AR,
Rust MTP and reference AR passed their generated assertions plus5011 `is_prime`
cases (-10 through5000). The preliminary reference MTP output passed its own
five assertions but failed347 additional cases, first at49: it omits the
`i + 2` divisor test. This is a failed generated Python answer, not by itself
evidence of an engine defect. It must not be reported as a successful code
benchmark merely because EOS was reached. Details are in `python-check.json`.

### Reference clean MTP and record integrity

The clean `reference-mtp-clean` run (both canaries disabled) completed all turns
with the same593/872/167 tokens, text, stop tokens and failed Python answer as
the diagnostic reference run. Its prefill times were5462.306/351.803/434.414ms;
decode21.483/16.754/26.143t/s. The preceding in-process diagnostic measured
23.623/19.287/30.440t/s. This spread must not be disguised as a port speedup or
accepted2% parity: it is one sequential comparison with different probe state,
not controlled repeated clean medians. Canary-on throughput is not the baseline.

The first internal reference run reported788 successful probes, but only787
were independently parseable: a watchdog resource record interrupted one
canary JSON record at a pipe-read boundary. That failed record is preserved in
`reference-mtp-inline.stderr.log`, not silently counted as missing/zero latency.
The supervisor now forwards complete lines in one locked stream write (with a
64KiB cap for newline-free output), while watchdog progress still consumes every
incoming chunk immediately. EOF flushes partial output. A split-record regression
test and all existing memory/start/continuation/long-run watchdog tests pass.
The reference diagnostic is repeated as `reference-mtp-inline-records` for a
fully parseable receipt; the earlier run is retained as the failure evidence.

That repeated reference run completed with **771/771 parseable, successful,
fully timestamped samples** and identical593/872/167 generated tokens/text/EOS.
Prefill p95/max was233.106/264.292ms (48 samples); decode p95/max was
0.226/19.338ms (713 samples). The worst prefill probe waited264.195ms before
GPU start, ran over0.001750ms, and returned to the host0.092083ms after GPU end.
No probe reached2s. The reference's prefill samples also include its short
warmup; the worst sample occurred during the measured summary prefill.
Its diagnostic throughput was23.893/19.177/32.090t/s, not the clean baseline.

| Matched native in-process probe | Prefill p95 ms | Prefill max ms | Decode max ms |
| --- | ---: | ---: | ---: |
| DS4Server, `after-mtp-timeline` |289.978|300.003|3.733|
| Original DS4, `reference-mtp-inline-records` |233.106|264.292|19.338|

These sequential diagnostics reproduce the same GPU-start-wait phenomenon in
the golden master. They do not excuse the remaining Rust prefill cost, establish
statistical latency equivalence, or measure another application's compositor.
Moving inference to another thread cannot by itself reproduce the independent
process's scheduling conditions; process isolation is a distinct architectural
option, not implemented or declared proven as a UI fix here.

Verification at this checkpoint: release all-target/all-feature build; release
all-target/all-feature Clippy with warnings denied; rustfmt and diff checks;
11 model-eval unit tests; both model-free native probe bindings; all4 supervisor
tests; earlier live GLM verifier/progress/HC guards and the full AR/MTP chats.
The updated supervisor fixes measurement transport, not inference scheduling.

## Sampling versus model execution — continued investigation

The prior follow-up made concrete progress (execution fixes plus a fair native
in-process latency reference), but did not establish the full three-model,
AR/speculative2% goal. This continuation addresses the different GLM outputs
before treating their different ongoing histories as matched performance work.

`reference --sampler-fixture` runs the original public `ds4_sample_logits` without
loading a model or using Metal. The checked-in
`tests/fixtures/ds4-sampling-ec7642c.json` contains64 cases: four vocabulary sizes,
eight temperature/top-k/top-p/min-p settings, seeds0/42,32 consecutive tokens
per case and the final RNG state. The original Rust test failed40 of64 cases.
The shared DS4/GLM sampler now preserves the first argmax tie and original
negative sentinel, skips RNG consumption for greedy/all-invalid and the DS4
full-vocabulary min-p fallback, and preserves seed0 until the original RNG's
zero-state substitution. Qwen's independent MTPLX sampler is untouched.
All64 oracle cases and the16 enabled sampling tests pass. Crucially, the positive
temperature/top-p benchmark cases at seed42 already passed before the fix:
these edge corrections are not the explanation for the observed GLM chat gap.

Optional `DS4_REFERENCE_LOGITS_TRACE` records the first32 summary logit rows
through the public original session API. It requires AR mode, creates a new
file rather than overwriting one, and does not change generated tokens or RNG.
The full `reference-ar-logits` chat retained exactly the779/1047/198 tokens,
text and stop tokens of `reference-ar-bootstrap`. Its timings are diagnostic,
not a clean performance baseline. The binary trace contains19,824,640 bytes
(32 rows of154,880 little-endian floats). Its path is serialized as an OsString
and decoded losslessly by the replay test.

`glm53_reference_logits_replay_separates_sampling_from_execution` first samples
those original C-produced rows through the production Rust sampler: **all32
tokens match**. It then opens the installed GLM at Power100/context32768,
prefills the same9-token bootstrap and exact summary suffix, and advances only
with reference-selected tokens. Thus histories never diverge during comparison.
On the Rust-generated rows the test **fails at step17**, choosing906 instead of
the reference320. Already the first post-prefill row has max absolute difference
5.722162 and RMS difference0.851167. All32 per-step row errors are retained in
`logits-replay.stderr.log`; the watched test terminates normally with failure
status in9s. This is a new, deliberately retained red parity test, not a passed
live validation or a speed result. No DS4/GLM/Metal/CPU diagnostic override was
present in the parent environment.

The next localization belongs in the model execution path: compare existing
original DS4 per-layer tensor dumps with the corresponding Rust HC/KDA/DSA/FFN
stages, starting at the first bootstrap/prefill block. Do not explain this away
as stochastic output variation or hide it with a lower chunk/power setting.
No speculative numerical tolerance or new scheduling workaround was applied.

## Root cause: GLM 5.2 chunk boundary applied to GLM 5.3

The active original indexed GLM 5.3 path deliberately keeps full2048-token
chunks across both the old2048 indexer threshold and the4096/8192 dense-attention
threshold. Rust was still applying the GLM 5.2 top-k boundary: after the9-token
bootstrap it evaluated2039 tokens, whereas DS4 evaluated2048. This changes the
recurrent prefill computation, despite identical total prompt tokens.

The original layer0 bootstrap `attn_out` and `ffn_out` dumps matched Rust
bit-for-bit. The original position9 dumps contain2048*4096 floats, establishing
the actual chunk geometry rather than inferring it from configuration.
Detailed HC dump hooks elsewhere in DS4 belong to an inactive dense path and
were not used as evidence for the active indexed execution.

Rust now retains complete GLM 5.3 chunks and splits only the attention slices
at the dense/sparse boundary, as DS4 does. This also removes the incorrect
whole-pair sparse override for a two-row verifier crossing that boundary.
GLM 5.2 retains its old top-k splitting. Unit checks cover both families and
the4096/8192 attention transitions. No smaller chunk, delay, or power reduction
was introduced.

After this correction, the same fixed-history replay is green: **all32 full
154880-value logit rows are bit-identical** to the original trace (max absolute
and RMS error both0), and all sampled tokens agree. This run had no stage
instrumentation enabled. Evidence is retained under
`local-eval-results/glm-stage-20260911.rwQBaJ/mixed-replay.*.log`.
The earlier red replay remains historical evidence, not the current result.
The optional Rust stage reader exists only under `cfg(test)` and validates
tensor geometry before comparing values; it adds no production GPU drains.

The live verifier at frontier4095/context32768 passed across the4096 boundary,
including acceptance, rejection, rewind to either retained frontier, scalar
fallback and recurrent-state restoration (`mixed-boundary.*.log`,70.64s).
The32-row replay alone is not a complete performance or output-parity claim.
The initial source-only note about one-token suffixes was incomplete: the
shared UI/headless consumer already routes one-token extensions through scalar
execution. The actual remaining crossover was two/three-token extensions;
see the subsequent common-consumer correction below.

### Complete chats after the chunk correction

Fresh clean runs used the same ongoing workload, Power100/Low, native EOS,
separate warmup and no active canary. Rust executable SHA256:
`4b23c04325c931854b98c23bd2c98df8a5c2362927aa9b1faed65019d07fd40d`.
The original reference retained its prior tokens/text/stops exactly.

| Mode / turn | Rust tokens | DS4 tokens | Rust decode t/s | DS4 decode t/s | Output + thinking identical |
| --- | ---: | ---: | ---: | ---: | --- |
| AR Summary |779|779|24.512|20.955|yes|
| AR Story |1047|1047|22.956|19.744|yes|
| AR Python |198|198|23.436|20.650|yes|
| MTP Summary |593|593|20.235|21.427|yes|
| MTP Story |905|872|18.039|18.007|no|
| MTP Python |169|167|29.443|29.511|no|

All six Rust turns and six reference turns ended naturally. AR prompt/cached
counts also match exactly. Receipts: `clean-comparison.json`,
`mixed-ar-output-check.json`, `mixed-mtp-output-check.json` in the stage evidence
directory. The AR reference was materially slower than earlier clean runs;
these sequential pairs are not a controlled speedup or2% acceptance claim.
MTP Summary is about5.6% slower in Rust in this pair; the later MTP throughput
numbers do not compare identical histories. Prefill UI-phase and original
session-sync timers still have different boundaries (raw values in the receipt).
Control-loop maxima of40–59ms are not GPU canary or compositor measurements.

### Second root cause: MTP stop token retained in the ongoing frontier

Although MTP Summary text/thinking and593 emitted tokens match, Rust starts
Story with3229 cached tokens and3249 prompt tokens; DS4 uses3228/3248.
The shared Rust generation consumer returned on an MTP stop token without
rewinding the already evaluated block. Both normal and raw original DS4 agent
consumers call `ds4_session_rewind(block_start + ti)` at that point. The standalone
reference's stop handling therefore agrees with its real agent, not just an
arbitrary benchmark convention.

The shared UI/headless consumer now calls `rewind_speculative_output`, a thin
GLM adapter over the existing two-row rollback, to keep exactly
`prompt_tokens + emitted_tokens` before retaining the chat.
This restores the saved two-row KDA state and replays the retained row; it does
not merely truncate IDs or re-render generated text. Invalid frontiers fail
explicitly. Both sampled and greedy generation use this consumer. Qwen's own
whole-turn controller is unchanged. Other model-specific speculative stop
contracts are not claimed validated by this GLM change.
The live verifier regression now exercises that same consumer rollback path.
`align_prompt` is intentionally not used: it retains one fewer token to force
logit recomputation during prompt synchronization, which is a different contract.
The full post-frontier-fix MTP measurement (`frontier-mtp.*.log`) now matches
the original for **all three turns**: text, thinking, emitted token count,
prompt count, cached frontier and natural stop. Emitted counts are593/872/167;
Story starts at3228 cached/3248 prompt tokens, Python at4120/4145. The executable
SHA256 is `b964336d64fbb90b3a9ca595a4705eda02e7afe9c39aedb4ea775e0d52fcf20e`.
`frontier-mtp-output-check.json` has three entries with every equality true;
the checked `jq -e` assertion requires all three entries and all five properties.
Decode rates are23.880/19.357/31.643t/s, versus21.427/18.007/29.511 in the directly
preceding clean original MTP run. This is one sequential pair, not repeated2%
acceptance. The Python answer is now exactly the reference's previously checked
incorrect answer (first counterexample49); matching the oracle does not waive
the independent generated-code quality failure.

### Final regression and responsiveness diagnostics

The final strict replay passes with bit-equal logits at all32 steps. The two
original layer0/position9 stage tensors each contain8388608 floats and also
match bit-for-bit (`final-replay.*.log`,10.21s). The updated live verifier at4095
passes through the same rollback entrypoint used by the consumer, including
invalid/unchanged-frontier checks, rejection and both retained rows
(`final-boundary.*.log`,82.27s). The five enabled GLM unit tests pass.

`final-canary` retained identical full MTP output/frontiers. Its in-memory
summary reports840 samples, no failures, prefill p95/max395.611/483.166ms,
decode max3.820ms and no sample crossing the configured2s threshold. However,
strict raw-log parsing found an interleaved canary/resource JSON record: the
model-eval parent inherited the child's stderr, and both processes serialized
JSON fragments to that descriptor. This raw file is retained as a **failed
record-integrity diagnostic**, not silently filtered into a complete sample set.

The model-eval supervisor now pipes child stderr and forwards complete lines
under the parent's shared stderr lock, the same lock used by resource samples.
Diagnostics do not refresh inference progress deadlines. Reader failures are
reported on join. This fixes the app harness counterpart of the earlier
standalone watchdog forwarding issue; it changes measurement transport, not
GPU scheduling or the UI inference graph.

The directly following original DS4 in-process probe run
(`final-reference-canary`) has841/841 parseable samples, no failures, unchanged
reference tokens/text/stops, prefill p95/max373.220/388.090ms and decode max4.370ms.
The worst prefill sample spent387.964ms before GPU start,0.002875ms over its
GPU interval and0.122ms returning to the host. Thus substantial prefill queue
waiting still occurs in the original oracle; the larger Rust spike is not
declared equivalent or explained away.

The repeated Rust run after the forwarding correction (`final-canary-records`)
completed the entire chat in83.744s and preserved all output/frontier fields.
Every JSON record beginning with `{` in stderr was parsed with `fromjson`
(no error suppression): **776/776 canary records and82/82 resource records**
match the independently reported totals. The checked receipt is
`final-canary-records-check.json`. There are no probe failures or observed2s
threshold crossings. Prefill p95/max is119.507/247.181ms; decode max1.854ms.
The worst sample waits247.062ms before GPU start, spans0.001750ms on the GPU,
and returns after0.115458ms. This lower maximum is not attributed to the
transport-only fix: the prior483ms Rust and388ms original spikes remain recorded,
and scheduling/throughput variability still requires repeated paired testing.
The optional probe remains off by default; no negligible-overhead claim is made.

Final source verification: release all-target/all-feature build and Clippy
with warnings denied; rustfmt/diff checks; five GLM unit tests; eleven
model-eval unit tests; sixteen sampling tests including the64-case original
sampler fixture; strict live logits/stage and consumer-rollback boundary tests.
The final CLI SHA256 is
`cbe04f8ce8f8d2fcb6c82b97c3d85b7bed561418893621a6a653d344d1aa6d85`.
The previously good bundle remains unchanged at SHA256
`ea4d555c2faf0940d9cbcf76d8638ca614a9cb2c6b034e3b2f80aeef86b0b339`.

## Common prompt timing and DS4 CPU sampling follow-up

Evidence for this continuation is under
`local-eval-results/glm-paired-20260911.eClfCS/`. The preceding goal turn made
verified progress (chunk scheduling and stop-token frontier fixes); it did not
establish the full six-cell performance goal.

The shared consumer now uses DS4's GLM5.3 resumed-prefill crossover of2 tokens,
not the generic4-token threshold. DS4 explicitly documents this choice as
measured on M5 Max/GB10 (`ds4.c:36784`). One-token continuations were already
scalar; cold/vision paths and the separate MTPLX whole-turn controller are
unchanged. The enabled crossover test covers GLM5.3 versus GLM5.2/DeepSeek.

DeepSeek/GLM now publish the existing `PromptTiming` at the shared prompt-
evaluation boundary: after restoration/bootstrap, around actual suffix execution
including its progress callbacks, before decode/checkpoint storage. Exact cache
hits report zero evaluated work. Separately unmeasured restore/history components
are `null`, not fabricated zeros; Qwen continues reporting the same measured
numeric values through `Some`. The new metric test and existing Qwen progress/
decode-timer test pass. The ordinary UI-prefill timer remains separately visible.

A fresh clean AR pair kept all three outputs/thinking/token counts/frontiers
identical. Rust's engine-prefill times were5235.183/269.769/320.767ms, original
DS4 session-sync8160.750/495.857/479.216ms; Rust decode24.811/23.114/23.410t/s
versus16.633/17.333/19.054. These large sequential-run differences are not a
controlled speedup or a completed repeat matrix (`baseline-ar-comparison.json`).
The reference driver now additionally queries and checks actual engine power100
after load, rather than only recording its requested options.

The CPU sampler still differed algorithmically: Rust sorted the full vocabulary
and drew from renormalized probabilities, while DS4 first tries a512-candidate
heap and draws from raw retained weights. A CPU-only replay uses the existing32
full logit rows, one32-draw warmup and16 measured batches (512 draws). The same
small runner serves the independent original public `ds4_sample_logits` and the
production Rust sampler. It loads no model and performs no Metal work; both are
supervised with1GiB memory/start30s/idle30s limits. The original public function
allocates a scratch buffer per call, unlike its session API, so its microbenchmark
is not an exact measure of session-sampler overhead.

Before alignment Rust took2.645ms/draw versus original0.834ms, with all512 tokens
equal. The aligned Rust path initially measured0.401ms/draw with the same512
tokens (`sampler-{before,after}-rust.json`, `sampler-reference.stdout.log`).
It uses stdlib `BinaryHeap`, DS4's logit/index tie order, bounded-nucleus fallback
without advancing RNG, original raw cumulative sampling, full-vocabulary/min-p
fallback and the original expf-verified log-space rejection boundary. Top-k
retains the original1024 cap. Separate distribution materialization for
speculative correction and Qwen's MTPLX sampler are untouched.

All64 original sampler fixture cases and17 enabled sampling tests pass, as does
the added missing-mass/near-one fallback, RNG and signed-zero tie check. Release
all-target/all-feature build and warnings-denied Clippy pass. The new executable
SHA256 is `ece6aed3601fb402e6dba6ac2e289d6e0c2dc86663600c3d4b1a4cc07e8fb42c`.
The first full post-sampler AR and MTP pairs both preserve all three outputs,
thinking, completion/prompt/cached counts and natural stops. The independently
queried reference engine reports power100. Receipts are
`sampler-{ar,mtp}-comparison.json`; these are single pairs, not the repeat matrix.

| Mode / turn | Rust / original engine-prefill ms | Rust / original decode t/s |
| --- | ---: | ---: |
| AR Summary | 5208.845 / 5548.775 | 26.129 / 24.711 |
| AR Story | 267.263 / 287.268 | 24.553 / 23.728 |
| AR Python | 319.127 / 341.355 | 24.964 / 24.520 |
| MTP Summary | 6574.400 / 5427.866 | 23.935 / 23.628 |
| MTP Story | 273.240 / 278.956 | 19.419 / 19.089 |
| MTP Python | 315.279 / 360.621 | 33.140 / 31.919 |

The Summary MTP prefill regression in this pair remains visible despite the
slightly faster Rust decode. Reversed-order repetitions are needed to distinguish
run variability from a repeatable graph cost. AR before/after the sampler keeps
the entire chat output identical and improves decode by5.310/6.225/6.638% in this
one sequential comparison (`sampler-ar-before-after.json`); no controlled causal
end-to-end percentage is inferred from that pair alone.

MTP is not universally beneficial in the original either: its Story decode is
19.089t/s versus23.728 AR, while Python is31.919 versus24.520. Rust's full MTP
cycle receipts show228/366,289/584 and82/86 accepted drafts respectively
(62.3%,49.5%,95.3%). The corresponding complete decode-loop time per cycle is
67.69/76.89/58.60ms. At1.62/1.49/1.94 emitted tokens per cycle, the Python case
amortizes the extra draft/verification work much better. These are whole-cycle
averages, not isolated kernel timings: the existing `verifier_ms` includes other
cycle work and must not be presented as an exclusive verification stage.
AR and MTP have different natural histories, so their t/s comparison is not a
matched-token microbenchmark. The previously recorded Python correctness failure
also remains open even though both implementations produce the same code.

### Reversed-order pairs: acceptance still fails

Both modes were repeated in original-then-Rust order, serially without builds
or canary probes. All twelve measured answers in these four processes again
match text/thinking/counts/cache frontiers and end naturally; all watchdogs
exit successfully. No slow run was discarded (`repeat2-*-comparison.json`).

| Mode / turn | Rust / original engine-prefill ms | Rust / original decode t/s |
| --- | ---: | ---: |
| AR Summary | 6738.280 / 5283.041 | 23.569 / 25.535 |
| AR Story | 309.647 / 281.591 | 21.743 / 24.186 |
| AR Python | 387.381 / 329.667 | 21.107 / 24.898 |
| MTP Summary | 9240.178 / 8835.102 | 17.706 / 17.678 |
| MTP Story | 358.905 / 402.447 | 15.146 / 14.581 |
| MTP Python | 403.605 / 491.547 | 25.843 / 23.655 |

AR decode now misses by7.70/10.10/15.23%; MTP Summary prefill misses by4.38%.
The subsequent original MTP run is itself much slower than its first run.
This excludes neither a Rust scheduling difference nor changing device clocks;
it does preclude a pass based on the favorable first pair or a selected median.
The required third pair and full six-cell acceptance remain outstanding.

Rust AR emits exactly9445/12612/2424 command buffers in both repetitions, with
the same outputs, but its GPU timestamp-interval sums increase from
34536/42061/8084ms to39285/47601/9590ms (`ar-drift-comparison.json`). Those sums
are `GPUEndTime - GPUStartTime` and may include preemption; they are not exclusive
kernel or hardware-clock measurements. The slowdown is not explained by changed
token counts or extra command buffers, and is not declared thermal throttling.
During the sequence, a read-only process snapshot showed only the intended
reference model process. macOS reported no recorded thermal/performance warning
and normal VM pressure (1), which does not exclude frequency changes. The
AGX PerformanceStatistics snapshot exposes utilization but no frequency field.
Hardware was freshly checked: Apple M5 Max,128GiB,18 logical CPUs.

## Remaining acceptance

- Compare repeated clean throughput pairs; GLM ongoing histories now match in
  both modes, but sequential run variability does not establish2% performance parity.
- Localize the remaining GLM prefill cost against original DS4's active indexed
  path, now that in-process GPU-start waiting is observable on both sides.
  The engine-prefill timer is now exposed separately from UI-phase timing;
  use that aligned boundary in the paired comparisons.
  No chunk reduction or extra waits are justified by these measurements alone.
- Verify the corrected short-extension crossover live where needed, and other
  model-specific speculative stop contracts; the recorded GLM workload does not
  cover every possible interaction. Full chats pass after CPU-sampler alignment;
  repeated timing acceptance remains separate.
- Validate the remaining SSD expert-streaming cases separately from resident
  scheduling. This is unrelated to replacing DS4 KV checkpoint persistence.
- Complete the DeepSeek AR/DSpark reference cells and Qwen residual performance
  analysis. Interactive confirmations are not a substitute for the six-cell
  numerical acceptance matrix.

No bundle replacement, commit or push has been performed by this follow-up so far.
All processes have terminated. The subsequent DeepSeek comparison and its
separate bootstrap/DSpark findings are recorded in
[DeepSeek follow-up](deepseek-reference-followup-20260911.md).