121 lines
6.9 KiB
Markdown
121 lines
6.9 KiB
Markdown
# Issue 104: retire the Qwen AR pipeline
|
|
|
|
The Qwen AR driver now samples every token through `sample_from_logits` and
|
|
the request's PCG64 stream, following MTPLX `9882703f3105363ddc37eca9f97aa09a1d387112`.
|
|
The obsolete pipeline option, admission, keyed decode loop, cached speculative
|
|
tail, model mode and compile-policy lane are removed. Classic tail capture
|
|
forwards the final sampled token before banking its state.
|
|
|
|
The server no longer supplies or reads `MTPLX_AR_PIPELINE`. The installed-model
|
|
reference harness cannot enable it either. Generic PLE staging/resident-table
|
|
fields remain for their independently tested callers. Their classic defaults
|
|
are preserved. The historical lazy sampling primitive and its scalar keyed
|
|
categorical wrapper are test-only, retained to reproduce the original sampling
|
|
law receipt and exercise the existing RNG fixtures.
|
|
|
|
## Baseline and oracle
|
|
|
|
Pre-fix production revision: `5773942c8ffe41352104d673ff0eef949df10c71`, with only
|
|
the installed-model diagnostic added. Model: unchanged Bare-Speed revision
|
|
`74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`; MLX 0.32.2; Apple M5 Max / 128 GiB.
|
|
Each model process ran serially under `test-supervisor 114688 60 120`.
|
|
|
|
| Configuration | Actual old lane | Temperature .6, p .95, k 20, seed 0, eight tokens |
|
|
| --- | --- | --- |
|
|
| SSD, pipeline disabled | Classic | 314, 279, 198, 2, 20732, 198, 2, 198 |
|
|
| SSD, pipeline enabled | Classic | Same tokens and every final-state hash |
|
|
| Full resident table, pipeline disabled | Classic | Same tokens and every final-state hash |
|
|
| Full resident table, pipeline enabled | Keyed pipeline | 314, 279, 35067, 82, 13, 220, 19, 15 |
|
|
|
|
The temperature-1 control also diverged at token 3 only with the resident
|
|
pipeline. Its classic tokens were `314,279,198,2,20732,20086,198,2053`; pipeline
|
|
tokens were `314,279,695,47,4920,82,5212,2502`. Full resident attachment was asserted,
|
|
and the old driver's actual `Decoded::pipeline` was recorded, rather than
|
|
inferring engagement from an environment flag.
|
|
|
|
`tools/issue104-reference.py` runs the pinned original `generate_ar` and
|
|
SessionBank against the installed artifacts. The checked-in
|
|
`tests/fixtures/issue104-ar-model.json` contains cold and warm token IDs, final
|
|
logits and every trunk-state array's shape/hash. Cold receipts equal both
|
|
pre-fix native classic baselines in full. The warm turn restores all 11 committed
|
|
tokens and adds suffix `[1075]` before decoding eight more tokens.
|
|
|
|
Native regression `issue104_installed_ar_stream_and_state` checks those complete
|
|
receipts with synchronous and asynchronous evaluation. It also cancels a warm
|
|
turn after three callbacks, checks that the saved state is intact, and retries
|
|
from that state. Every completed turn asserts that PLE staging remains enabled
|
|
and resident preference remains in classic mode. Run with `MTPLX_AR_PIPELINE=1`
|
|
to exercise an explicit obsolete flag without recreating the removed lane.
|
|
|
|
## Scope review
|
|
|
|
The checked route is request/session ownership → `Execution::decode_ar` →
|
|
`ar::decode` → classic sampling → final-token capture. No production AR caller
|
|
can reach the lazy primitive. Every former admission and mode call in policy,
|
|
model, decoder, binding and installed reference tests was removed. GDN compilation
|
|
still obeys its two existing environment gates, including explicit-off precedence.
|
|
Normal prefill and MTP retain their compilation/staging paths. DS4/GLM sampling,
|
|
RNG, inference and Metal kernels have no changes.
|
|
|
|
The 1,120 fresh MTPLX orchestration fixtures cover both obsolete flag values,
|
|
sync/async evaluation, official .6/.95/k20, 1/.95/k20, .5/k2, p=0/1, greedy,
|
|
presence penalty, zero/one/two/three/twelve-token limits, stops and final capture.
|
|
Existing sampler fixtures additionally exercise tied logits, frequency penalties
|
|
and exact PCG64 continuation. The historical primitive-law receipt remains in
|
|
`docs/issue103-analysis/native-sampling-final.log`.
|
|
|
|
The server-policy oracle's `--server-only` mode compares all existing native
|
|
defaults and verifies pipeline removal across 268 environment/MTP cases.
|
|
Current upstream additionally supplies defaults for other, unported features and
|
|
has replaced its old boundary-count diagnostic API; this scoped check does not
|
|
claim those unrelated policy extensions are implemented.
|
|
|
|
## Timing interpretation
|
|
|
|
`receipts.jsonl` records measured decode seconds separately from state equality
|
|
and process memory peaks. These are bounded eight-token correctness probes,
|
|
including first-use and memory-pressure effects, not steady-state throughput
|
|
claims. Old resident classic/pipeline times at .6 were 7.576/17.518 seconds;
|
|
at 1.0 they were 1.332/5.452 seconds. SSD classic repeated at .6 took
|
|
0.649/1.120 seconds and at 1.0 took 1.280/1.273 seconds. No overlap is retained
|
|
to preserve these old timings.
|
|
|
|
After removal, synchronous SSD decode took 0.642/1.309 seconds at temperatures
|
|
.6/1.0; synchronous resident decode took 0.813/1.427 seconds. Async controls took
|
|
1.182/1.298 seconds (SSD) and 1.435/1.426 seconds (resident). All produce the
|
|
classic stream and state. Peak process physical footprint was 72.5 GB for the
|
|
SSD regression and 104.5 GB for the resident regression.
|
|
|
|
## Completed validation
|
|
|
|
- Both installed-model regressions passed with the obsolete flag explicitly on,
|
|
including warm state, cancellation/retry, sync/async and pre-fix equality.
|
|
- All 1,120 AR orchestration fixtures, 1,080 sampler-entry fixtures, the penalty
|
|
and distribution/RNG fixtures, the original sampling-law diagnostic, and the
|
|
connected decoder/cache control passed under the supervisor.
|
|
- The scoped 268-case current server-default comparison passed. The ordinary
|
|
suite includes DS4's 64 exact token/RNG fixtures and compile-policy controls.
|
|
- `cargo fmt --all -- --check`, warnings-denied Clippy, `make bundle` including
|
|
signing, and `cargo test --all-features` passed (308 application tests plus
|
|
two tool suites of four tests each; 211 opt-in application tests ignored).
|
|
- Every artifact SHA256 was re-read and matched the unchanged revision;
|
|
see `environment.json`. Compact run results are in `checks.json` and
|
|
`receipts.jsonl`. Source review found no DS4/GLM changes and no remaining
|
|
production AR-pipeline admission or mode path.
|
|
|
|
## Reproduction
|
|
|
|
Build the test executable with `cargo test --all-features --bin ds4-server
|
|
--no-run`; pass that executable, not Cargo, to the supervisor. Set
|
|
`DS4_QWEN38_ARTIFACTS` to the installed artifact directory, and set
|
|
`MTPLX_COMPILED_GDN=1 MTPLX_QWEN4EXP_COMPILE=1`. Run
|
|
`issue104_installed_ar_stream_and_state --ignored --nocapture --test-threads=1`
|
|
once with default SSD policy and once with `MTPLX_NGRAM_RESIDENT=1` and adequate
|
|
RAM. Set `MTPLX_AR_PIPELINE=1` on both runs. Optional `DS4_ISSUE104_BASELINE`
|
|
compares the cold records to a separately saved pre-fix classic receipt.
|
|
|
|
For the isolated Python oracle, point `PYTHONPATH` at pinned MTPLX, use the
|
|
MLX-0.32.2 environment, the same compilation flags, and
|
|
`MTPLX_SUSTAINED_PREFILL=1 MTPLX_PREFILL_CHUNK_SIZE=2048`. Run
|
|
`tools/issue104-reference.py MODEL_DIRECTORY` under the same supervisor.
|