hugo/DS4Server
1
0 Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Retire the Qwen AR pipeline

Browse Source
This commit is contained in:
Georg Bauer committed 2026-10-04 12:49:43 +02:00
1 parent 5773942c8f
commit 1de78fa3ef
22 files changed
+6429 -667
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/issue104-ar/README.md
+120
View File
@@ -0,0 +1,120 @@
# Issue 104: retire the Qwen AR pipeline
The Qwen AR driver now samples every token through `sample_from_logits` and
the request's PCG64 stream, following MTPLX `9882703f3105363ddc37eca9f97aa09a1d387112`.
The obsolete pipeline option, admission, keyed decode loop, cached speculative
tail, model mode and compile-policy lane are removed. Classic tail capture
forwards the final sampled token before banking its state.
The server no longer supplies or reads `MTPLX_AR_PIPELINE`. The installed-model
reference harness cannot enable it either. Generic PLE staging/resident-table
fields remain for their independently tested callers. Their classic defaults
are preserved. The historical lazy sampling primitive and its scalar keyed
categorical wrapper are test-only, retained to reproduce the original sampling
law receipt and exercise the existing RNG fixtures.
## Baseline and oracle
Pre-fix production revision: `5773942c8ffe41352104d673ff0eef949df10c71`, with only
the installed-model diagnostic added. Model: unchanged Bare-Speed revision
`74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`; MLX 0.32.2; Apple M5 Max / 128 GiB.
Each model process ran serially under `test-supervisor 114688 60 120`.
| Configuration | Actual old lane | Temperature .6, p .95, k 20, seed 0, eight tokens |
| --- | --- | --- |
| SSD, pipeline disabled | Classic | 314, 279, 198, 2, 20732, 198, 2, 198 |
| SSD, pipeline enabled | Classic | Same tokens and every final-state hash |
| Full resident table, pipeline disabled | Classic | Same tokens and every final-state hash |
| Full resident table, pipeline enabled | Keyed pipeline | 314, 279, 35067, 82, 13, 220, 19, 15 |
The temperature-1 control also diverged at token 3 only with the resident
pipeline. Its classic tokens were `314,279,198,2,20732,20086,198,2053`; pipeline
tokens were `314,279,695,47,4920,82,5212,2502`. Full resident attachment was asserted,
and the old driver's actual `Decoded::pipeline` was recorded, rather than
inferring engagement from an environment flag.
`tools/issue104-reference.py` runs the pinned original `generate_ar` and
SessionBank against the installed artifacts. The checked-in
`tests/fixtures/issue104-ar-model.json` contains cold and warm token IDs, final
logits and every trunk-state array's shape/hash. Cold receipts equal both
pre-fix native classic baselines in full. The warm turn restores all 11 committed
tokens and adds suffix `[1075]` before decoding eight more tokens.
Native regression `issue104_installed_ar_stream_and_state` checks those complete
receipts with synchronous and asynchronous evaluation. It also cancels a warm
turn after three callbacks, checks that the saved state is intact, and retries
from that state. Every completed turn asserts that PLE staging remains enabled
and resident preference remains in classic mode. Run with `MTPLX_AR_PIPELINE=1`
to exercise an explicit obsolete flag without recreating the removed lane.
## Scope review
The checked route is request/session ownership → `Execution::decode_ar` →
`ar::decode` → classic sampling → final-token capture. No production AR caller
can reach the lazy primitive. Every former admission and mode call in policy,
model, decoder, binding and installed reference tests was removed. GDN compilation
still obeys its two existing environment gates, including explicit-off precedence.
Normal prefill and MTP retain their compilation/staging paths. DS4/GLM sampling,
RNG, inference and Metal kernels have no changes.
The 1,120 fresh MTPLX orchestration fixtures cover both obsolete flag values,
sync/async evaluation, official .6/.95/k20, 1/.95/k20, .5/k2, p=0/1, greedy,
presence penalty, zero/one/two/three/twelve-token limits, stops and final capture.
Existing sampler fixtures additionally exercise tied logits, frequency penalties
and exact PCG64 continuation. The historical primitive-law receipt remains in
`docs/issue103-analysis/native-sampling-final.log`.
The server-policy oracle's `--server-only` mode compares all existing native
defaults and verifies pipeline removal across 268 environment/MTP cases.
Current upstream additionally supplies defaults for other, unported features and
has replaced its old boundary-count diagnostic API; this scoped check does not
claim those unrelated policy extensions are implemented.
## Timing interpretation
`receipts.jsonl` records measured decode seconds separately from state equality
and process memory peaks. These are bounded eight-token correctness probes,
including first-use and memory-pressure effects, not steady-state throughput
claims. Old resident classic/pipeline times at .6 were 7.576/17.518 seconds;
at 1.0 they were 1.332/5.452 seconds. SSD classic repeated at .6 took
0.649/1.120 seconds and at 1.0 took 1.280/1.273 seconds. No overlap is retained
to preserve these old timings.
After removal, synchronous SSD decode took 0.642/1.309 seconds at temperatures
.6/1.0; synchronous resident decode took 0.813/1.427 seconds. Async controls took
1.182/1.298 seconds (SSD) and 1.435/1.426 seconds (resident). All produce the
classic stream and state. Peak process physical footprint was 72.5 GB for the
SSD regression and 104.5 GB for the resident regression.
## Completed validation
- Both installed-model regressions passed with the obsolete flag explicitly on,
including warm state, cancellation/retry, sync/async and pre-fix equality.
- All 1,120 AR orchestration fixtures, 1,080 sampler-entry fixtures, the penalty
and distribution/RNG fixtures, the original sampling-law diagnostic, and the
connected decoder/cache control passed under the supervisor.
- The scoped 268-case current server-default comparison passed. The ordinary
suite includes DS4's 64 exact token/RNG fixtures and compile-policy controls.
- `cargo fmt --all -- --check`, warnings-denied Clippy, `make bundle` including
signing, and `cargo test --all-features` passed (308 application tests plus
two tool suites of four tests each; 211 opt-in application tests ignored).
- Every artifact SHA256 was re-read and matched the unchanged revision;
see `environment.json`. Compact run results are in `checks.json` and
`receipts.jsonl`. Source review found no DS4/GLM changes and no remaining
production AR-pipeline admission or mode path.
## Reproduction
Build the test executable with `cargo test --all-features --bin ds4-server
--no-run`; pass that executable, not Cargo, to the supervisor. Set
`DS4_QWEN38_ARTIFACTS` to the installed artifact directory, and set
`MTPLX_COMPILED_GDN=1 MTPLX_QWEN4EXP_COMPILE=1`. Run
`issue104_installed_ar_stream_and_state --ignored --nocapture --test-threads=1`
once with default SSD policy and once with `MTPLX_NGRAM_RESIDENT=1` and adequate
RAM. Set `MTPLX_AR_PIPELINE=1` on both runs. Optional `DS4_ISSUE104_BASELINE`
compares the cold records to a separately saved pre-fix classic receipt.
For the isolated Python oracle, point `PYTHONPATH` at pinned MTPLX, use the
MLX-0.32.2 environment, the same compilation flags, and
`MTPLX_SUSTAINED_PREFILL=1 MTPLX_PREFILL_CHUNK_SIZE=2048`. Run
`tools/issue104-reference.py MODEL_DIRECTORY` under the same supervisor.
docs/issue104-ar/checks.json
+36
View File
@@ -0,0 +1,36 @@
{
"clippy": [
" Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.49s"
],
"bundle": [
" Finished `release` profile [optimized] target(s) in 49.59s",
"target/release/DS4Server.app: replacing existing signature"
],
"tests": [
" Finished `test` profile [unoptimized + debuginfo] target(s) in 0.25s",
"test result: ok. 308 passed; 0 failed; 211 ignored; 0 measured; 0 filtered out; finished in 6.01s",
"test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s",
"test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.30s"
],
"mtplx_ar_driver_matches_reference": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 2.10s"
],
"mtplx_sampler_entry_matches_reference": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 56.60s"
],
"mtplx_sampler_penalties_match_reference": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 7.94s"
],
"mtplx_sampler_distribution_and_rng_match_reference": [
"test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 519 filtered out; finished in 0.00s"
],
"issue103_sampling_receipts": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 1.48s"
],
"mtplx_decoder_connected_attention_and_moe_advance_caches": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 518 filtered out; finished in 0.79s"
],
"fmt": "passed",
"diff_check": "passed",
"scoped_current_server_policy": "268 cases passed"
}
docs/issue104-ar/environment.json
+45
View File
@@ -0,0 +1,45 @@
{
"base_revision": "5773942c8ffe41352104d673ff0eef949df10c71",
"oracle_revision": "9882703f3105363ddc37eca9f97aa09a1d387112",
"model_revision": "74559cdf34fbfc0b593de72d17e93f37fd4f9ea7",
"model_root": "/Users/gb/Library/Application Support/de.rfc1437.ds4server/models/qwen3.8-flash-next",
"artifact_verification": "Every SHA256 re-read and matched on 2026-10-04",
"artifact_sha256": {
"LICENSE": "a0dc422560841fd68e06d974907f8b4c709bca44a67daad2b528437bdf676c08",
"chat_template.jinja": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041",
"config.json": "826302af9ff3293cfc9975c9a5a96546fa0aa9f0808505adfc30a62fef806965",
"generation_config.json": "e70c136c1b78ddc1fb0905bac8e733a4dc448d4f852a5dd75143fffc70be550e",
"merges.txt": "a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d",
"model-00001-of-00017.safetensors": "a27232c9434b9d8961f198cf36f44e228b3f23a5f8d874ba4ca8af32a1b23ffc",
"model-00002-of-00017.safetensors": "6268ae80950f6b935fca672fae5aa455eb3ea4f139730e54e6e5f8581faa94dc",
"model-00003-of-00017.safetensors": "71a9abbd420dce567f2c06593bfee2d6cea681989d51eaef1b1c5dffd65a844b",
"model-00004-of-00017.safetensors": "c8112da735bb1c8dbc4757926ab8ec1eba7a707a8767c7e43c6150586cf63a4c",
"model-00005-of-00017.safetensors": "48258b931e7596c1b43c8b38c044060787ef6d85cca2c1935852bc962801f792",
"model-00006-of-00017.safetensors": "f357d2d66f7b768ebb19ba464556bbe46c572cf3cd1f8226faddf9216b150d0c",
"model-00007-of-00017.safetensors": "d3152e808d013e3afa4a9c391f6d8088e74a10303c89d45fb61dad7e457160f6",
"model-00008-of-00017.safetensors": "1d93bd4872f338fa19416bac64ebeb05aedf2721a07ac48e37c53921252c95c1",
"model-00009-of-00017.safetensors": "d22d4be580296e3d4fa5f4fe5902c8fc448d8204b6dcde4641f87b9c675930ca",
"model-00010-of-00017.safetensors": "111d0166c888cabb7d6fc19c4bb7eb42e2b51946f71b4983dd750b031755637e",
"model-00011-of-00017.safetensors": "73dad882cf94a5486cfe0dcda876b0f740556bc469608d2ced7a07540e699803",
"model-00012-of-00017.safetensors": "a6057d5a06276461760a4f4e88991550a43abdc5f8afacef06e860f5e7843393",
"model-00013-of-00017.safetensors": "ff25dc6e1c250c9b9ece83d59111a4c4055dad1b1f160546c614dcc17395b152",
"model-00014-of-00017.safetensors": "d8c9de3ae28abccc8e3da09dc92fbbed4039d6d65dabb2657edebc11b2b09652",
"model-00015-of-00017.safetensors": "a5438d468074c709febdeadb9f6a2db54fa9fb30a37c33de78584f28d6fa1191",
"model-00016-of-00017.safetensors": "4a07409bd568cd1435641495d90c805dcb0d0e5d8dc64409273d604a7d06b342",
"model-00017-of-00017.safetensors": "9c064e0b8dc0054e2b453fe2b8450c287e1ede6e74bb74ecf5aaed1ef9271913",
"model-vision.safetensors": "5aff6983f36f52acbecc1f61f703ae7b927e74525bf3b1133ba276bb7a5b52b1",
"model.safetensors.index.json": "81e77f5b25bb29e25e239ae2f9f324c9cada379ee600441d6c229f1d95a13443",
"mtp.safetensors": "1bd4b2fc20a93b53f22b4cab0b14d68bc0fe666bb64e4e2f898872c30d7d26ae",
"mtplx_runtime.json": "381c7666872f878542494a48a874bb0784c72a955d18cc077324d3c9e74b8cdb",
"ngram-table.safetensors": "af7f0880c1798e60353b07cc6b0faa0d41f78734818d858ff013ab7e3bce9de5",
"preprocessor_config.json": "27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516",
"processor_config.json": "dcc7ce0121d6d1c514804e68e5673e04c239ad2547339ea6330289d272a748f8",
"tokenizer.json": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3",
"tokenizer_config.json": "b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27",
"video_preprocessor_config.json": "7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13",
"vocab.json": "ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003"
},
"physical_memory_bytes": 137438953472,
"os": "macOS 26.6.2 (25G83)",
"hardware": "Apple M5 Max"
}
docs/issue104-ar/receipts.jsonl
+37
View File
@@ -0,0 +1,37 @@
{"run":"before-ssd","event":"issue104_environment","resident_tables":[false],"mtp":false}
{"run":"before-ssd","event":"issue104_ar","temperature":0.6,"legacy":false,"pipeline":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":0.64901575,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
{"run":"before-ssd","event":"issue104_ar","temperature":0.6,"legacy":true,"pipeline":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":1.119514625,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
{"run":"before-ssd","event":"issue104_ar","temperature":1.0,"legacy":false,"pipeline":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.279887375,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
{"run":"before-ssd","event":"issue104_ar","temperature":1.0,"legacy":true,"pipeline":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.273476917,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
{"run":"before-ssd","event":"test_resource_summary","elapsed_ms":34988,"peak":{"cpu_time_ns":32497220166,"resident_bytes":59135393792,"physical_bytes":72378445512,"peak_physical_bytes":72378478280,"disk_read_bytes":71780679680,"disk_write_bytes":9781248},"error":null}
{"run":"before-resident","event":"issue104_environment","resident_tables":[true],"mtp":false}
{"run":"before-resident","event":"issue104_ar","temperature":0.6,"legacy":false,"pipeline":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":7.5761748749999995,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
{"run":"before-resident","event":"issue104_ar","temperature":0.6,"legacy":true,"pipeline":true,"tokens":[314,279,35067,82,13,220,19,15],"seconds":17.51813525,"state_sha256":"780fc40e50d844db43f003860b865aa123e28d84be3ad7ac585c5197ac3e502f"}
{"run":"before-resident","event":"issue104_ar","temperature":1.0,"legacy":false,"pipeline":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.332490958,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
{"run":"before-resident","event":"issue104_ar","temperature":1.0,"legacy":true,"pipeline":true,"tokens":[314,279,695,47,4920,82,5212,2502],"seconds":5.452344583,"state_sha256":"6d85808248625e1691696f38ebd1cccc4e46d5e96439d7e0fa156841014b33ff"}
{"run":"before-resident","event":"test_resource_summary","elapsed_ms":115309,"peak":{"cpu_time_ns":72117394541,"resident_bytes":55280517120,"physical_bytes":104389150752,"peak_physical_bytes":104440955808,"disk_read_bytes":119938990080,"disk_write_bytes":18446744073709518848},"error":null}
{"run":"after-ssd","event":"issue104_environment","resident_tables":[false],"mtp":false}
{"run":"after-ssd","event":"issue104_ar","temperature":0.6,"asynchronous":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":0.642156958,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
{"run":"after-ssd","event":"issue104_warm","temperature":0.6,"asynchronous":false,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
{"run":"after-ssd","event":"issue104_ar","temperature":0.6,"asynchronous":true,"tokens":[314,279,198,2,20732,198,2,198],"seconds":1.182015375,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
{"run":"after-ssd","event":"issue104_warm","temperature":0.6,"asynchronous":true,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
{"run":"after-ssd","event":"issue104_ar","temperature":1.0,"asynchronous":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.30897475,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
{"run":"after-ssd","event":"issue104_warm","temperature":1.0,"asynchronous":false,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
{"run":"after-ssd","event":"issue104_ar","temperature":1.0,"asynchronous":true,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.298059,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
{"run":"after-ssd","event":"issue104_warm","temperature":1.0,"asynchronous":true,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
{"run":"after-ssd","event":"test_resource_summary","elapsed_ms":62881,"peak":{"cpu_time_ns":65904669875,"resident_bytes":61079650304,"physical_bytes":72499870168,"peak_physical_bytes":72499870168,"disk_read_bytes":71786532864,"disk_write_bytes":344064},"error":null}
{"run":"after-resident","event":"issue104_environment","resident_tables":[true],"mtp":false}
{"run":"after-resident","event":"issue104_ar","temperature":0.6,"asynchronous":false,"tokens":[314,279,198,2,20732,198,2,198],"seconds":0.813106875,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
{"run":"after-resident","event":"issue104_warm","temperature":0.6,"asynchronous":false,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
{"run":"after-resident","event":"issue104_ar","temperature":0.6,"asynchronous":true,"tokens":[314,279,198,2,20732,198,2,198],"seconds":1.435416458,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
{"run":"after-resident","event":"issue104_warm","temperature":0.6,"asynchronous":true,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
{"run":"after-resident","event":"issue104_ar","temperature":1.0,"asynchronous":false,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.426774333,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
{"run":"after-resident","event":"issue104_warm","temperature":1.0,"asynchronous":false,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
{"run":"after-resident","event":"issue104_ar","temperature":1.0,"asynchronous":true,"tokens":[314,279,198,2,20732,20086,198,2053],"seconds":1.4262525,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
{"run":"after-resident","event":"issue104_warm","temperature":1.0,"asynchronous":true,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
{"run":"after-resident","event":"test_resource_summary","elapsed_ms":115119,"peak":{"cpu_time_ns":89721565333,"resident_bytes":62532075520,"physical_bytes":104507069232,"peak_physical_bytes":104507069232,"disk_read_bytes":104724783104,"disk_write_bytes":0},"error":null}
{"run":"oracle-warm","event":"issue104_oracle","temperature":0.6,"warm":false,"tokens":[314,279,198,2,20732,198,2,198],"cached_tokens":0,"seconds":4.613554999988992,"state_sha256":"62ab1517fbe639323f74633514726721d025b6d51635f4f81de73bb8851b94b0"}
{"run":"oracle-warm","event":"issue104_oracle","temperature":0.6,"warm":true,"tokens":[314,279,198,2,20732,198,2,198],"cached_tokens":11,"seconds":0.18006633399636485,"state_sha256":"cd0e50af7ec373da52da7e32d94d7a55ce11c9cb3fc68d7d1d2c0d4607a927bc"}
{"run":"oracle-warm","event":"issue104_oracle","temperature":1.0,"warm":false,"tokens":[314,279,198,2,20732,20086,198,2053],"cached_tokens":0,"seconds":0.20107879099668935,"state_sha256":"9e37724c8c634904d51e3fd982587549c7321e2d12aab1217df951189a730642"}
{"run":"oracle-warm","event":"issue104_oracle","temperature":1.0,"warm":true,"tokens":[25,198,2,198,15666,25835,1608,678],"cached_tokens":11,"seconds":0.17781587500940077,"state_sha256":"9d5d1ac4b918ff915aa9bcd4fc81d8c561094fa0b8cb71a2d759c0924e2a9b54"}
{"run":"oracle-warm","event":"test_resource_summary","elapsed_ms":15510,"peak":{"cpu_time_ns":18162759666,"resident_bytes":61776969728,"physical_bytes":72906603160,"peak_physical_bytes":72906619568,"disk_read_bytes":103772155904,"disk_write_bytes":0},"error":null}