DS4Server/docs/issue104-ar/README.md

Issue 104: retire the Qwen AR pipeline

The Qwen AR driver now samples every token through sample_from_logits and the request's PCG64 stream, following MTPLX 9882703f3105363ddc37eca9f97aa09a1d387112. The obsolete pipeline option, admission, keyed decode loop, cached speculative tail, model mode and compile-policy lane are removed. Classic tail capture forwards the final sampled token before banking its state.

The server no longer supplies or reads MTPLX_AR_PIPELINE. The installed-model reference harness cannot enable it either. Generic PLE staging/resident-table fields remain for their independently tested callers. Their classic defaults are preserved. The historical lazy sampling primitive and its scalar keyed categorical wrapper are test-only, retained to reproduce the original sampling law receipt and exercise the existing RNG fixtures.

Baseline and oracle

Pre-fix production revision: 5773942c8ffe41352104d673ff0eef949df10c71, with only the installed-model diagnostic added. Model: unchanged Bare-Speed revision 74559cdf34fbfc0b593de72d17e93f37fd4f9ea7; MLX 0.32.2; Apple M5 Max / 128 GiB. Each model process ran serially under test-supervisor 114688 60 120.

Configuration	Actual old lane	Temperature .6, p .95, k 20, seed 0, eight tokens
SSD, pipeline disabled	Classic	314, 279, 198, 2, 20732, 198, 2, 198
SSD, pipeline enabled	Classic	Same tokens and every final-state hash
Full resident table, pipeline disabled	Classic	Same tokens and every final-state hash
Full resident table, pipeline enabled	Keyed pipeline	314, 279, 35067, 82, 13, 220, 19, 15

The temperature-1 control also diverged at token 3 only with the resident pipeline. Its classic tokens were 314,279,198,2,20732,20086,198,2053; pipeline tokens were 314,279,695,47,4920,82,5212,2502. Full resident attachment was asserted, and the old driver's actual Decoded::pipeline was recorded, rather than inferring engagement from an environment flag.

tools/issue104-reference.py runs the pinned original generate_ar and SessionBank against the installed artifacts. The checked-in tests/fixtures/issue104-ar-model.json contains cold and warm token IDs, final logits and every trunk-state array's shape/hash. Cold receipts equal both pre-fix native classic baselines in full. The warm turn restores all 11 committed tokens and adds suffix [1075] before decoding eight more tokens.

Native regression issue104_installed_ar_stream_and_state checks those complete receipts with synchronous and asynchronous evaluation. It also cancels a warm turn after three callbacks, checks that the saved state is intact, and retries from that state. Every completed turn asserts that PLE staging remains enabled and resident preference remains in classic mode. Run with MTPLX_AR_PIPELINE=1 to exercise an explicit obsolete flag without recreating the removed lane.

Scope review

The checked route is request/session ownership → Execution::decode_ar → ar::decode → classic sampling → final-token capture. No production AR caller can reach the lazy primitive. Every former admission and mode call in policy, model, decoder, binding and installed reference tests was removed. GDN compilation still obeys its two existing environment gates, including explicit-off precedence. Normal prefill and MTP retain their compilation/staging paths. DS4/GLM sampling, RNG, inference and Metal kernels have no changes.

The 1,120 fresh MTPLX orchestration fixtures cover both obsolete flag values, sync/async evaluation, official .6/.95/k20, 1/.95/k20, .5/k2, p=0/1, greedy, presence penalty, zero/one/two/three/twelve-token limits, stops and final capture. Existing sampler fixtures additionally exercise tied logits, frequency penalties and exact PCG64 continuation. The historical primitive-law receipt remains in docs/issue103-analysis/native-sampling-final.log.

The server-policy oracle's --server-only mode compares all existing native defaults and verifies pipeline removal across 268 environment/MTP cases. Current upstream additionally supplies defaults for other, unported features and has replaced its old boundary-count diagnostic API; this scoped check does not claim those unrelated policy extensions are implemented.

Timing interpretation

receipts.jsonl records measured decode seconds separately from state equality and process memory peaks. These are bounded eight-token correctness probes, including first-use and memory-pressure effects, not steady-state throughput claims. Old resident classic/pipeline times at .6 were 7.576/17.518 seconds; at 1.0 they were 1.332/5.452 seconds. SSD classic repeated at .6 took 0.649/1.120 seconds and at 1.0 took 1.280/1.273 seconds. No overlap is retained to preserve these old timings.

After removal, synchronous SSD decode took 0.642/1.309 seconds at temperatures .6/1.0; synchronous resident decode took 0.813/1.427 seconds. Async controls took 1.182/1.298 seconds (SSD) and 1.435/1.426 seconds (resident). All produce the classic stream and state. Peak process physical footprint was 72.5 GB for the SSD regression and 104.5 GB for the resident regression.

Completed validation

• Both installed-model regressions passed with the obsolete flag explicitly on, including warm state, cancellation/retry, sync/async and pre-fix equality.
• All 1,120 AR orchestration fixtures, 1,080 sampler-entry fixtures, the penalty and distribution/RNG fixtures, the original sampling-law diagnostic, and the connected decoder/cache control passed under the supervisor.
• The scoped 268-case current server-default comparison passed. The ordinary suite includes DS4's 64 exact token/RNG fixtures and compile-policy controls.
• cargo fmt --all -- --check, warnings-denied Clippy, make bundle including signing, and cargo test --all-features passed (308 application tests plus two tool suites of four tests each; 211 opt-in application tests ignored).
• Every artifact SHA256 was re-read and matched the unchanged revision; see environment.json. Compact run results are in checks.json and receipts.jsonl. Source review found no DS4/GLM changes and no remaining production AR-pipeline admission or mode path.

Reproduction

Build the test executable with cargo test --all-features --bin ds4-server --no-run; pass that executable, not Cargo, to the supervisor. Set DS4_QWEN38_ARTIFACTS to the installed artifact directory, and set MTPLX_COMPILED_GDN=1 MTPLX_QWEN4EXP_COMPILE=1. Run issue104_installed_ar_stream_and_state --ignored --nocapture --test-threads=1 once with default SSD policy and once with MTPLX_NGRAM_RESIDENT=1 and adequate RAM. Set MTPLX_AR_PIPELINE=1 on both runs. Optional DS4_ISSUE104_BASELINE compares the cold records to a separately saved pre-fix classic receipt.

For the isolated Python oracle, point PYTHONPATH at pinned MTPLX, use the MLX-0.32.2 environment, the same compilation flags, and MTPLX_SUSTAINED_PREFILL=1 MTPLX_PREFILL_CHUNK_SIZE=2048. Run tools/issue104-reference.py MODEL_DIRECTORY under the same supervisor.