Port Qwen row-map SwiGLU expert gathers
This commit is contained in:
Georg Bauer committed
1 parent
8965e77ea1
commit
7b894fbfee
42 files changed
+11466
-42
No files matched your search
@@ -0,0 +1,157 @@
|
|||||||
|
# Qwen row-map expert gather (#112)
|
||||||
|
|
||||||
|
The sorted Qwen gate/up path now retains the uint32 token row map, builds GPU
|
||||||
|
per-expert row/tile prefix sums, and dispatches the pinned 64×64×64 NAX kernel.
|
||||||
|
The kernel reads original token rows and emits SwiGLU directly. Down projection
|
||||||
|
continues through the #110 guarded stock gather. The unsorted decode/verify
|
||||||
|
path is unchanged. Packed and separate gate/up weights preserve their original
|
||||||
|
activation rounding and fallback ordering.
|
||||||
|
|
||||||
|
Oracle: MTPLX `9882703f3105363ddc37eca9f97aa09a1d387112`, MLX 0.32.2.
|
||||||
|
Installed Bare-Speed artifact: `74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`;
|
||||||
|
artifact hashes and machine details are in ../issue104-ar/environment.json.
|
||||||
|
Native pre-port baseline: `8965e77ea173c932e00f70759d0a83ac1c5072f0`.
|
||||||
|
|
||||||
|
Only GPU shader source is shipped from the oracle: the MTPLX kernel body and
|
||||||
|
its MLX 0.32.2 GPU template/functor closure, with MIT licenses in `metal/`.
|
||||||
|
`tools/issue112-extract-metal.py` reproduces those files from the pinned
|
||||||
|
reference environment. All scheduling, policy, graph construction, checking,
|
||||||
|
model loading and sessions remain Rust. There is no production Python/MLX/C
|
||||||
|
runtime dependency and the pinned runtime metallib is unchanged.
|
||||||
|
|
||||||
|
## Route and failure contract
|
||||||
|
|
||||||
|
- All original tensor/layout predicates, affine BF16/F16, b4/b8, groups
|
||||||
|
32/64/128, N/K multiples of 64, and the 4,096-routed-row threshold apply.
|
||||||
|
- Below-threshold, non-NAX and ineligible inputs retain the stock path.
|
||||||
|
`MTPLX_MOE_SORTED_GATHER_KERNEL=0` and the GPU-family rehearsal switch are
|
||||||
|
resolved by Qwen policy without mutating the process environment.
|
||||||
|
- Whole-first-call canaries are keyed by compiled instantiation and the
|
||||||
|
under/from-64-rows-per-expert regime. They compare uint16 bits, including
|
||||||
|
signed zero, then reuse the checked result. Compile/dispatch failures or
|
||||||
|
mismatches disable that key for the process with a reason and counters.
|
||||||
|
- A failed packed SwiGLU canary tries the separately checked plain row-map
|
||||||
|
gather before guarded stock fallback, as upstream does. Separate SwitchGLU
|
||||||
|
returns to its original up/gate/compiled-activation chain.
|
||||||
|
- The only generalized helper is Qwen's GatherQMM: its existing BF16 behavior
|
||||||
|
is retained, with the F16/group-128 specializations needed for the oracle's
|
||||||
|
supported canary/fallback contract. All callers are in the Qwen module.
|
||||||
|
|
||||||
|
## Isolated verification
|
||||||
|
|
||||||
|
The native synthetic matrix covers 72 cases: BF16/F16 × b4/b8 × g32/64/128 ×
|
||||||
|
packed/separate × 4,096/8,193/32,769 routed rows. It includes empty expert runs,
|
||||||
|
partial tiles, both canary regimes, a strided uint32 row map and int64 expert
|
||||||
|
IDs. Every result equals guarded stock and the pinned oracle bit for bit;
|
||||||
|
48 first-use canaries pass. Actual installed layer-0 projections additionally
|
||||||
|
cover 410/2,048/3,277/4,095/4,096/8,192 tokens (12 packed/separate cases).
|
||||||
|
|
||||||
|
Failure injection uses actual invalid Metal source, a changed output sign bit,
|
||||||
|
and explicitly differing positive/negative zero. Each disables its key and
|
||||||
|
subsequent calls use exact stock output without another canary. Metadata and
|
||||||
|
policy tests cover all exclusion predicates and switch spellings. The full
|
||||||
|
model off-switch run checks the unchanged output/state and absence of kernel
|
||||||
|
activation.
|
||||||
|
|
||||||
|
The full-model large-prefix control reuses #110's 7,374-token cold prompt and
|
||||||
|
3,278-token warm suffix at chunks 2,048 and 4,096. It checks every logit and
|
||||||
|
cache array against the pre-port, pinned-oracle receipts, plus cancellation
|
||||||
|
and retry. Explicit hidden-state tests use an MTP-capable runtime with cold
|
||||||
|
4,096-token and warm 2,048-token-suffix prefills; an AR-only runtime correctly
|
||||||
|
omits warm hidden arrays in the oracle. The AR timing tests compare complete
|
||||||
|
state after one generated token. State reads occur after measured intervals.
|
||||||
|
|
||||||
|
## Paired release measurements
|
||||||
|
|
||||||
|
M5 Max, 128 GiB; serial before/after processes, three repetitions per chunk
|
||||||
|
and phase, identical installed artifacts and token IDs. Both default boundary
|
||||||
|
splitting (tail interval 256) and full chunks (tail interval 0) are measured.
|
||||||
|
The latter exercises the wide row regime and guarded partial tails. The
|
||||||
|
release baseline uses unchanged committed production source with only the
|
||||||
|
same measurement test module added. No other GPU workload or build ran during
|
||||||
|
these measurements.
|
||||||
|
|
||||||
|
Cells show median ± sample standard deviation across all three repetitions.
|
||||||
|
Repetition zero includes first-use initialization/canary work; this is not an
|
||||||
|
isolated measurement of canary overhead. All individual times and means are
|
||||||
|
in release-performance.json. TTFT starts at the request call and ends at its
|
||||||
|
first token callback, after model loading.
|
||||||
|
|
||||||
|
| Tail policy / chunk / phase | Before TTFT s | After TTFT s | Before prefill tok/s | After prefill tok/s |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| Default / 2048 / cold | 2.959 ± 0.473 | 2.953 ± 0.480 | 1385.831 ± 183.340 | 1388.955 ± 186.819 |
|
||||||
|
| Default / 2048 / warm | 1.839 ± 0.075 | 1.900 ± 0.088 | 1115.426 ± 45.227 | 1079.363 ± 52.619 |
|
||||||
|
| Default / 4096 / cold | 3.690 ± 0.063 | 3.684 ± 0.029 | 1111.739 ± 19.009 | 1113.593 ± 8.843 |
|
||||||
|
| Default / 4096 / warm | 2.012 ± 0.013 | 1.989 ± 0.019 | 1019.529 ± 6.863 | 1031.159 ± 10.182 |
|
||||||
|
| Full / 2048 / cold | 2.411 ± 0.475 | 2.504 ± 0.428 | 1699.510 ± 259.818 | 1636.600 ± 239.080 |
|
||||||
|
| Full / 2048 / warm | 1.262 ± 0.063 | 1.334 ± 0.052 | 1623.785 ± 78.593 | 1536.347 ± 64.230 |
|
||||||
|
| Full / 4096 / cold | 3.100 ± 0.116 | 2.999 ± 0.136 | 1321.520 ± 49.755 | 1366.360 ± 58.429 |
|
||||||
|
| Full / 4096 / warm | 1.767 ± 0.057 | 1.634 ± 0.086 | 1159.717 ± 36.444 | 1254.103 ± 61.907 |
|
||||||
|
|
||||||
|
Peak physical footprint: default 84.724 → 84.669 GB; full chunks
|
||||||
|
88.579 → 87.952 GB. The results are mixed and do not establish an overall
|
||||||
|
speedup. In particular, the default 256-token tail grid often keeps the small
|
||||||
|
stock route; setting chunk size 4096 does not imply every body call has that
|
||||||
|
width. The port fulfills the current-route obligation without assuming the
|
||||||
|
upstream isolated-kernel speedup applies to this server benchmark.
|
||||||
|
|
||||||
|
## Reproduction
|
||||||
|
|
||||||
|
Set `DS4_QWEN38_ARTIFACTS` to the installed artifact directory. Build the release
|
||||||
|
test executable with `cargo test --release --all-features --bin ds4-server
|
||||||
|
issue112 --no-run`, then copy the executable reported by Cargo outside `target`
|
||||||
|
before running it. Run each ignored test in its own serial process under
|
||||||
|
`target/debug/test-supervisor MEMORY_MIB 60 120 --command TEST_EXECUTABLE FILTER
|
||||||
|
--ignored --nocapture --test-threads=1`:
|
||||||
|
|
||||||
|
- `issue112_all_kernel_variants` and `issue112_canary_failures`: 4096 MiB.
|
||||||
|
- `issue112_installed_gate_up`: 16384 MiB.
|
||||||
|
- `issue112_full_model_state`: 114688 MiB,
|
||||||
|
`DS4_ISSUE110_DIR=OUTPUT_DIRECTORY`, `MTPLX_GDN_BOUNDARY_TAIL_INTERVAL=0`.
|
||||||
|
- `issue112_hidden_prefill`: 114688 MiB,
|
||||||
|
`DS4_ISSUE112_DIR=OUTPUT_DIRECTORY`, `MTPLX_GDN_BOUNDARY_TAIL_INTERVAL=0`.
|
||||||
|
- `issue112_paired_prefill`: 114688 MiB,
|
||||||
|
`DS4_ISSUE112_DIR=OUTPUT_DIRECTORY`, optionally tail interval 0;
|
||||||
|
`DS4_ISSUE112_REPEATS` defaults to 3. Also run a one-repetition full-chunk
|
||||||
|
control with `MTPLX_MOE_SORTED_GATHER_KERNEL=0`.
|
||||||
|
|
||||||
|
Keep child stdout/stderr separate from the supervisor's resource output to
|
||||||
|
avoid interleaved JSON writes. Under the isolated MLX 0.32.2 reference Python,
|
||||||
|
run `tools/issue112-reference.py PINNED_CHECKOUT NATIVE_VARIANT_OUTPUT`,
|
||||||
|
`tools/issue112-installed-reference.py PINNED_CHECKOUT MODEL_DIR NATIVE_OUTPUT`,
|
||||||
|
and `tools/issue112-model-reference.py PINNED_CHECKOUT MODEL_DIR NATIVE_DIR
|
||||||
|
ORACLE_DIR` (add `--hidden` for the hidden-state run; use matching tail policy).
|
||||||
|
The full-model oracle uses the server's lazy-snapshot policy for cache
|
||||||
|
fingerprints, matching the native session entry; a deep clone adds zero and
|
||||||
|
would change negative-zero key bits during measurement itself.
|
||||||
|
|
||||||
|
For the before arm, build commit `8965e77` with only `issue112_bench.rs` and its
|
||||||
|
`#[cfg(test)] mod issue112_bench;` registration added. Do not use an off-switch
|
||||||
|
run as the only baseline: the preserved pre-port executable is the baseline.
|
||||||
|
Both arms use the same test harness. Run timing arms without other builds or
|
||||||
|
GPU work; correctness checks may run alongside CPU-only compilation.
|
||||||
|
|
||||||
|
## Completion review
|
||||||
|
|
||||||
|
All 72 synthetic and 12 installed projection receipts match guarded stock
|
||||||
|
and pinned MTPLX; compile/sign-bit/signed-zero failures pass sticky-fallback
|
||||||
|
checks. All four large-prefix snapshots match #110's pre-port/oracle fixtures.
|
||||||
|
The 24 release before/after state comparisons and four explicit hidden-state
|
||||||
|
comparisons are exact. The direct pinned oracle matches all four representative
|
||||||
|
release states and all four hidden-prefill states, including every cache bit.
|
||||||
|
The off-switch matches all four baseline states with zero kernel activation.
|
||||||
|
|
||||||
|
DeepSeek's `resident_multi_session_switching_preserves_each_kv_frontier` and
|
||||||
|
GLM's `glm53_two_row_verifier_restores_recurrent_state` both pass on the preserved
|
||||||
|
before binary and the final after binary with installed models. Source review
|
||||||
|
confirms no DS4/GLM model, sampling, cache, shader, or orchestration path changed.
|
||||||
|
Their DS4 oracle behavior remains assigned to those families; this is the Qwen
|
||||||
|
MTPLX route port, not a new DS4Server-only feature. Non-NAX predicates are tested;
|
||||||
|
no physical M1–M4 run is claimed.
|
||||||
|
|
||||||
|
All required commit gates pass: formatting, warning-free Clippy, signed app
|
||||||
|
bundle, and all-features tests (312 application + 8 tool tests; 234 explicitly
|
||||||
|
ignored hardware/reference tests). Relevant ignored tests above were run
|
||||||
|
separately under the resource supervisor. Machine-readable evidence, exact
|
||||||
|
state fingerprints, gate logs, resource summaries and binary/source provenance
|
||||||
|
are stored beside this file. No acceptance item is deferred to a stub.
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
|
||||||
|
running 1 test
|
||||||
|
test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
|
||||||
|
ds4: Metal 4 tensor API enabled for Tensor kernels
|
||||||
|
ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
|
||||||
|
ds4: Metal model views created in 2.169 ms, residency requested in 7304.599 ms, warmup 5.656 ms (mapped 82697.67 MiB from offset 5.09 MiB)
|
||||||
|
ds4: Metal mapped mmaped model as 2 overlapping shared buffers
|
||||||
|
ok
|
||||||
|
|
||||||
|
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 539 filtered out; finished in 7.89s
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
ds4: GLM MTP draft 5733 at position 12 in 2.3 ms
|
||||||
|
verify2 scalar boundary cap=2 cancelled=true passed
|
||||||
|
verify2 prefill 0
|
||||||
|
verify2 prefill 4
|
||||||
|
verify2 prefill 8
|
||||||
|
verify2 prefill 12
|
||||||
|
ds4: GLM MTP draft 5733 at position 12 in 2.4 ms
|
||||||
|
engine-lifetime counters across new and restored chats passed
|
||||||
|
ok
|
||||||
|
|
||||||
|
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 539 filtered out; finished in 9.18s
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
cargo build --release --all-targets --all-features
|
||||||
|
Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
|
||||||
|
Finished `release` profile [optimized] target(s) in 29.05s
|
||||||
|
warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
|
||||||
|
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
|
||||||
|
cargo packager --release --formats app
|
||||||
|
codesign --force --deep --sign - --timestamp=none target/release/DS4Server.app
|
||||||
|
target/release/DS4Server.app: replacing existing signature
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
Checking ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
|
||||||
|
Finished `dev` profile [unoptimized + debuginfo] target(s) in 5.14s
|
||||||
|
warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
|
||||||
|
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
|
||||||
|
running 1 test
|
||||||
|
test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
|
||||||
|
ds4: Metal 4 tensor API enabled for Tensor kernels
|
||||||
|
ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
|
||||||
|
ds4: Metal model views created in 1.781 ms, residency requested in 8958.289 ms, warmup 2.850 ms (mapped 82697.67 MiB from offset 5.09 MiB)
|
||||||
|
ds4: Metal mapped mmaped model as 2 overlapping shared buffers
|
||||||
|
ok
|
||||||
|
|
||||||
|
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 545 filtered out; finished in 9.53s
|
||||||
Whitespace-only changes.
@@ -0,0 +1 @@
|
|||||||
|
{"metrics":{"chunk":2048,"rep":0,"phase":"cold","prompt":4096,"cached":0,"ttft_seconds":3.126132542,"prefill_seconds":3.1225067920000003,"prefill_tokens_per_second":1311.766562203846,"elapsed_seconds":3.147092209},"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,Line truncated
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"metrics":{"chunk":2048,"rep":0,"phase":"warm","prompt":6145,"cached":4097,"ttft_seconds":1.262224125,"prefill_seconds":1.258582084,"prefill_tokens_per_second":1627.2279941337542,"elapsed_seconds":1.282883709},"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374Line truncated
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"metrics":{"chunk":4096,"rep":0,"phase":"cold","prompt":4096,"cached":0,"ttft_seconds":2.955218083,"prefill_seconds":2.9541866239999997,"prefill_tokens_per_second":1386.50685326507,"elapsed_seconds":2.9757388750000002},"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11Line truncated
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"metrics":{"chunk":4096,"rep":0,"phase":"warm","prompt":6145,"cached":4097,"ttft_seconds":1.606924,"prefill_seconds":1.6059842079999997,"prefill_tokens_per_second":1275.2304722538095,"elapsed_seconds":1.62792175},"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,Line truncated
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
ds4: GLM MTP draft 5733 at position 12 in 2.3 ms
|
||||||
|
verify2 scalar boundary cap=2 cancelled=true passed
|
||||||
|
verify2 prefill 0
|
||||||
|
verify2 prefill 4
|
||||||
|
verify2 prefill 8
|
||||||
|
verify2 prefill 12
|
||||||
|
ds4: GLM MTP draft 5733 at position 12 in 2.4 ms
|
||||||
|
engine-lifetime counters across new and restored chats passed
|
||||||
|
ok
|
||||||
|
|
||||||
|
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 545 filtered out; finished in 11.86s
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"case":"hidden_prefill","chunk":2048,"phase":"cold","cached":0,"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,2Line truncated
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"case":"hidden_prefill","chunk":2048,"phase":"warm","cached":4096,"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,37Line truncated
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"case":"hidden_prefill","chunk":4096,"phase":"cold","cached":0,"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,2Line truncated
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"case":"hidden_prefill","chunk":4096,"phase":"warm","cached":4096,"state":{"ids":[9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,374,264,1092,13,9707,11,1075,37Line truncated
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,19 @@
|
|||||||
|
{
|
||||||
|
"baseline": "8965e77ea173c932e00f70759d0a83ac1c5072f0",
|
||||||
|
"mtplx": "9882703f3105363ddc37eca9f97aa09a1d387112",
|
||||||
|
"mlx": "0.32.2",
|
||||||
|
"artifact": "74559cdf34fbfc0b593de72d17e93f37fd4f9ea7",
|
||||||
|
"sha256": {
|
||||||
|
"metal/mtplx-moe-sorted-body.metal": "b639a135d2b9532d45c9587d94a503a093553078d8f5b5a9470c34d1bd9fca2e",
|
||||||
|
"metal/mtplx-moe-sorted-header.metal": "d3251a6cfcdbac5372c3e7e1f97720df48a91f0e456a44883c42b98cc8520e4a",
|
||||||
|
"metal/mtplx-runtime-0.32.2.metallib": "dc59d1cceb1a5c7e578232e6e41e28e2c73c9463ac6dbc3886c3ee17ffc270ed",
|
||||||
|
"/private/tmp/ds4-issue112-before-release-test": "9718805d3827bf5e998e2f574b48b7e453a45ed3de578ea9df6a450019aea0d6",
|
||||||
|
"/private/tmp/ds4-issue112-after-release-test": "9bcc33ddebe79fa026d7cb4a7c2a4dfcdba2645f66861a1e74778cf55e434912",
|
||||||
|
"/private/tmp/ds4-issue112-final-test": "f313f1ae0a43a05d359ce234977f00dba87ef8d1320034acf856592f2e2eba72"
|
||||||
|
},
|
||||||
|
"binary_scope": {
|
||||||
|
"before-release-test": "Full-chunk timing, hidden-state and family controls. The earlier default-tail run used the same baseline production source before the hidden-only test configuration was corrected.",
|
||||||
|
"after-release-test": "Default-tail and full-chunk timings; production identical to final-test.",
|
||||||
|
"final-test": "Final kernel, model, hidden-state, off-switch and family checks; corrected hidden-only test configuration."
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,838 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"configuration": "default-tail",
|
||||||
|
"arm": "before",
|
||||||
|
"records": [
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.7007944999999998,
|
||||||
|
"prefill_seconds": 3.6969075010000005,
|
||||||
|
"prefill_tokens_per_second": 1107.9530658779117,
|
||||||
|
"elapsed_seconds": 3.721188
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.7724733750000001,
|
||||||
|
"prefill_seconds": 1.769735542,
|
||||||
|
"prefill_tokens_per_second": 1157.23505088536,
|
||||||
|
"elapsed_seconds": 1.793335125
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.820473209,
|
||||||
|
"prefill_seconds": 2.81689246,
|
||||||
|
"prefill_tokens_per_second": 1454.0846191906098,
|
||||||
|
"elapsed_seconds": 2.841268834
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.838782375,
|
||||||
|
"prefill_seconds": 1.836070168,
|
||||||
|
"prefill_tokens_per_second": 1115.4257803942492,
|
||||||
|
"elapsed_seconds": 1.8597940419999999
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.959423792,
|
||||||
|
"prefill_seconds": 2.9556270000000002,
|
||||||
|
"prefill_tokens_per_second": 1385.831162051233,
|
||||||
|
"elapsed_seconds": 2.98031925
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.922527375,
|
||||||
|
"prefill_seconds": 1.919643709,
|
||||||
|
"prefill_tokens_per_second": 1066.8646428492007,
|
||||||
|
"elapsed_seconds": 1.9441567499999999
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.626180458,
|
||||||
|
"prefill_seconds": 3.6201352510000007,
|
||||||
|
"prefill_tokens_per_second": 1131.4494393181994,
|
||||||
|
"elapsed_seconds": 3.64758525
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.994229042,
|
||||||
|
"prefill_seconds": 1.9914031639999998,
|
||||||
|
"prefill_tokens_per_second": 1028.4205815392588,
|
||||||
|
"elapsed_seconds": 2.016047875
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.6903119589999998,
|
||||||
|
"prefill_seconds": 3.6843170420000004,
|
||||||
|
"prefill_tokens_per_second": 1111.7392866322168,
|
||||||
|
"elapsed_seconds": 3.711886
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 2.011648875,
|
||||||
|
"prefill_seconds": 2.0087712079999998,
|
||||||
|
"prefill_tokens_per_second": 1019.5287506331085,
|
||||||
|
"elapsed_seconds": 2.033881333
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.752376958,
|
||||||
|
"prefill_seconds": 3.7459722089999996,
|
||||||
|
"prefill_tokens_per_second": 1093.4411072669013,
|
||||||
|
"elapsed_seconds": 3.774499291
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 2.020803542,
|
||||||
|
"prefill_seconds": 2.017895623,
|
||||||
|
"prefill_tokens_per_second": 1014.918698795354,
|
||||||
|
"elapsed_seconds": 2.043202292
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"summary": [
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"phase": "cold",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 3.1602305003333333,
|
||||||
|
"median": 2.959423792,
|
||||||
|
"sample_stddev": 0.47326936772912714
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1315.9562823732515,
|
||||||
|
"median": 1385.831162051233,
|
||||||
|
"sample_stddev": 183.340222420435
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"phase": "warm",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 1.844594375,
|
||||||
|
"median": 1.838782375,
|
||||||
|
"sample_stddev": 0.07519564639658331
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1113.1751580429366,
|
||||||
|
"median": 1115.4257803942492,
|
||||||
|
"sample_stddev": 45.227222310038556
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"phase": "cold",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 3.689623125,
|
||||||
|
"median": 3.6903119589999998,
|
||||||
|
"sample_stddev": 0.06310106989799923
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1112.2099444057726,
|
||||||
|
"median": 1111.7392866322168,
|
||||||
|
"sample_stddev": 19.00853664502584
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"phase": "warm",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 2.0088938196666666,
|
||||||
|
"median": 2.011648875,
|
||||||
|
"sample_stddev": 0.013499768886162316
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1020.9560103225738,
|
||||||
|
"median": 1019.5287506331085,
|
||||||
|
"sample_stddev": 6.863163415905101
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"resources": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 41097,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 19599062416,
|
||||||
|
"resident_bytes": 25489604608,
|
||||||
|
"physical_bytes": 84724073856,
|
||||||
|
"peak_physical_bytes": 84724139512,
|
||||||
|
"disk_read_bytes": 64097386496,
|
||||||
|
"disk_write_bytes": 442368
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default-tail",
|
||||||
|
"arm": "after",
|
||||||
|
"records": [
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.699041667,
|
||||||
|
"prefill_seconds": 3.6953074989999997,
|
||||||
|
"prefill_tokens_per_second": 1108.4327897227588,
|
||||||
|
"elapsed_seconds": 3.719328292
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.776398,
|
||||||
|
"prefill_seconds": 1.7737045,
|
||||||
|
"prefill_tokens_per_second": 1154.6455455235075,
|
||||||
|
"elapsed_seconds": 1.797202167
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.80434375,
|
||||||
|
"prefill_seconds": 2.8008109150000005,
|
||||||
|
"prefill_tokens_per_second": 1462.4336038050606,
|
||||||
|
"elapsed_seconds": 2.825135334
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.900156542,
|
||||||
|
"prefill_seconds": 1.897415875,
|
||||||
|
"prefill_tokens_per_second": 1079.3627411808177,
|
||||||
|
"elapsed_seconds": 1.9216478750000001
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.952652333,
|
||||||
|
"prefill_seconds": 2.948979292,
|
||||||
|
"prefill_tokens_per_second": 1388.9551585226932,
|
||||||
|
"elapsed_seconds": 2.973752041
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.947212708,
|
||||||
|
"prefill_seconds": 1.944330169,
|
||||||
|
"prefill_tokens_per_second": 1053.3190466582735,
|
||||||
|
"elapsed_seconds": 1.9688718330000001
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.637618958,
|
||||||
|
"prefill_seconds": 3.631528458,
|
||||||
|
"prefill_tokens_per_second": 1127.8997390139687,
|
||||||
|
"elapsed_seconds": 3.659774833
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.9611875410000001,
|
||||||
|
"prefill_seconds": 1.958409373,
|
||||||
|
"prefill_tokens_per_second": 1045.7466289914453,
|
||||||
|
"elapsed_seconds": 1.982932208
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.6904271250000003,
|
||||||
|
"prefill_seconds": 3.684310291,
|
||||||
|
"prefill_tokens_per_second": 1111.7413237440048,
|
||||||
|
"elapsed_seconds": 3.71194925
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.988992875,
|
||||||
|
"prefill_seconds": 1.98611425,
|
||||||
|
"prefill_tokens_per_second": 1031.159209496634,
|
||||||
|
"elapsed_seconds": 2.010877
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.684088083,
|
||||||
|
"prefill_seconds": 3.6781829150000003,
|
||||||
|
"prefill_tokens_per_second": 1113.5933406944227,
|
||||||
|
"elapsed_seconds": 3.705752166
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.998740333,
|
||||||
|
"prefill_seconds": 1.9958163330000003,
|
||||||
|
"prefill_tokens_per_second": 1026.1465276825147,
|
||||||
|
"elapsed_seconds": 2.021001667
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"summary": [
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"phase": "cold",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 3.1520125833333332,
|
||||||
|
"median": 2.952652333,
|
||||||
|
"sample_stddev": 0.4795096169325091
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1319.940517350171,
|
||||||
|
"median": 1388.9551585226932,
|
||||||
|
"sample_stddev": 186.81918963228082
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"phase": "warm",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 1.8745890833333332,
|
||||||
|
"median": 1.900156542,
|
||||||
|
"sample_stddev": 0.0882308751191163
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1095.775777787533,
|
||||||
|
"median": 1079.3627411808177,
|
||||||
|
"sample_stddev": 52.619441949355554
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"phase": "cold",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 3.6707113886666667,
|
||||||
|
"median": 3.684088083,
|
||||||
|
"sample_stddev": 0.02883361907462447
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1117.7448011507988,
|
||||||
|
"median": 1113.5933406944227,
|
||||||
|
"sample_stddev": 8.843051732845977
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"phase": "warm",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 1.9829735830000002,
|
||||||
|
"median": 1.988992875,
|
||||||
|
"sample_stddev": 0.019486583946006573
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1034.3507887235314,
|
||||||
|
"median": 1031.159209496634,
|
||||||
|
"sample_stddev": 10.18236840333609
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"resources": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 40871,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 19458778041,
|
||||||
|
"resident_bytes": 25328418816,
|
||||||
|
"physical_bytes": 84668646808,
|
||||||
|
"peak_physical_bytes": 84668761376,
|
||||||
|
"disk_read_bytes": 69973549056,
|
||||||
|
"disk_write_bytes": 18446744073709289472
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full-chunks",
|
||||||
|
"arm": "before",
|
||||||
|
"records": [
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.182152,
|
||||||
|
"prefill_seconds": 3.17999279,
|
||||||
|
"prefill_tokens_per_second": 1288.0532348628376,
|
||||||
|
"elapsed_seconds": 3.202144875
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.219195625,
|
||||||
|
"prefill_seconds": 1.21829175,
|
||||||
|
"prefill_tokens_per_second": 1681.042328325707,
|
||||||
|
"elapsed_seconds": 1.240087208
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.317122125,
|
||||||
|
"prefill_seconds": 2.3159108330000002,
|
||||||
|
"prefill_tokens_per_second": 1768.6345871503593,
|
||||||
|
"elapsed_seconds": 2.338011208
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.26216725,
|
||||||
|
"prefill_seconds": 1.261251084,
|
||||||
|
"prefill_tokens_per_second": 1623.7845310743853,
|
||||||
|
"elapsed_seconds": 1.283571459
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.411353459,
|
||||||
|
"prefill_seconds": 2.410105791,
|
||||||
|
"prefill_tokens_per_second": 1699.510459373026,
|
||||||
|
"elapsed_seconds": 2.431755125
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.34331275,
|
||||||
|
"prefill_seconds": 1.34238875,
|
||||||
|
"prefill_tokens_per_second": 1525.6385305672443,
|
||||||
|
"elapsed_seconds": 1.364779
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.2133175,
|
||||||
|
"prefill_seconds": 3.212275166,
|
||||||
|
"prefill_tokens_per_second": 1275.1086965879188,
|
||||||
|
"elapsed_seconds": 3.233724875
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.7273878329999999,
|
||||||
|
"prefill_seconds": 1.7264041659999998,
|
||||||
|
"prefill_tokens_per_second": 1186.2807332915115,
|
||||||
|
"elapsed_seconds": 1.7481919160000001
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.980801042,
|
||||||
|
"prefill_seconds": 2.979894292,
|
||||||
|
"prefill_tokens_per_second": 1374.5454028340412,
|
||||||
|
"elapsed_seconds": 3.001205167
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.7668409999999999,
|
||||||
|
"prefill_seconds": 1.765948334,
|
||||||
|
"prefill_tokens_per_second": 1159.7168278197203,
|
||||||
|
"elapsed_seconds": 1.787861709
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.100441792,
|
||||||
|
"prefill_seconds": 3.0994622499999998,
|
||||||
|
"prefill_tokens_per_second": 1321.5195635952657,
|
||||||
|
"elapsed_seconds": 3.121023875
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.838995291,
|
||||||
|
"prefill_seconds": 1.838062583,
|
||||||
|
"prefill_tokens_per_second": 1114.216686059378,
|
||||||
|
"elapsed_seconds": 1.860065208
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"summary": [
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"phase": "cold",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 2.6368758613333334,
|
||||||
|
"median": 2.411353459,
|
||||||
|
"sample_stddev": 0.4745676312458291
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1585.3994271287409,
|
||||||
|
"median": 1699.510459373026,
|
||||||
|
"sample_stddev": 259.81840735698063
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"phase": "warm",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 1.274891875,
|
||||||
|
"median": 1.26216725,
|
||||||
|
"sample_stddev": 0.06302937601316841
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1610.1551299891123,
|
||||||
|
"median": 1623.7845310743853,
|
||||||
|
"sample_stddev": 78.59329182503896
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"phase": "cold",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 3.098186778,
|
||||||
|
"median": 3.100441792,
|
||||||
|
"sample_stddev": 0.11627463019215153
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1323.7245543390752,
|
||||||
|
"median": 1321.5195635952657,
|
||||||
|
"sample_stddev": 49.755011058230345
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"phase": "warm",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 1.7777413746666666,
|
||||||
|
"median": 1.7668409999999999,
|
||||||
|
"sample_stddev": 0.05659655286508827
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1153.4047490568698,
|
||||||
|
"median": 1159.7168278197203,
|
||||||
|
"sample_stddev": 36.444320265304924
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"resources": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 35856,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 21540742125,
|
||||||
|
"resident_bytes": 25113673728,
|
||||||
|
"physical_bytes": 88578784936,
|
||||||
|
"peak_physical_bytes": 88578866928,
|
||||||
|
"disk_read_bytes": 69525413888,
|
||||||
|
"disk_write_bytes": 524288
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full-chunks",
|
||||||
|
"arm": "after",
|
||||||
|
"records": [
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.126132542,
|
||||||
|
"prefill_seconds": 3.1225067920000003,
|
||||||
|
"prefill_tokens_per_second": 1311.766562203846,
|
||||||
|
"elapsed_seconds": 3.147092209
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.262224125,
|
||||||
|
"prefill_seconds": 1.258582084,
|
||||||
|
"prefill_tokens_per_second": 1627.2279941337542,
|
||||||
|
"elapsed_seconds": 1.282883709
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.304937333,
|
||||||
|
"prefill_seconds": 2.303636208,
|
||||||
|
"prefill_tokens_per_second": 1778.058525810426,
|
||||||
|
"elapsed_seconds": 2.325319833
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.333947917,
|
||||||
|
"prefill_seconds": 1.3330325,
|
||||||
|
"prefill_tokens_per_second": 1536.346638210246,
|
||||||
|
"elapsed_seconds": 1.354474625
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.504060958,
|
||||||
|
"prefill_seconds": 2.502749457,
|
||||||
|
"prefill_tokens_per_second": 1636.6000953646394,
|
||||||
|
"elapsed_seconds": 2.524487958
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.363442375,
|
||||||
|
"prefill_seconds": 1.362460209,
|
||||||
|
"prefill_tokens_per_second": 1503.1631650388993,
|
||||||
|
"elapsed_seconds": 1.3840412500000001
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.955218083,
|
||||||
|
"prefill_seconds": 2.9541866239999997,
|
||||||
|
"prefill_tokens_per_second": 1386.50685326507,
|
||||||
|
"elapsed_seconds": 2.9757388750000002
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 0,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.606924,
|
||||||
|
"prefill_seconds": 1.6059842079999997,
|
||||||
|
"prefill_tokens_per_second": 1275.2304722538095,
|
||||||
|
"elapsed_seconds": 1.62792175
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 3.209116167,
|
||||||
|
"prefill_seconds": 3.2081535000000003,
|
||||||
|
"prefill_tokens_per_second": 1276.7468888256126,
|
||||||
|
"elapsed_seconds": 3.230131833
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 1,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.767990417,
|
||||||
|
"prefill_seconds": 1.767019791,
|
||||||
|
"prefill_tokens_per_second": 1159.0136174088839,
|
||||||
|
"elapsed_seconds": 1.789304875
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "cold",
|
||||||
|
"prompt": 4096,
|
||||||
|
"cached": 0,
|
||||||
|
"ttft_seconds": 2.998665042,
|
||||||
|
"prefill_seconds": 2.997745792,
|
||||||
|
"prefill_tokens_per_second": 1366.3600198959098,
|
||||||
|
"elapsed_seconds": 3.019893375
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"rep": 2,
|
||||||
|
"phase": "warm",
|
||||||
|
"prompt": 6145,
|
||||||
|
"cached": 4097,
|
||||||
|
"ttft_seconds": 1.633976833,
|
||||||
|
"prefill_seconds": 1.633039209,
|
||||||
|
"prefill_tokens_per_second": 1254.1033850951462,
|
||||||
|
"elapsed_seconds": 1.656772416
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"summary": [
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"phase": "cold",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 2.645043611,
|
||||||
|
"median": 2.504060958,
|
||||||
|
"sample_stddev": 0.4283660515917751
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1575.475061126304,
|
||||||
|
"median": 1636.6000953646394,
|
||||||
|
"sample_stddev": 239.08001001547228
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 2048,
|
||||||
|
"phase": "warm",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 1.3198714723333334,
|
||||||
|
"median": 1.333947917,
|
||||||
|
"sample_stddev": 0.052056635063226324
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1555.5792657942998,
|
||||||
|
"median": 1536.346638210246,
|
||||||
|
"sample_stddev": 64.22959541694213
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"phase": "cold",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 3.054333097333333,
|
||||||
|
"median": 2.998665042,
|
||||||
|
"sample_stddev": 0.1357949135761868
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1343.2045873288641,
|
||||||
|
"median": 1366.3600198959098,
|
||||||
|
"sample_stddev": 58.42895678910406
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"chunk": 4096,
|
||||||
|
"phase": "warm",
|
||||||
|
"ttft_seconds": {
|
||||||
|
"mean": 1.6696304166666667,
|
||||||
|
"median": 1.633976833,
|
||||||
|
"sample_stddev": 0.0862495286509525
|
||||||
|
},
|
||||||
|
"prefill_tokens_per_second": {
|
||||||
|
"mean": 1229.4491582526132,
|
||||||
|
"median": 1254.1033850951462,
|
||||||
|
"sample_stddev": 61.90688583198103
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"resources": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 34614,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 20995139625,
|
||||||
|
"resident_bytes": 25576030208,
|
||||||
|
"physical_bytes": 87952375368,
|
||||||
|
"peak_physical_bytes": 87952457288,
|
||||||
|
"disk_read_bytes": 64869085184,
|
||||||
|
"disk_write_bytes": 442368
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
{
|
||||||
|
"final-variants": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 440,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 245994500,
|
||||||
|
"resident_bytes": 281100288,
|
||||||
|
"physical_bytes": 399917944,
|
||||||
|
"peak_physical_bytes": 419464080,
|
||||||
|
"disk_read_bytes": 0,
|
||||||
|
"disk_write_bytes": 0
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"final-failures": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 219,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 53213250,
|
||||||
|
"resident_bytes": 70057984,
|
||||||
|
"physical_bytes": 177390336,
|
||||||
|
"peak_physical_bytes": 177636096,
|
||||||
|
"disk_read_bytes": 0,
|
||||||
|
"disk_write_bytes": 0
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"final-installed": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 760,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 504222750,
|
||||||
|
"resident_bytes": 2076639232,
|
||||||
|
"physical_bytes": 4867475808,
|
||||||
|
"peak_physical_bytes": 5012703704,
|
||||||
|
"disk_read_bytes": 0,
|
||||||
|
"disk_write_bytes": 0
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"final-variants-oracle": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 762,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 562100250,
|
||||||
|
"resident_bytes": 142884864,
|
||||||
|
"physical_bytes": 359269216,
|
||||||
|
"peak_physical_bytes": 387105656,
|
||||||
|
"disk_read_bytes": 0,
|
||||||
|
"disk_write_bytes": 0
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"final-installed-oracle": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 1200,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 959967166,
|
||||||
|
"resident_bytes": 2091220992,
|
||||||
|
"physical_bytes": 3261778056,
|
||||||
|
"peak_physical_bytes": 4137257216,
|
||||||
|
"disk_read_bytes": 421888,
|
||||||
|
"disk_write_bytes": 0
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"after-hidden": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 19796,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 16081049958,
|
||||||
|
"resident_bytes": 25542393856,
|
||||||
|
"physical_bytes": 87610048312,
|
||||||
|
"peak_physical_bytes": 87611588312,
|
||||||
|
"disk_read_bytes": 65141735424,
|
||||||
|
"disk_write_bytes": 147456
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"final-model": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 29057,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 23619215916,
|
||||||
|
"resident_bytes": 25820266496,
|
||||||
|
"physical_bytes": 101435424688,
|
||||||
|
"peak_physical_bytes": 101440126416,
|
||||||
|
"disk_read_bytes": 71619133440,
|
||||||
|
"disk_write_bytes": 208896
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"off": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 18110,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 14743490958,
|
||||||
|
"resident_bytes": 25949437952,
|
||||||
|
"physical_bytes": 88457051624,
|
||||||
|
"peak_physical_bytes": 88460278864,
|
||||||
|
"disk_read_bytes": 66400653312,
|
||||||
|
"disk_write_bytes": 147456
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"final-model-oracle": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 19987,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 17635902708,
|
||||||
|
"resident_bytes": 25024905216,
|
||||||
|
"physical_bytes": 84086635888,
|
||||||
|
"peak_physical_bytes": 84086652296,
|
||||||
|
"disk_read_bytes": 103826788352,
|
||||||
|
"disk_write_bytes": 311296
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"hidden-oracle": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 19800,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 17457184041,
|
||||||
|
"resident_bytes": 25029591040,
|
||||||
|
"physical_bytes": 84092452352,
|
||||||
|
"peak_physical_bytes": 84092468760,
|
||||||
|
"disk_read_bytes": 103815700480,
|
||||||
|
"disk_write_bytes": 18446744073709027328
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"deepseek": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 9561,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 2757894458,
|
||||||
|
"resident_bytes": 130564096,
|
||||||
|
"physical_bytes": 307839816,
|
||||||
|
"peak_physical_bytes": 319898440,
|
||||||
|
"disk_read_bytes": 86739410944,
|
||||||
|
"disk_write_bytes": 0
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"glm": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 11928,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 3587549375,
|
||||||
|
"resident_bytes": 905478144,
|
||||||
|
"physical_bytes": 1335444992,
|
||||||
|
"peak_physical_bytes": 1488717432,
|
||||||
|
"disk_read_bytes": 96510627840,
|
||||||
|
"disk_write_bytes": 0
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"before-deepseek": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 7984,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 2510786166,
|
||||||
|
"resident_bytes": 130416640,
|
||||||
|
"physical_bytes": 301171504,
|
||||||
|
"peak_physical_bytes": 319750984,
|
||||||
|
"disk_read_bytes": 69381001216,
|
||||||
|
"disk_write_bytes": 0
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
},
|
||||||
|
"before-glm": {
|
||||||
|
"event": "test_resource_summary",
|
||||||
|
"elapsed_ms": 9244,
|
||||||
|
"peak": {
|
||||||
|
"cpu_time_ns": 2799064958,
|
||||||
|
"resident_bytes": 905216000,
|
||||||
|
"physical_bytes": 1335182800,
|
||||||
|
"peak_physical_bytes": 1488471648,
|
||||||
|
"disk_read_bytes": 68293218304,
|
||||||
|
"disk_write_bytes": 0
|
||||||
|
},
|
||||||
|
"error": null
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,218 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "2048-0-cold",
|
||||||
|
"before": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
|
||||||
|
"after": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "2048-0-warm",
|
||||||
|
"before": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
|
||||||
|
"after": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "2048-1-cold",
|
||||||
|
"before": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
|
||||||
|
"after": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "2048-1-warm",
|
||||||
|
"before": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
|
||||||
|
"after": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "2048-2-cold",
|
||||||
|
"before": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
|
||||||
|
"after": "a2703e87cc7c4ad2875ce10acfbf1f3d9f52f9a2e60ebb0deca96eabdbe1df8a",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "2048-2-warm",
|
||||||
|
"before": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
|
||||||
|
"after": "01419a6cc7e7df73215f36a5f0f69516cb19ed537e54d379ffa105f0b258b3b4",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "4096-0-cold",
|
||||||
|
"before": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
|
||||||
|
"after": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "4096-0-warm",
|
||||||
|
"before": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
|
||||||
|
"after": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "4096-1-cold",
|
||||||
|
"before": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
|
||||||
|
"after": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "4096-1-warm",
|
||||||
|
"before": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
|
||||||
|
"after": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "4096-2-cold",
|
||||||
|
"before": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
|
||||||
|
"after": "40261bef7a25aea30203d13bf91d34dc27c6ecfb2be56c777a1ed7e2defbb48f",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "default",
|
||||||
|
"case": "4096-2-warm",
|
||||||
|
"before": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
|
||||||
|
"after": "b4bf12137a534f71947b81f413a956e14d4bb42d493907f82ce5fb1be8fca3e8",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "2048-0-cold",
|
||||||
|
"before": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
|
||||||
|
"after": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "2048-0-warm",
|
||||||
|
"before": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
|
||||||
|
"after": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "2048-1-cold",
|
||||||
|
"before": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
|
||||||
|
"after": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "2048-1-warm",
|
||||||
|
"before": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
|
||||||
|
"after": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "2048-2-cold",
|
||||||
|
"before": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
|
||||||
|
"after": "8458c6beaa14c37b7adab9708f4d74d2d491726d5b80241398ecbdc37081b352",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "2048-2-warm",
|
||||||
|
"before": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
|
||||||
|
"after": "84b32d2e43daf4b57f6e6c58f5a9625227ce996e89e7f3a6f029df720892a891",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "4096-0-cold",
|
||||||
|
"before": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
|
||||||
|
"after": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "4096-0-warm",
|
||||||
|
"before": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
|
||||||
|
"after": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "4096-1-cold",
|
||||||
|
"before": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
|
||||||
|
"after": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "4096-1-warm",
|
||||||
|
"before": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
|
||||||
|
"after": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "4096-2-cold",
|
||||||
|
"before": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
|
||||||
|
"after": "466f9143f42f0520e3d51cde389e6cfca65a2396b1fcc9356bad3cf7fb3f3a8b",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "full",
|
||||||
|
"case": "4096-2-warm",
|
||||||
|
"before": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
|
||||||
|
"after": "2e0e79e826860dc131b0ac7b458981658a3b4af838e2b6b2953bd542f0fd609f",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "hidden",
|
||||||
|
"case": "2048-cold",
|
||||||
|
"before": "9616a339d8ec57e48fd4342d244b0fbeb86148722f1e6733f5ff4ea5c4862bf5",
|
||||||
|
"after": "9616a339d8ec57e48fd4342d244b0fbeb86148722f1e6733f5ff4ea5c4862bf5",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "hidden",
|
||||||
|
"case": "2048-warm",
|
||||||
|
"before": "fe0116cf8aebc195d3fb01970cb8be133d17750bf0284db1825747e6e21f0e0b",
|
||||||
|
"after": "fe0116cf8aebc195d3fb01970cb8be133d17750bf0284db1825747e6e21f0e0b",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "hidden",
|
||||||
|
"case": "4096-cold",
|
||||||
|
"before": "40d9efadffeeec8cc6e42d31241276a1fb727bac9fba3e0621343057e7588403",
|
||||||
|
"after": "40d9efadffeeec8cc6e42d31241276a1fb727bac9fba3e0621343057e7588403",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "hidden",
|
||||||
|
"case": "4096-warm",
|
||||||
|
"before": "2a40fec32283b8a2434181655260daa14fe9b3731b37e6a82c842a75df14d586",
|
||||||
|
"after": "2a40fec32283b8a2434181655260daa14fe9b3731b37e6a82c842a75df14d586",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "off",
|
||||||
|
"case": "2048-0-cold",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "off",
|
||||||
|
"case": "4096-0-warm",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "off",
|
||||||
|
"case": "4096-0-cold",
|
||||||
|
"exact": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"configuration": "off",
|
||||||
|
"case": "2048-0-warm",
|
||||||
|
"exact": true
|
||||||
|
}
|
||||||
|
]
|
||||||
@@ -0,0 +1,589 @@
|
|||||||
|
Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
|
||||||
|
Finished `test` profile [unoptimized + debuginfo] target(s) in 6.12s
|
||||||
|
warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
|
||||||
|
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
|
||||||
|
Running unittests src/main.rs (target/debug/deps/ds4_server-5679c54d8848b708)
|
||||||
|
|
||||||
|
running 546 tests
|
||||||
|
test agent::tests::anchored_edits_require_unique_head_and_tail ... ok
|
||||||
|
test agent::tests::ai_risk_response_requires_structured_risk_and_reason ... ok
|
||||||
|
test a2ui::tests::streaming_parser_only_accepts_complete_a2ui_jsonl_lines ... ok
|
||||||
|
test agent::tests::ai_shell_risk_assessment_receives_the_dev_brain_root ... ok
|
||||||
|
test a2ui::tests::embedded_catalog_and_validator_cover_the_same_components ... ok
|
||||||
|
test agent::tests::dev_brain_paths_are_trusted_by_shell_risk_assessment ... ok
|
||||||
|
test a2ui::tests::basic_formatters_follow_catalog_examples ... ok
|
||||||
|
test a2ui::tests::catalog_validation_rejects_invalid_variants_and_function_arguments ... ok
|
||||||
|
test a2ui::tests::catalog_accepts_pie_donut_and_heatmap_charts ... ok
|
||||||
|
test a2ui::tests::dismissal_boundaries_replay_history_and_a_fresh_active_epoch ... ok
|
||||||
|
test a2ui::tests::lifecycle_updates_incrementally_and_json_pointer_upserts ... ok
|
||||||
|
test a2ui::tests::latest_created_surface_is_the_active_surface ... ok
|
||||||
|
test agent::tests::present_svg_parsers_recover_exact_source_for_both_model_transports ... ok
|
||||||
|
test a2ui::tests::catalog_validation_and_actions_use_current_local_data ... ok
|
||||||
|
test agent::tests::ralph_loop_stops_on_cap_blocker_failure_and_cancellation ... ok
|
||||||
|
test a2ui::tests::all_basics_compose_into_one_surface ... ok
|
||||||
|
test a2ui::tests::v1_action_and_function_round_trips_follow_current_schema ... ok
|
||||||
|
test agent::tests::ralph_report_semantics_and_handoff_bound_are_enforced ... ok
|
||||||
|
test agent::tests::risky_shell_commands_require_one_time_approval ... ok
|
||||||
|
test agent::tests::generated_schemas_match_the_executable_tool_contracts ... ok
|
||||||
|
test agent::tests::model_adapters_preserve_primitives_and_report_recoverable_syntax_errors ... ok
|
||||||
|
test agent::tests::scripted_ralph_backend_gets_fresh_prompts_and_only_validated_handoffs ... ok
|
||||||
|
test agent::tests::repeated_call_advisories_use_canonical_arguments_and_reset_per_user_turn ... ok
|
||||||
|
test agent::tests::project_boundary_rejects_parent_paths ... ok
|
||||||
|
test a2ui::export::tests::static_svg_renderer_covers_the_complete_catalog ... ok
|
||||||
|
test agent::tests::ralph_report_schemas_are_child_only_and_parse_for_both_models ... ok
|
||||||
|
test agent::tests::stored_tool_cards_restore_ai_approval_reasons_by_call ... ok
|
||||||
|
test agent::tests::tool_output_keeps_terminal_lines_but_parameter_summaries_do_not ... ok
|
||||||
|
test agent::tests::present_svg_executes_without_side_effects_and_maps_only_completed_cards ... ok
|
||||||
|
test agent::web::tests::browser_helpers_preserve_ds4_wire_values ... ok
|
||||||
|
test agent::web::tests::headless_browser_extracts_a_local_page ... ignored, requires an installed Chrome browser
|
||||||
|
test agent::tests::prompts_and_parsers_expose_the_reference_tool_set ... ok
|
||||||
|
test agent::tests::validation_is_precise_and_prevents_side_effects_until_corrected ... ok
|
||||||
|
test app::generation::tests::attached_image_marker_is_model_only ... ok
|
||||||
|
test agent::tests::present_svg_validation_enforces_the_static_vector_boundary ... ok
|
||||||
|
test agent::tests::glob_confinement_skips_symlinks_and_honors_cancellation ... ok
|
||||||
|
test agent::tests::successful_structured_file_tools_report_project_paths ... ok
|
||||||
|
test agent::tests::glob_finds_all_regular_file_names_deterministically_without_reading_them ... ok
|
||||||
|
test agent::tests::standard_agent_skills_are_discovered_prompted_and_read_only ... ok
|
||||||
|
test agent::tests::shell_commands_skip_user_startup_files ... ok
|
||||||
|
test app::generation::tests::plain_chat_uses_the_same_unmodified_prompt_in_ui_and_headless ... ok
|
||||||
|
test app::generation::tests::queued_guidance_is_one_reference_style_user_turn ... ok
|
||||||
|
test app::generation::tests::title_context_starts_at_the_first_user_and_ignores_system_rows ... ok
|
||||||
|
test app::generation::tests::turn_summary_accumulates_model_continuations ... ok
|
||||||
|
test agent::tests::shell_environment_is_loaded_once_from_login_and_interactive_startup_files ... ok
|
||||||
|
test agent::tests::standard_file_tools_can_maintain_only_managed_dev_brain_pages ... ok
|
||||||
|
test app::git::tests::configured_git_diff_options_change_the_rendered_patch ... ok
|
||||||
|
test app::git::tests::unified_diff_parser_aligns_replacements_for_split_view ... ok
|
||||||
|
test agent::tests::ralph_round_contexts_are_unique_and_child_jobs_are_cleaned_up ... ok
|
||||||
|
test app::git::tests::unstage_works_before_the_first_commit ... ok
|
||||||
|
test app::preferences::tests::runtime_config_updates_after_lock_poisoning ... ok
|
||||||
|
test app::git::tests::libgit2_status_merges_staged_and_worktree_changes ... ok
|
||||||
|
test app::projects::tests::discarding_a_session_removes_every_checkpoint_stage ... ok
|
||||||
|
test app::projects::tests::draft_titles_follow_the_stored_session_count ... ok
|
||||||
|
test agent::tests::denial_and_stop_cancel_commands_awaiting_approval ... ok
|
||||||
|
test app::projects::tests::session_states_are_exclusive_and_ordered ... ok
|
||||||
|
test app::projects::tests::sweeping_removes_checkpoints_of_sessions_that_no_longer_exist ... ok
|
||||||
|
test app::projects::tests::git_state_lists_and_switches_local_branches ... ok
|
||||||
|
test app::tests::background_activity_drives_polling_indicators_and_quit_confirmation ... ok
|
||||||
|
test agent::tests::shell_environment_probe_stops_blocked_startup_files ... ok
|
||||||
|
test app::git::tests::origin_fetch_pull_and_push_use_the_current_branch ... ok
|
||||||
|
test app::tests::chat_html_export_embeds_loaded_a2ui_images_as_data_urls ... ok
|
||||||
|
test agent::tests::oversized_shell_output_is_bounded_and_retained_until_cleanup ... ok
|
||||||
|
test app::tests::chat_only_follows_updates_while_near_the_tail ... ok
|
||||||
|
test app::tests::ds4_gib_and_streaming_cache_inputs_are_typed ... ok
|
||||||
|
test app::tests::one_shot_titles_are_reduced_to_a_single_sidebar_line ... ok
|
||||||
|
test app::tests::owned_runtime_paths_stay_in_application_support ... ok
|
||||||
|
test app::tests::preferences_shortcut_and_dspark_support_are_explicit ... ok
|
||||||
|
test app::tests::source_does_not_depend_on_a_sibling_ds4_checkout ... ok
|
||||||
|
test app::tests::stored_session_always_selects_its_model ... ok
|
||||||
|
test app::tests::tab_scrolls_a_field_back_into_the_preferences_viewport ... ok
|
||||||
|
test app::view::a2ui::tests::renderer_input_helpers_cover_dates_numbers_choices_and_weights ... ok
|
||||||
|
test app::view::chat::tests::chat_divider_groups_token_counts_by_thousands ... ok
|
||||||
|
test app::view::chat::tests::composer_uses_shift_enter_for_line_breaks ... ok
|
||||||
|
test app::view::git::tests::split_diff_width_covers_the_longest_side_and_tabs ... ok
|
||||||
|
test app::tests::chat_export_preserves_completed_svg_and_keeps_failures_generic ... ok
|
||||||
|
test app::view::tests::action_buttons_share_geometry ... ok
|
||||||
|
test app::view::tests::context_pie_wedge_follows_the_fill_grade ... ok
|
||||||
|
test app::view::tests::download_measurements_are_readable ... ok
|
||||||
|
test app::view::tests::raised_surfaces_stay_dark ... ok
|
||||||
|
test compaction::tests::a_new_summary_replaces_the_previous_rebuild_summary ... ok
|
||||||
|
test compaction::tests::hard_trigger_and_retry_reserve_match_the_reference ... ok
|
||||||
|
test compaction::tests::soft_trigger_matches_reference_thresholds ... ok
|
||||||
|
test compaction::tests::summary_budget_keeps_answer_room_and_rejects_exhausted_contexts ... ok
|
||||||
|
test compaction::tests::summary_control_markup_is_private ... ok
|
||||||
|
test compaction::tests::tail_prefers_the_first_user_boundary_after_target ... ok
|
||||||
|
test config::tests::defaults_write_an_empty_file_and_reload_unchanged ... ok
|
||||||
|
test config::tests::deprecated_flash_config_migrates_and_persists_only_0731 ... ok
|
||||||
|
test config::tests::explicit_0731_dspark_opt_out_survives_the_default ... ok
|
||||||
|
test config::tests::invalid_files_are_reported_instead_of_reset ... ok
|
||||||
|
test config::tests::legacy_flat_model_settings_are_dropped_but_the_prompt_is_kept ... ok
|
||||||
|
test config::tests::model_profiles_expose_only_their_upstream_thinking_modes ... ok
|
||||||
|
test app::preferences::tests::selectors_rehydrate_their_generation_and_model_profiles ... ok
|
||||||
|
test config::tests::only_changed_values_are_written_and_read_back ... ok
|
||||||
|
test config::tests::qwen_selection_and_native_defaults_survive_restart ... ok
|
||||||
|
test app::preferences::tests::multiline_system_prompt_round_trips_through_the_editor ... ok
|
||||||
|
test app::generation::tests::legacy_generation_rows_collapse_into_their_user_turn ... ok
|
||||||
|
test app::generation::tests::model_history_uses_persisted_content_without_changing_the_transcript ... ok
|
||||||
|
test app::view::stats::tests::current_session_summary_uses_persisted_chat_facts ... ok
|
||||||
|
test app::generation::tests::assistant_markdown_keeps_llm_tables ... ok
|
||||||
|
test app::generation::tests::forced_compaction_requires_new_visible_chat ... ok
|
||||||
|
test app::generation::tests::last_compaction_selects_its_tail_without_hiding_history ... ok
|
||||||
|
test app::tests::assistant_stream_keeps_reasoning_separate_from_the_answer ... ok
|
||||||
|
test app::generation::tests::extension_context_deduplicates_session_start_and_rearms_after_compaction ... ok
|
||||||
|
test app::generation::tests::complete_tool_call_in_reasoning_gets_one_correction ... ok
|
||||||
|
test app::tests::chat_export_formats_visible_turns_reasoning_and_bash_tools ... ok
|
||||||
|
test app::tests::chat_html_export_is_safe_standalone_and_renders_embedded_surfaces ... ok
|
||||||
|
test app::generation::tests::a2ui_auto_switch_waits_for_a_renderable_surface ... ok
|
||||||
|
test dev_brain::tests::path_confinement_and_unrelated_notes_are_preserved ... ok
|
||||||
|
test database::tests::context_work_reactivates_an_archived_session ... ok
|
||||||
|
test dsml::tests::parses_reference_and_legacy_dsml_syntaxes ... ok
|
||||||
|
test dsml::tests::recovers_complete_invokes_before_an_incomplete_one ... ok
|
||||||
|
test dsml::tests::recovers_complete_invokes_without_the_outer_closing_tag ... ok
|
||||||
|
test engine::ds4_sampling::tests::heap_fallback_preserves_rng_and_signed_zero_ties ... ok
|
||||||
|
test engine::gguf::tests::accepts_exact_mxfp4_block_layout ... ok
|
||||||
|
test database::tests::a2ui_dismissal_persists_a_fresh_surface_boundary ... ok
|
||||||
|
test engine::gguf::tests::keeps_unknown_tensor_kinds_rejected ... ok
|
||||||
|
test engine::gguf::tests::mxfp4_scalar_dot_matches_independent_values ... ok
|
||||||
|
test engine::gguf::tests::maps_metadata_and_tensor_payload_without_copying_weights ... ok
|
||||||
|
test engine::gguf::tests::rejects_mxfp4_dimension_overflow ... ok
|
||||||
|
test database::tests::inline_svg_tool_call_survives_session_switch_compaction_and_reopen ... ok
|
||||||
|
test database::tests::sessions_are_ordered_by_recent_use_within_sidebar_groups ... ok
|
||||||
|
test engine::gguf::tests::rejects_mxfp4_rows_that_are_not_block_aligned ... ok
|
||||||
|
test engine::gguf::tests::rejects_truncated_mxfp4_payload ... ok
|
||||||
|
test engine::metal::glm::tests::dsa_indexer_schedule_matches_the_reference ... ok
|
||||||
|
test engine::metal::glm::tests::glm53_m5_text_performance_gate ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal
|
||||||
|
test engine::metal::glm::tests::glm53_m5_vision_performance_and_oracle_gate ... ignored, requires GLM 5.3 Q2, its vision sidecar, a fixed image, and Apple Metal
|
||||||
|
test engine::metal::glm::tests::glm53_reference_logits_replay_separates_sampling_from_execution ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal
|
||||||
|
test engine::metal::glm::tests::glm53_reference_prompt_tokens_match_shared_runtime ... ignored, installed GLM and DS4SERVER_GLM_REFERENCE_LOG from the standalone reference
|
||||||
|
test engine::metal::glm::tests::glm53_two_row_verifier_restores_recurrent_state ... ignored, installed GLM 5.3 and Metal; run under test-supervisor
|
||||||
|
test engine::metal::glm::tests::glm_byte_budget_reserves_prefill_before_dynamic_experts ... ok
|
||||||
|
test engine::metal::glm::tests::glm_mtp_preserves_target_tokens_and_drafts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
|
||||||
|
test engine::metal::glm::tests::glm_prefill_schedule_matches_ds4_boundaries ... ok
|
||||||
|
test engine::metal::glm::tests::glm_resident_decode_flushes_periodically_without_a_final_or_ssd_flush ... ok
|
||||||
|
test engine::metal::glm::tests::repeated_glm_prompt_rewinds_one_token_for_logits ... ok
|
||||||
|
test engine::metal::glm::tests::resident_and_streamed_glm_match_ds4_decode_oracles ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
|
||||||
|
test engine::metal::glm::tests::streamed_glm_uses_ds4_indexed_prefill_for_long_prompts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
|
||||||
|
test engine::metal::gpu::qwen_mtplx_allocation_tests::backend_allocation_policy_is_explicit_inside_and_outside_qwen_scope ... ignored, requires Apple Metal; allocation only, no GPU dispatch or model
|
||||||
|
test engine::metal::gpu::qwen_mtplx_allocation_tests::dispatch_retains_allocation_without_changing_data_donation ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::gpu::qwen_mtplx_allocation_tests::empty_storage_preserves_data_identity_views_and_zero_allocator_usage ... ignored, requires Apple Metal context; empty descriptors, no GPU allocation
|
||||||
|
test engine::metal::gpu::qwen_mtplx_allocation_tests::mtplx_tensor_cpu_access_releases_native_storage ... ignored, requires Apple Metal; CPU buffer access only, no GPU dispatch or model
|
||||||
|
test engine::metal::markov::tests::installed_markov_worker_dispatch ... ignored, requires installed 0731 DSpark support; CPU dispatch diagnostic, not parity acceptance
|
||||||
|
test engine::metal::markov::tests::markov_workers_reuse_threads_and_buffers_with_ordered_ties ... ok
|
||||||
|
test engine::metal::power_throttle_keeps_ds4_ewma_and_full_power_bypass ... ok
|
||||||
|
test engine::kvstore::tests::opening_reclaims_checkpoints_the_index_cannot_reach ... ok
|
||||||
|
test engine::metal::qwen::tests::affine_split_k_matches_reference_shapes_and_scratch_bound ... ok
|
||||||
|
test engine::metal::qwen::tests::dense_attention_crossover_is_phase_specific ... ok
|
||||||
|
test engine::kvstore::tests::budget_evicts_the_least_valuable_unprotected_entry ... ok
|
||||||
|
test engine::metal::qwen::tests::qwen_affine_qmv_and_qmm_match_mlx_reference ... ignored, requires the pinned Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_affine_qmv_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_core_boundary_and_checkpoint_are_stable ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_fused_gdn_input_rows_match_separate_projections ... ignored, requires the pinned Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_gdn_conv_norm_rows_matches_mtplx ... ignored, requires Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_gdn_delta_rows_matches_mtplx ... ignored, requires Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_lazy_target_probabilities_match_batched_sampling ... ignored, requires the pinned Qwen metadata and Apple Metal; no model inference
|
||||||
|
test dev_brain::tests::skill_index_migration_does_not_overwrite_an_unrelated_root_note ... ok
|
||||||
|
test engine::metal::qwen::tests::qwen_m5_performance_gate ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_metal_primitives_match_reference_vectors ... ignored, requires Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_mtp_matches_target_and_restores_its_cache ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_ple_gather_copies_only_the_requested_row ... ok
|
||||||
|
test database::tests::session_model_migration_preserves_data_and_accepts_catalog ... ok
|
||||||
|
test engine::metal::qwen::tests::qwen_qsa_dense_mask_accepts_unsorted_blocks ... ignored, requires Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_qsa_full_context_preallocation_succeeds ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qsa_selection_and_capacity_cover_the_full_native_window ... ok
|
||||||
|
test engine::metal::qwen::tests::qwen_qsa_logical_depth_oracles_cover_prefill_decode_and_state ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_qsa_real_layer_executes_the_first_sparse_token ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_qsa_selection_and_attention_remain_bounded_at_native_depths ... ignored, requires Apple Metal and allocates the full QSA score window
|
||||||
|
test engine::metal::qwen::tests::qwen_reference_layer_trace ... ignored, diagnostic trace against the pinned MLX-LM reference
|
||||||
|
test engine::metal::qwen::tests::qwen_reference_sdpa_trace ... ignored, diagnostic comparison with captured MTPLX SDPA inputs
|
||||||
|
test engine::metal::qwen::tests::qwen_reference_stateful_trace ... ignored, diagnostic trace against the pinned MLX-LM reference
|
||||||
|
test engine::metal::qwen::tests::qwen_reference_verify_trace ... ignored, diagnostic trace against the pinned MTPLX verifier
|
||||||
|
test engine::metal::qwen::tests::qwen_sorted_routed_prefill_matches_grouped_output ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_stage_counters_preserve_async_batches ... ignored, requires pinned Qwen embeddings and Apple Metal; stage profiler opt-in
|
||||||
|
test engine::metal::qwen::tests::qwen_target_submission_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen::tests::qwen_verify_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
|
||||||
|
test engine::metal::qwen_mtplx::allocator::mtplx_allocations_preserve_logical_bounds_with_reference_rounding ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_heap_limits_and_gpu_lifetime ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_replays_reference_allocations ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::ar::mtplx_ar_driver_matches_reference ... ignored, requires Apple Metal; unchanged MTPLX generate_ar orchestration oracle
|
||||||
|
test engine::metal::qwen_mtplx::array::mtplx_array_detach_clears_every_siblings_inputs_and_primitive ... ok
|
||||||
|
test engine::metal::qwen::tests::qwen_ple_hash_contract_matches_golden_boundaries ... ok
|
||||||
|
test engine::metal::qwen_mtplx::array::mtplx_array_materialization_keeps_geometry_and_rejects_short_storage ... ignored, requires Apple Metal; typed materialized descriptors without model weights
|
||||||
|
test engine::metal::qwen_mtplx::array::mtplx_array_layout_preserves_dtype_shape_stride_and_materialization_flags ... ok
|
||||||
|
test engine::metal::qwen_mtplx::array::mtplx_dtype_promotion_matches_all_reference_pairs ... ok
|
||||||
|
test engine::kvstore::tests::longest_compatible_prefix_wins_and_hits_survive_reopen ... ok
|
||||||
|
test engine::metal::qwen_mtplx::attention::mtplx_qsa_dense_mask_graph_matches_reference ... ignored, requires Apple Metal; original MTPLX block mask including sentinel and tails
|
||||||
|
test engine::metal::qwen_mtplx::attention::mtplx_qsa_eager_array_graph_matches_reference ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::attention::mtplx_qsa_prefill_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX tiled prefill gather
|
||||||
|
test engine::metal::qwen_mtplx::attention::mtplx_qsa_rows_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX QSA rows-gather attention
|
||||||
|
test engine::metal::qwen_mtplx::attention::mtplx_sdpa_fallback_graph_matches_reference ... ignored, requires Apple M5 Max; actual Qwen SDPA fallback reference
|
||||||
|
test engine::metal::qwen_mtplx::binary::mtplx_graph_binary_preserves_reference_branches ... ignored, requires Apple Metal; binary graph broadcast/layout and donation branches
|
||||||
|
test engine::metal::qwen_mtplx::binding::mtplx_installed_module_binding_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS and SSD policy; only small norms and four table rows evaluated
|
||||||
|
test engine::metal::qwen_mtplx::cache::mtplx_kv_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX KVCache growth, trim and state receipts
|
||||||
|
test engine::metal::qwen_mtplx::cache::mtplx_qsa_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX positional cache, mirror and held state aliases
|
||||||
|
test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_scalar_bits_and_custom_kernel_identity ... ignored, requires Metal buffers and external supervisor; scalar ownership only, no GPU dispatch
|
||||||
|
test engine::metal::qwen_mtplx::context_copy::prompt_copy_match_and_backoff_preserve_reference_policy ... ok
|
||||||
|
test engine::metal::qwen_mtplx::decoder::mtplx_decoder_connected_attention_and_moe_advance_caches ... ignored, requires Apple Metal and external test-supervisor; complete decoder/cache smoke
|
||||||
|
test engine::metal::qwen_mtplx::decoder::mtplx_decoder_residual_preserves_graph_order ... ok
|
||||||
|
test engine::metal::qwen_mtplx::dense::mtplx_dense_attention_batches_match_reference ... ignored, requires Apple M5 Max; actual BF16 batched matmul reference
|
||||||
|
test engine::metal::qwen_mtplx::dense::mtplx_dense_float_graph_matches_reference ... ignored, requires Apple M5 Max; floating Matmul factory and original dispatch
|
||||||
|
test engine::metal::qwen_mtplx::compiled::mtplx_replay_replaces_descriptors_without_rebuilding_primitives ... ok
|
||||||
|
test engine::metal::qwen_mtplx::dense::mtplx_qsa_score_array_graph_matches_reference ... ignored, requires Apple M5 Max; canonical QSA F32 graph with the original TF32 setting
|
||||||
|
test engine::metal::qwen_mtplx::encoder::mtplx_empty_bindings_preserve_nil_dependencies_without_dummy_storage ... ignored, requires Apple Metal; empty original-kernel grids and nil bindings
|
||||||
|
test engine::metal::qwen_mtplx::allocator::mtplx_allocator_cache_matches_live_reference_trace ... ok
|
||||||
|
test engine::metal::qwen_mtplx::encoder::mtplx_encoder_commits_after_primitives_and_balances_completion ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::encoder::mtplx_encoder_scope_routes_operators_and_allocator ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::encoder::mtplx_encoder_shared_events_and_error_completion_order ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::encoder::mtplx_encoder_untracked_dependencies_across_encoders_and_commands ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_drains_failure_preserves_cause_and_retries_only_failed_nodes ... ignored, requires Apple Metal; supervised evaluator failure and retry
|
||||||
|
test database::tests::migrations_and_crud_work ... ok
|
||||||
|
test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_releases_intermediates_and_preserves_descriptor_donation ... ignored, requires Apple Metal; graph ownership and last-consumer release
|
||||||
|
test engine::metal::qwen_mtplx::encoder::mtplx_encoder_barrier_epochs_and_commit_thresholds ... ok
|
||||||
|
test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_retains_tracer_topology_without_input_donation ... ignored, requires Apple Metal; evaluator tracer donation and retained topology
|
||||||
|
test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_runs_cpu_gpu_siblings_and_async_continuations ... ignored, requires Apple Metal; tiny supervised CPU/GPU evaluator graph
|
||||||
|
test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_scheduled_detach_and_tracer_errors ... ok
|
||||||
|
test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ar_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_AR_REFERENCE required
|
||||||
|
test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_tracks_stream_fences_per_leaf_and_cross_device_union ... ok
|
||||||
|
test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_bfs_width_and_sibling_choice ... ok
|
||||||
|
test engine::metal::qwen_mtplx::execution::mtplx_full_installed_mtp_prefill_matches_reference ... ignored, full installed MTP model; external guard and DS4_QWEN38_MTP_PREFILL_REFERENCE required
|
||||||
|
test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ping_forward_matches_reference ... ignored, full installed model; requires external memory guard and DS4_QWEN38_FORWARD_REFERENCE
|
||||||
|
test engine::metal::qwen_mtplx::execution::qwen_power_throttle_drains_before_sleep_and_bypasses_at_full_power ... ignored, requires Apple Metal; run under test-supervisor
|
||||||
|
test engine::metal::qwen_mtplx::fused::mtplx_compiled_fusion_gpu_matches_reference ... ignored, requires Metal and external supervisor; generated fusion vs pinned SiLU plus multi-output/scalar dispatch
|
||||||
|
test engine::metal::qwen_mtplx::fused::mtplx_compiled_constant_spelling ... ok
|
||||||
|
test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_forward_matches_model_reference ... ignored, requires Apple Metal; actual MTPLX GDN forward and cache receipts
|
||||||
|
test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_prefill_and_verify_match_model_reference ... ignored, requires Apple Metal; complete GDN Conv1d prefill and fused verify graphs
|
||||||
|
test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_staged_primitives_match_reference ... ignored, requires Apple Metal; 80 actual delta-update receipts and six conv-normalization kernels
|
||||||
|
test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_step_preserves_views_and_siblings ... ignored, requires Apple Metal; GDN custom primitive strided copies and sibling roots
|
||||||
|
test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_event_snapshots_preserve_values_streams_and_shared_errors ... ignored, requires Apple Metal events, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fence_coherence_preserves_empty_and_partial_word_spans ... ignored, requires Apple Metal3, model-free empty coherent dispatch
|
||||||
|
test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fences_transfer_cpu_gpu_gpu_cpu_in_both_modes ... ignored, requires Apple Metal3, tiny model-free coherent fence kernels
|
||||||
|
test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_forward_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX GatedResidual reference receipts
|
||||||
|
test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_pack_and_reads_match_reference ... ignored, requires Apple Metal; original hyper primitives and runtime pack
|
||||||
|
test engine::metal::qwen_mtplx::indexing::mtplx_graph_indexing_preserves_layouts_and_offsets ... ignored, requires Apple Metal; Select layout branches, GatherAxis views and empty arguments
|
||||||
|
test engine::metal::qwen_mtplx::indexing::mtplx_scatter_axis_graph_matches_reference ... ignored, requires Apple Metal; pinned Bool ScatterAxis layout and broadcast paths
|
||||||
|
test engine::metal::qwen_mtplx::issue103_analysis::issue103_numeric_receipts ... ignored, #103 diagnostic; Metal required, eager versus compiled numeric operands
|
||||||
|
test engine::metal::qwen_mtplx::issue103_analysis::issue103_sampling_receipts ... ignored, #103 diagnostic; Metal required, pins currently broken sampling law and invalid rows
|
||||||
|
test engine::metal::qwen_mtplx::issue103_analysis::issue103_sorted_gather_receipts ... ignored, #103 diagnostic; M5 and pinned runtime shader required; initialized sentinels
|
||||||
|
test engine::metal::qwen_mtplx::issue103_analysis::issue103_tokenizer_receipts ... ignored, #103 diagnostic; installed tokenizer and latest oracle JSONL; no GPU
|
||||||
|
test engine::metal::qwen_mtplx::issue103_analysis::issue103_vision_position_receipts ... ignored, #103 diagnostic; small active attention module on Metal; mixed position boundary
|
||||||
|
test engine::metal::qwen_mtplx::issue104::issue104_installed_ar_stream_and_state ... ignored, full pinned Bare-Speed model; external memory guard required
|
||||||
|
test engine::metal::qwen_mtplx::issue105::issue105_batched_greedy_rejects_each_invalid_row ... ignored, Metal required; run serially under test-supervisor
|
||||||
|
test engine::metal::qwen_mtplx::issue106::issue106_complete_state_and_checkpoint_recovery ... ignored, installed model and actual pre-fix checkpoint; external memory guard required
|
||||||
|
test engine::metal::qwen_mtplx::issue106::issue106_upstream_exit_matrix ... ignored, installed model, scripted logits on real caches; external memory guard required
|
||||||
|
test engine::metal::qwen_mtplx::issue108::issue108_attention_chunk_positions ... ignored, Metal and pinned MTPLX chunk fixtures; external supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::issue109::issue109_gate_phase_and_dtype_laws ... ignored, Metal and exhaustive pinned gate fixtures; external supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::issue110::issue110_actual_projection_rows_match_oracle ... ignored, installed layer weights and pinned oracle receipts; external memory supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::issue110::issue110_guard_allocation_refusal_releases_graph_and_retries ... ignored, Metal allocation refusal and retry; external supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::issue110::issue110_large_cold_and_warm_prefill ... ignored, full installed model and large partial chunks; serial memory supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::issue112_bench::issue112_hidden_prefill ... ignored, full installed model hidden-state prefill; serial supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::issue112_bench::issue112_paired_prefill ... ignored, full installed model; serial memory supervisor and explicit output directory
|
||||||
|
test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_dequantize_matches_reference ... ignored, requires Apple Metal; original table dequantize dtype/group/bit/layout receipts
|
||||||
|
test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_quantize_matches_reference ... ignored, requires Apple Metal; original affine quantize dtype/group/bit/layout receipts
|
||||||
|
test engine::metal::qwen_mtplx::linear::mtplx_graph_gather_qmm_matches_reference ... ignored, requires Apple Metal; original GatherQMM array layouts and expert routing
|
||||||
|
test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_batched_identity ... ignored, requires Apple Metal and external supervisor; strided/broadcast batched affine dispatch
|
||||||
|
test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_matches_reference ... ignored, requires Apple M5 Max Metal; 78 pinned affine fixtures through lazy graph
|
||||||
|
test engine::metal::qwen_mtplx::load::mtplx_safetensors_load_laziness_boundaries_and_lifetime ... ignored, requires Apple Metal; bounded CPU Load and dependent GPU graph
|
||||||
|
test engine::metal::qwen_mtplx::mlp::mtplx_graph_shared_mlp_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX shared MLP reference receipts
|
||||||
|
test engine::metal::qwen_mtplx::linear::sorted_gather_guard_matches_pinned_call_predicates ... ok
|
||||||
|
test engine::metal::qwen_mtplx::mlp::mtplx_graph_split_and_compiled_silu_layouts ... ignored, requires Apple Metal; Split sibling ownership and compiled SiLU layouts
|
||||||
|
test engine::metal::qwen_mtplx::moe::mtplx_graph_sparse_moe_matches_reference ... ignored, requires Apple M5 Max; full MoE Array graph, no model weights; use test-supervisor
|
||||||
|
test engine::metal::qwen_mtplx::model::mtplx_model_compile_policy_and_run_boundaries ... ok
|
||||||
|
test engine::metal::qwen_mtplx::moe_sorted::issue112_all_kernel_variants ... ignored, Metal synthetic all supported row-map variants; external supervisor
|
||||||
|
test engine::metal::qwen_mtplx::moe_sorted::issue112_canary_failures ... ignored, Metal canary rejection and sticky fallback; external supervisor
|
||||||
|
test engine::metal::qwen_mtplx::moe_sorted::issue112_full_model_state ... ignored, full installed model; validates all cold/warm state receipts against the pre-port oracle
|
||||||
|
test engine::metal::qwen_mtplx::moe_sorted::issue112_installed_gate_up ... ignored, installed projection weights; external supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::moe_sorted::row_map_applicability ... ok
|
||||||
|
test engine::metal::qwen_mtplx::ngram::mtplx_ngram_embedding_matches_reference ... ignored, requires Apple Metal; full resident NGram graph and lazy decode history
|
||||||
|
test engine::metal::qwen_mtplx::ngram::mtplx_ngram_gpu_rows_and_history_match_reference ... ignored, requires Apple Metal; original lazy NGram GPU row graph, no table
|
||||||
|
test engine::metal::qwen_mtplx::ngram::mtplx_ngram_resident_matches_reference ... ignored, requires Apple Metal; original NGramTable resident gather/dequantize receipts
|
||||||
|
test engine::metal::qwen_mtplx::ngram::mtplx_ngram_table_priority ... ignored, requires Apple Metal; NGram table routing and missing-sidecar behavior
|
||||||
|
test engine::metal::qwen_mtplx::ngram_sidecar::mtplx_ngram_sidecar_matches_reference ... ignored, requires Apple Metal; actual file-backed MTPLX sidecar/LRU/dequantization receipts
|
||||||
|
test database::tests::chat_and_compaction_history_survive_reopen_and_session_deletion ... ok
|
||||||
|
test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_allclose_branches ... ignored, requires Apple Metal; stage verification special values and original reduction branches
|
||||||
|
test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_matches_reference ... ignored, requires Apple Metal; original complete stage lifecycle and file-backed tables
|
||||||
|
test engine::metal::qwen_mtplx::normalization::mtplx_graph_norm_modules_match_reference ... ignored, requires Apple Metal; connected Qwen normalization graph golden fixtures
|
||||||
|
test engine::metal::qwen_mtplx::normalization::mtplx_graph_rms_norm_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph RMSNorm donation, copy, broadcast and looped kernels
|
||||||
|
test engine::metal::qwen_mtplx::normalization::mtplx_logsumexp_graph_matches_reference ... ignored, requires Apple Metal; original sampler LogSumExp branches
|
||||||
|
test engine::metal::qwen_mtplx::normalization::mtplx_softmax_graph_storage_matches_reference ... ignored, requires Apple Metal; original Softmax type and storage branches
|
||||||
|
test engine::metal::qwen_mtplx::ops::mtplx_concatenate_preserves_general_general_copy_semantics ... ignored, requires Apple Metal; Concatenate promotion, offsets, strided writers and validation
|
||||||
|
test engine::metal::qwen_mtplx::ops::mtplx_owned_operations_build_and_execute_qsa_layout_graph ... ignored, requires Apple Metal; self-contained lazy QSA-layout recipe, no callback map
|
||||||
|
test engine::metal::qwen_mtplx::ops::mtplx_slice_and_contiguous_preserve_cache_window_semantics ... ignored, requires Apple Metal; normalized Slice and Contiguous cache-window graph
|
||||||
|
test engine::metal::qwen_mtplx::owner::issue105_product::issue105_product_ar_cleanup_and_http_retry ... ignored, installed Qwen model; serial test-supervisor with memory guard required
|
||||||
|
test engine::metal::qwen_mtplx::owner::issue105_product::issue105_product_mtp_cleanup_and_http_retry ... ignored, installed Qwen MTP model; serial test-supervisor with memory guard required
|
||||||
|
test engine::metal::qwen_mtplx::owner::issue106_product::issue106_product_discards_old_checkpoint_and_recovers_cold ... ignored, installed model and actual pre-fix checkpoint; external memory guard required
|
||||||
|
test engine::metal::qwen_mtplx::owner::issue107_product::issue107_vision_product ... ignored, installed model plus vision tower; serial external memory supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::owner::issue109_product::issue109_verify_rows_and_warm_checkpoint ... ignored, installed model; serial external memory supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::fused::mtplx_compiled_emitter_matches_all_pinned_silu_bodies ... ok
|
||||||
|
test engine::metal::qwen_mtplx::owner::issue111_product::issue111_token_prefix_cache ... ignored, installed model; serial external memory supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::owner::qwen_product_generator_chat_matches_reference ... ignored, full installed model through product Generator; external supervisor and chat reference required
|
||||||
|
test engine::metal::qwen_mtplx::owner::qwen_vision_chat_checkpoint_preserves_image_identity ... ignored, requires installed Qwen, Metal, and DS4_QWEN_VISION_IMAGE; run under test-supervisor
|
||||||
|
test engine::metal::qwen_mtplx::fused::mtplx_compile_fusion_preserves_parent_and_output_boundaries ... ok
|
||||||
|
test engine::metal::qwen_mtplx::owner::qwen_mtp_progress_publishes_without_turn_completion_or_double_counting ... ok
|
||||||
|
test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_connected_matches_reference ... ignored, requires Apple M5 Max; complete original PLELayer with real lookup
|
||||||
|
test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_projection_and_cache_match_reference ... ignored, requires Apple M5 Max; actual MTPLX PLELayer after embedding lookup
|
||||||
|
test engine::metal::qwen_mtplx::policy::qwen_server_policy_matches_original_functions ... ignored, requires DS4_QWEN38_POLICY_REFERENCE from pinned MTPLX; CPU-only
|
||||||
|
test engine::metal::qwen_mtplx::prefill::mtplx_mtp_prefill_driver_matches_reference ... ignored, requires Apple Metal; original streaming MTP history prefill
|
||||||
|
test engine::metal::qwen_mtplx::policy::sorted_moe_switch_and_family_rehearsal ... ok
|
||||||
|
test engine::metal::qwen_mtplx::prefill::mtplx_prefill_driver_matches_reference ... ignored, requires Apple Metal; original _prefill control flow and raw cache roots
|
||||||
|
test engine::metal::qwen_mtplx::qsa_attention::mtplx_qsa_attention_module_matches_reference ... ignored, requires Metal and unchanged whole-Attention receipts; no model download
|
||||||
|
test engine::metal::qwen_mtplx::qsa_compile::mtplx_compiled_indexer_array_replay_matches_reference ... ignored, requires Metal and original compiled indexer receipts; no model weights
|
||||||
|
test engine::metal::qwen_mtplx::qsa_core::mtplx_qsa_core_host_matches_reference ... ignored, requires Metal and original host-core receipts; no model weights
|
||||||
|
test engine::metal::qwen_mtplx::qsa_graph::qsa_cache_donation_requires_descriptor_data_and_allocation_contract ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::qsa_graph::qsa_completion_owns_buffers_until_gpu_finishes ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::qsa_graph::qsa_evaluator_retains_inputs_and_siblings_not_primary_data ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::qsa_indexer::issue107::issue107_real_geometry_query_and_block_start_positions ... ignored, Metal and independently verified pinned indexer fixtures; external supervisor required
|
||||||
|
test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_canonical_indexer_routes_match_reference ... ignored, requires Metal and original ongoing indexer routing receipts; no weights
|
||||||
|
test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_compiled_commit_matches_reference ... ignored, requires Metal and original indexer lifecycle receipts; no model weights
|
||||||
|
test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_prefix_matches_reference ... ignored, requires Apple Metal, original stateful QSAIndexer receipts, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::qsa_prefill::mtplx_qsa_prefill_graph_matches_reference ... ignored, requires Metal and original two-stage prefill receipts, no weights
|
||||||
|
test engine::metal::qwen_mtplx::qsa_prepare::mtplx_qsa_prepare_graph_matches_reference ... ignored, requires Apple Metal; parameterized original QSA preparation and traced positions
|
||||||
|
test engine::metal::qwen_mtplx::qsa_select::mtplx_qsa_fused_graph_matches_reference ... ignored, requires Apple Metal and original QSA selector receipts; no model weights
|
||||||
|
test engine::metal::qwen_mtplx::random::mtplx_sampler_prefix_matches_reference ... ignored, requires Apple Metal; original target-prefix sampler and default RNG
|
||||||
|
test engine::metal::qwen_mtplx::random::mtplx_sampler_resident_matches_reference ... ignored, requires Apple Metal; original MTPLX resident sampling and keyed RNG
|
||||||
|
test engine::metal::qwen_mtplx::qsa_graph::qsa_schedule_preserves_the_selected_sibling ... ok
|
||||||
|
test engine::metal::qwen_mtplx::policy::retired_ar_pipeline_flag_cannot_change_server_or_turn_policy ... ok
|
||||||
|
test engine::metal::qwen_mtplx::reduce::mtplx_sampler_argmax_matches_reference ... ignored, requires Apple Metal; original MTPLX greedy ArgReduce and storage
|
||||||
|
test engine::metal::qwen_mtplx::reduce::mtplx_sum_graph_matches_reference ... ignored, requires Apple Metal; original shared sum dispatch and QSA mean
|
||||||
|
test engine::metal::qwen_mtplx::residency::mtplx_residency_matches_reference_lifecycle_cases ... ignored, requires Apple Metal 3 / macOS 15, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::rope::mtplx_rope_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX RoPE, static YaRN and multi-axis receipts
|
||||||
|
test engine::metal::qwen_mtplx::sampling::mtplx_sampler_batched_matches_reference ... ignored, requires Apple Metal; original MTPLX bound and serial batch distributions
|
||||||
|
test engine::metal::qwen_mtplx::sampling::mtplx_sampler_entry_matches_reference ... ignored, requires Apple Metal; actual MTPLX generator sampling entry
|
||||||
|
test engine::metal::qwen_mtplx::sampling::mtplx_sampler_fallback_matches_reference ... ignored, requires Apple Metal; original sparse fallback and host dense sampler
|
||||||
|
test engine::metal::qwen_mtplx::request::qwen_request_progress_uses_ui_frontiers_and_decode_timer ... ok
|
||||||
|
test engine::metal::qwen_mtplx::request::qwen_request_settings_override_backend_policy ... ok
|
||||||
|
test engine::metal::qwen_mtplx::sampling::mtplx_sampler_penalties_match_reference ... ignored, requires Apple Metal; original GPU and CPU raw-logit penalties
|
||||||
|
test engine::metal::qwen_mtplx::sampling::mtplx_sampler_selection_matches_reference ... ignored, requires Apple Metal; original MTPLX sampler selection, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::sampling::mtplx_sampler_serial_support_matches_reference ... ignored, requires Apple Metal; actual MTPLX serial candidate route, no weights
|
||||||
|
test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_primitive_holds_data_and_temporaries_in_fifo_order ... ignored, requires Apple Metal shared storage, no GPU dispatch or model weights
|
||||||
|
test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_dispatch_counts_tenth_and_completes_after_failed_work ... ok
|
||||||
|
test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_simplify_preserves_reference_parent_rules ... ok
|
||||||
|
test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_cpu_events_propagate_errors_without_consuming_source ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::sdpa::mtplx_sdpa_vector_matches_reference ... ignored, requires Apple M5 Max; actual pinned fused SDPA kernels
|
||||||
|
test engine::metal::qwen_mtplx::session_cache::mtplx_full_installed_chat_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_CHAT_REFERENCE required
|
||||||
|
test engine::metal::qwen_mtplx::session_cache::mtplx_session_boundary_policy ... ok
|
||||||
|
test engine::metal::qwen_mtplx::session_cache::mtplx_session_final_state_requires_capture_and_safe_cache ... ok
|
||||||
|
test engine::metal::qwen_mtplx::session_checkpoint::qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state ... ignored, model-free Metal checkpoint; external supervisor and DS4_QWEN38_CHECKPOINT_DIR required
|
||||||
|
test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_host_import_matches_reference ... ignored, model-free Metal imports; external supervisor and DS4_HOST_IMPORT_REFERENCE required
|
||||||
|
test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_matches_original_blobs_and_restore ... ignored, tiny GPU arrays; external supervisor and DS4_SESSION_CODEC_REFERENCE required
|
||||||
|
test engine::metal::qwen_mtplx::sort::mtplx_sort_graph_matches_reference ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_streams_preserve_fifo_errors_and_shutdown ... ok
|
||||||
|
test engine::metal::qwen_mtplx::sparse_attention::mtplx_sparse_attention_graph_matches_reference ... ignored, requires Apple M5; actual MTPLX sparse Decode and TensorOps Prefill consumers
|
||||||
|
test engine::metal::qwen_mtplx::speculative::mtplx_full_installed_mtp_generation_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_MTP_GENERATION_REFERENCE required
|
||||||
|
test engine::metal::qwen_mtplx::stream::mtplx_stream_routes_queues_events_and_finalizes_all_under_pressure ... ignored, requires Apple Metal, tiny model-free two-queue copy
|
||||||
|
test engine::metal::profile::tests::profile_reports_locality_and_lru_hits ... ok
|
||||||
|
test engine::metal::qwen_mtplx::submission::mtplx_submission_attaches_at_flush_eval_finish_and_drop ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::unary::mtplx_graph_unary_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph unary layout, donation and kernel boundaries
|
||||||
|
test engine::metal::qwen_mtplx::views::mtplx_dtype_conversion_and_views_preserve_values_layout_and_donation ... ignored, requires Apple Metal; original conversion/view kernels and donation in lazy eval
|
||||||
|
test engine::metal::qwen_mtplx::views::mtplx_dynamic_slice_graph_matches_reference ... ignored, requires Metal and original dynamic slice receipts; no model weights
|
||||||
|
test engine::metal::qwen_mtplx::views::mtplx_reshape_shares_or_copies_in_the_connected_evaluator ... ignored, requires Apple Metal; original Reshape copy kernels and shared backing
|
||||||
|
test engine::metal::qwen_mtplx::views::mtplx_views_preserve_aliases_offsets_strides_and_gpu_output ... ignored, requires Apple Metal; connected zero-copy views and offset-aware original copy kernel
|
||||||
|
test engine::metal::qwen_mtplx::vision::qwen_vision_tower_matches_mtplx_image ... ignored, requires Metal and pinned Qwen vision artifacts/reference values
|
||||||
|
test engine::metal::qwen_mtplx::weights::mtplx_checkpoint_projection_fusions_match_reference ... ignored, requires Apple Metal; bounded lazy checkpoint Load and fusion
|
||||||
|
test engine::metal::qwen_mtplx::weights::mtplx_installed_checkpoint_fusion_metadata_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS; headers only, never evaluate weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_affine_selection_preserves_reference_boundaries ... ok
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_canonical_hyper_and_gdn_kernels_match_reference ... ignored, requires Apple Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::stream::mtplx_stream_registry_preserves_thread_defaults_templates_and_clear ... ok
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_connected_delta_update_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_connected_gather_sort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_decode_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_forward_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_staged_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_connected_norm_modules_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_connected_router_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_connected_sparse_moe_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_connected_switch_glu_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_gate_up_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_cache_lifecycle_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_backings_and_dynamic_copies_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_routing_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_window_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_capacity_matches_reference ... ok
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_indexer_modes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_scalar_sources_match_reference ... ok
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_connected_cache_selection_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_eager_topk_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_f32_scores_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_preparation_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_routes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_producer_dispatch_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_canonical_source_bodies_preserve_pinned_hashes ... ok
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_selector_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_runtime_affine_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_runtime_argsort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_runtime_gather_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
|
||||||
|
test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_activity_preserves_reference_pressure_and_wait_rules ... ok
|
||||||
|
test engine::metal::tests::batch_raw_span_includes_the_chunk_and_previous_window ... ok
|
||||||
|
test engine::metal::tests::compression_schedule_tracks_the_deepseek_model_shape ... ok
|
||||||
|
test engine::metal::tests::decode_feature_policy_keeps_device_generations_and_rollbacks_separate ... ok
|
||||||
|
test engine::metal::tests::decode_raw_span_tracks_the_logical_sliding_window ... ok
|
||||||
|
test engine::metal::tests::directional_steering_matches_the_ds4_token_oracle ... ignored, requires the installed Flash GGUF, DS4 steering fixture, and a Metal device
|
||||||
|
test engine::metal::tests::dspark_confidence_quantizes_q8_input_like_ds4 ... ok
|
||||||
|
test engine::metal::tests::dspark_markov_activation_matches_q8_rounding ... ok
|
||||||
|
test engine::metal::tests::dspark_matches_original_summary_cycles ... ignored, requires installed 0731/DSpark and DS4SERVER_DSPARK_REFERENCE cycle recording
|
||||||
|
test engine::metal::tests::dspark_runs_a_target_owned_greedy_cycle ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures
|
||||||
|
test engine::metal::tests::dspark_scheduler_matches_the_ds4_default_window ... ok
|
||||||
|
test engine::metal::tests::dspark_verifier_selects_only_the_committed_row ... ok
|
||||||
|
test engine::metal::tests::flash_0731_long_context_crosses_indexed_prefill_boundary ... ignored, requires a 0731 Flash GGUF and an Apple M5 device
|
||||||
|
test engine::metal::tests::flash_0731_m5_decode_performance_gate ... ignored, requires a 0731 Flash GGUF and an Apple M5 device
|
||||||
|
test engine::metal::tests::flash_0731_resident_and_ssd_streaming_choose_the_same_tokens ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
|
||||||
|
test engine::metal::tests::flash_0731_runs_exact_sampled_dspark ... ignored, requires the installed 0731 target and checkpoint-specific DSpark GGUF fixtures
|
||||||
|
test engine::metal::tests::flash_0731_ssd_streaming_maps_and_seeds_batched_prefill_layers ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
|
||||||
|
test engine::metal::tests::generated_hotlists_match_each_model_shape ... ok
|
||||||
|
test engine::metal::tests::half_conversion_handles_normal_and_subnormal_values ... ok
|
||||||
|
test engine::metal::tests::pro_q4_model_spans_remain_isolated ... ok
|
||||||
|
test engine::metal::tests::pro_resident_admission_includes_weights_context_and_scratch ... ok
|
||||||
|
test engine::metal::tests::pro_ssd_streaming_full_layer_and_selected_batch_agree ... ignored, requires the installed DeepSeek V4 Pro GGUF and Apple Metal
|
||||||
|
test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
|
||||||
|
test engine::metal::tests::ssd_streaming_supports_dspark ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures
|
||||||
|
test engine::metal::tests::streaming_memory_plan_matches_ds4_graph_formulas ... ok
|
||||||
|
test engine::metal::vision::tests::glm53_resize_matches_reference_token_grids ... ok
|
||||||
|
test engine::metal::qwen_mtplx::submission::mtplx_all_native_command_commits_pass_the_submission_hook ... ok
|
||||||
|
test engine::qwen::tests::metadata_and_safetensors_boundaries_fail_closed ... ok
|
||||||
|
test engine::qwen::tests::pinned_artifact_set_loads_and_renders_goldens ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
|
||||||
|
test engine::qwen::tests::pinned_qwen_metadata_matches_product_without_executor_state ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
|
||||||
|
test engine::qwen_vision::qwen_image_cache_keys_distinguish_pixels_and_preserve_prior_text ... ok
|
||||||
|
test engine::qwen::tests::frozen_inventory_and_memory_categories_are_exact ... ok
|
||||||
|
test engine::qwen_vision::qwen_vision_positions_and_resize_contract ... ok
|
||||||
|
test dev_brain::tests::parser_and_index_rebuild_ignore_links_in_code ... ok
|
||||||
|
test engine::sampling_tests::checkpoint_rebuilds_explain_compatibility_and_history_misses ... ok
|
||||||
|
test engine::sampling_tests::checkpoint_tag_accepts_an_unchanged_prefix_before_reminders ... ok
|
||||||
|
test engine::metal::qwen_mtplx::sampling::mtplx_sampler_distribution_and_rng_match_reference ... ok
|
||||||
|
test engine::sampling_tests::checkpoint_headers_identify_the_exact_model ... ok
|
||||||
|
test engine::sampling_tests::checkpoint_tag_covers_the_canonical_chat_state ... ok
|
||||||
|
test engine::sampling_tests::ds4_sampler_real_logits_cpu_benchmark ... ignored, CPU-only; requires DS4_SAMPLER_RECORDING with the recorded GLM logits
|
||||||
|
test engine::sampling_tests::exact_delta_sampling_accepts_or_corrects_the_draft ... ok
|
||||||
|
test engine::sampling_tests::generated_text_drains_incomplete_bytes_before_phase_changes_and_finish ... ok
|
||||||
|
test engine::sampling_tests::metal_executes_real_flash_token ... ignored, requires the 80 GiB Flash checkpoint and Apple Metal
|
||||||
|
test engine::sampling_tests::generated_text_preserves_thinking_utf8_and_split_stops ... ok
|
||||||
|
test engine::sampling_tests::qwen_rng_matches_numpy_pcg64 ... ok
|
||||||
|
test engine::sampling_tests::qwen_sampling_uses_mtplx_global_nucleus_mass_before_top_k ... ok
|
||||||
|
test engine::sampling_tests::resumed_prefill_crossover_matches_each_ds4_graph ... ok
|
||||||
|
test engine::sampling_tests::sampling_probabilities_match_ds4_filter_order ... ok
|
||||||
|
test engine::sampling_tests::split_utf8_token_bytes_are_joined_before_decoding ... ok
|
||||||
|
test engine::sampling_tests::top_k_and_stream_stops_are_applied_before_output ... ok
|
||||||
|
test engine::sampling_tests::workspace_instruction_identity_participates_in_append_only_kv_keys ... ok
|
||||||
|
test engine::sampling_tests::zero_temperature_is_greedy ... ok
|
||||||
|
test engine::tokenizer::issue111::issue111_ds4_families ... ignored, CPU-only; installed DS4 GGUFs and exact oracle IDs
|
||||||
|
test engine::tokenizer::issue111::issue111_every_unicode_scalar_split ... ignored, CPU-only; exhaustive pinned Qwen split receipt
|
||||||
|
test engine::tokenizer::issue111::issue111_qwen_tokens_and_requests ... ignored, CPU-only; installed Qwen tokenizer and pinned exact IDs
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_workspace_matches_reference ... ok
|
||||||
|
test engine::tokenizer::tests::character_helpers_are_total_at_every_byte_offset ... ok
|
||||||
|
test engine::tokenizer::tests::ds4_chat_matches_original_session_tokens ... ignored, CPU-only; requires DS4SERVER_CHAT_REFERENCE and its installed GGUF
|
||||||
|
test engine::tokenizer::tests::qwen_checkpoint_continuation_matches_full_conversation ... ignored, requires the installed Qwen tokenizer, but no GPU or model weights
|
||||||
|
test engine::tokenizer::tests::qwen_plain_chat_matches_mtplx_server_tokens ... ignored, requires local tokenizer and tools/qwen-chat-reference.py output; no inference
|
||||||
|
test engine::tokenizer::tests::reasoning_prefixes_match_each_model_family_contract ... ok
|
||||||
|
test engine::validation::tests::configured_glm53_main_fixture_passes_the_exact_layout ... ok
|
||||||
|
test engine::validation::tests::configured_glm53_vision_fixture_passes_the_exact_layout ... ok
|
||||||
|
test engine::tokenizer::issue111::qwen_split_boundaries ... ok
|
||||||
|
test engine::validation::tests::installed_dspark_fixture_passes_the_target_layout ... ok
|
||||||
|
test engine::validation::tests::mxfp4_is_limited_to_routed_experts ... ok
|
||||||
|
test engine::metal::qwen_mtplx::ngram::mtplx_ngram_cpu_rows_and_raw_history_match_reference ... ok
|
||||||
|
test extensions::tests::duplicate_skill_names_and_update_rollback_are_deterministic ... ok
|
||||||
|
test dev_brain::tests::validation_names_every_drifted_source_and_requires_semantic_refresh ... ok
|
||||||
|
test extensions::tests::hook_output_is_bounded_typed_and_event_matched ... ok
|
||||||
|
test engine::sampling_tests::probability_ratio_speculation_preserves_the_target_distribution ... ok
|
||||||
|
test extensions::tests::manifest_and_paths_are_validated_without_following_escapes ... ok
|
||||||
|
test extensions::tests::pinned_ponytail_runs_all_lifecycle_modes ... ignored, downloads the pinned Ponytail reference integration
|
||||||
|
test extensions::tests::registry_persists_enable_disable_errors_and_uninstall ... ok
|
||||||
|
test extensions::tests::symlink_and_variable_escapes_are_rejected ... ok
|
||||||
|
test instructions::tests::changes_removals_resume_and_compaction_are_append_only ... ok
|
||||||
|
test engine::metal::qwen_mtplx::array::mtplx_array_releases_sibling_cycles_and_deep_input_graphs ... ok
|
||||||
|
test instructions::tests::discovery_is_global_to_local_deduplicated_and_confined ... ok
|
||||||
|
test instructions::tests::empty_and_repaired_baselines_do_not_repeat_prompt_content ... ok
|
||||||
|
test instructions::tests::newly_crossed_scope_is_added_once_and_unavailable_content_is_retained ... ok
|
||||||
|
test metrics::tests::cache_report_splits_buckets_and_folds_index_files_into_their_checkpoint ... ok
|
||||||
|
test metrics::tests::decode_timer_excludes_later_work_and_resets_between_requests ... ok
|
||||||
|
test metrics::tests::prompt_timing_keeps_unmeasured_components_distinct_from_zero ... ok
|
||||||
|
test metrics::tests::removed_preview_metric_code_stays_decodable ... ok
|
||||||
|
test metrics::tests::snapshots_track_runtime_and_server_counters ... ok
|
||||||
|
test model::qwen::qwen_vision_is_an_independent_managed_artifact_set ... ok
|
||||||
|
test model::qwen::tests::embedded_manifest_is_the_frozen_complete_set ... ok
|
||||||
|
test model::qwen::tests::verification_rejects_hash_mismatch_and_honors_cancellation ... ok
|
||||||
|
test model::transfer::tests::cancellation_keeps_the_partial_file_for_the_next_run ... ok
|
||||||
|
test model::transfer::tests::catalog_and_checksum_verification_are_explicit ... ok
|
||||||
|
test model::transfer::tests::managed_artifact_inventory_and_delete_include_partial_files ... ok
|
||||||
|
test model::transfer::tests::qwen_set_verifies_reloads_and_removes_as_one_managed_artifact ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
|
||||||
|
test instructions::tests::frames_escape_control_tags_and_enforce_utf8_byte_limits ... ok
|
||||||
|
test model::transfer::tests::restart_resumes_at_the_existing_partial_byte ... ok
|
||||||
|
test model::transfer::tests::verification_reports_bytes_read ... ok
|
||||||
|
test model_eval::tests::acceleration_receipt_reports_effective_sampling_contract ... ok
|
||||||
|
test dev_brain::tests::skill_validation_and_default_guide_restore_are_strict ... ok
|
||||||
|
test model_eval::tests::acceleration_switch_uses_each_models_native_path ... ok
|
||||||
|
test model_eval::tests::canary_cross_phase_samples_keep_their_start_phase ... ok
|
||||||
|
test model_eval::tests::evaluations_exclude_overlapping_workers_and_release_on_exit ... ok
|
||||||
|
test model_eval::tests::external_canary_observes_phases_and_finishes_on_eof ... ignored, Metal probe only; DS4SERVER_CANARY_BINARY must point to the current release binary
|
||||||
|
test model_eval::tests::external_canary_protocol_accepts_only_known_phases ... ok
|
||||||
|
test model_eval::tests::parses_safe_defaults_and_overrides ... ok
|
||||||
|
test model_eval::tests::rejects_unbounded_or_invalid_limits ... ok
|
||||||
|
test model_eval::tests::repeated_prompts_form_one_ordered_conversation ... ok
|
||||||
|
test model_eval::tests::warmup_timing_separates_loading_without_inventing_missing_durations ... ok
|
||||||
|
test model_eval::tests::worker_progress_events_advance_the_watchdog ... ok
|
||||||
|
test native_edit::tests::edit_history_supports_undo_and_redo ... ok
|
||||||
|
test native_edit::tests::paste_replays_a_complete_command_shortcut ... ok
|
||||||
|
test native_media::tests::media_player_only_accepts_remote_http_urls ... ok
|
||||||
|
test native_menu::tests::edit_items_are_custom_menu_events ... ok
|
||||||
|
test native_menu::tests::user_guide_uses_unmodified_f1 ... ok
|
||||||
|
test native_menu::tests::view_items_keep_the_selected_pane_checked ... ok
|
||||||
|
test runtime::tests::dropping_active_generation_cancels_it ... ok
|
||||||
|
test runtime::tests::operation_failures_use_the_matching_event ... ok
|
||||||
|
test runtime::tests::operation_tracking_is_consistent ... ok
|
||||||
|
test runtime::tests::runtime_panic_recovery_accepts_the_next_operation ... ok
|
||||||
|
test server::request::tests::reasoning_requests_preserve_each_models_exact_modes ... ok
|
||||||
|
test server::tests::canonical_tool_replay_preserves_argument_types ... ok
|
||||||
|
test server::tests::connection_slots_enforce_the_limit_and_release_on_drop ... ok
|
||||||
|
test server::tests::content_arrays_match_ds4_text_projection ... ok
|
||||||
|
test server::tests::context_limit_errors_use_protocol_standard_fields ... ok
|
||||||
|
test server::tests::cors_headers_are_opt_in_and_cover_preflight ... ok
|
||||||
|
test server::tests::dsml_tool_stream_projects_incremental_json_arguments ... ok
|
||||||
|
test server::tests::generated_dsml_becomes_openai_tool_calls ... ok
|
||||||
|
test server::tests::generated_tool_parser_accepts_ds4_recovery_syntaxes ... ok
|
||||||
|
test server::tests::model_metadata_uses_the_requested_alias_and_default_token_limit ... ok
|
||||||
|
test server::tests::protocol_tool_results_require_live_or_replayed_call_ids ... ok
|
||||||
|
test dev_brain::tests::source_hash_drift_excludes_page_from_authoritative_queries ... ok
|
||||||
|
test server::tests::responses_parser_rejects_dropped_state_and_preserves_hosted_calls ... ok
|
||||||
|
test server::tests::sampled_tool_replay_survives_server_restart ... ok
|
||||||
|
test server::tests::streaming_errors_match_ds4_server_framing ... ok
|
||||||
|
test dev_brain::tests::verified_skills_are_indexed_and_injected_on_demand ... ok
|
||||||
|
test server::tests::tool_prompt_matches_ds4_server_text ... ok
|
||||||
|
test server::tests::tool_stream_projects_ds4_recovery_syntaxes ... ok
|
||||||
|
test settings::tests::captured_ds4_cli_options_are_uniquely_classified ... ok
|
||||||
|
test settings::tests::effective_settings_build_one_engine_and_turn_configuration ... ok
|
||||||
|
test settings::tests::effective_settings_preserve_unset_model_defaults ... ok
|
||||||
|
test settings::tests::execution_settings_keep_ds4_automatic_values_and_glm_rules ... ok
|
||||||
|
test settings::tests::kv_cache_gates_follow_the_ds4_store_rules ... ok
|
||||||
|
test settings::tests::reasoning_selection_is_exact_and_model_specific ... ok
|
||||||
|
test settings::tests::runtime_settings_preserve_automatic_values_and_cross_field_rules ... ok
|
||||||
|
test settings::tests::speculative_settings_match_acceleration_defaults_and_dependencies ... ok
|
||||||
|
test dev_brain::tests::validation_keeps_broken_links_editable_and_reports_provenance_errors ... ok
|
||||||
|
test server::tests::streaming_prefill_keeps_the_client_connection_alive ... ok
|
||||||
|
test dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree ... ok
|
||||||
|
test model_eval::tests::cpu_accounting_matches_getrusage_units ... ok
|
||||||
|
test engine::validation::tests::installed_ds4_fixture_opens_and_renders_a_prompt ... ok
|
||||||
|
test model::transfer::tests::stalled_download_times_out_and_keeps_the_partial_file ... ok
|
||||||
|
test server::tests::request_deadline_stops_slow_clients ... ok
|
||||||
|
test extensions::tests::direct_hook_runner_isolated_environment_and_handles_failures ... ok
|
||||||
|
test engine::sampling_tests::ds4_sampler_matches_original_reference_tokens_and_rng ... ok
|
||||||
|
test agent::tests::execution_failures_and_timeouts_have_distinct_codes ... ok
|
||||||
|
kill: 91003: No such process
|
||||||
|
test agent::tests::dropping_session_tools_stops_live_jobs_and_removes_output ... ok
|
||||||
|
test agent::tests::local_file_and_bash_tools_execute_in_the_project ... ok
|
||||||
|
test engine::qwen_vision::qwen_image_preprocessing_matches_pillow_mtplx ... ok
|
||||||
|
test extensions::tests::lifecycle_matchers_preserve_declared_event_order ... ok
|
||||||
|
test engine::metal::qwen_mtplx_tests::mtplx_runtime_shaders_are_pinned ... ok
|
||||||
|
test extensions::tests::hook_failures_are_bounded_actionable_and_non_shell ... ok
|
||||||
|
test engine::metal::qwen_mtplx::sampling::mtplx_sampler_mixed_distributions_match_reference ... ok
|
||||||
|
|
||||||
|
test result: ok. 312 passed; 0 failed; 234 ignored; 0 measured; 0 filtered out; finished in 5.76s
|
||||||
|
|
||||||
|
Running unittests tools/qwen38-artifacts.rs (target/debug/deps/qwen38_artifacts-1288d410762a1a77)
|
||||||
|
|
||||||
|
running 4 tests
|
||||||
|
test tests::artifact_names_cannot_escape_the_output_directory ... ok
|
||||||
|
test tests::affine_sample_digest_uses_packed_values_scales_and_biases ... ok
|
||||||
|
test tests::safetensors_validation_rejects_duplicate_names_and_invalid_ranges ... ok
|
||||||
|
test tests::artifact_download_resumes_an_authenticated_partial ... ok
|
||||||
|
|
||||||
|
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
|
||||||
|
|
||||||
|
Running unittests tools/test-supervisor.rs (target/debug/deps/test_supervisor-df80f419c8443a12)
|
||||||
|
|
||||||
|
running 4 tests
|
||||||
|
test supervisor::tests::direct_worker_arguments_are_not_rust_test_arguments ... ok
|
||||||
|
test supervisor::tests::split_records_cannot_be_interleaved_with_watchdog_samples ... ok
|
||||||
|
test supervisor::tests::split_probe_records_do_not_reset_progress ... ok
|
||||||
|
ready{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
ready
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
{"event":"gpu_canary_sample","ok":true}
|
||||||
|
progressprogressprogressprogressprogressprogressprogressprogresstest supervisor::tests::watchdog_stops_memory_and_silence_but_not_total_runtime ... ok
|
||||||
|
|
||||||
|
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.38s
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright © 2023 Apple Inc.
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
@@ -0,0 +1,199 @@
|
|||||||
|
// MTPLX 9882703, MIT; see MTPLX-LICENSE.txt.
|
||||||
|
|
||||||
|
constexpr int BM = 64;
|
||||||
|
constexpr int BN = 64;
|
||||||
|
constexpr int BK = 64;
|
||||||
|
constexpr int WN = 2;
|
||||||
|
constexpr short SM = 32;
|
||||||
|
constexpr short SN = 32;
|
||||||
|
constexpr short SK = 32;
|
||||||
|
constexpr short TM = SM / 16;
|
||||||
|
constexpr short TN = SN / 16;
|
||||||
|
constexpr short TK = SK / 16;
|
||||||
|
constexpr int PACK = get_pack_factor<BITS, 8>();
|
||||||
|
constexpr int PACK_BYTES = get_bytes_per_pack<BITS>();
|
||||||
|
constexpr int BK_PAD = BK + 16 / sizeof(T);
|
||||||
|
constexpr int ROW_BYTES = KD * PACK_BYTES / PACK;
|
||||||
|
constexpr int ROW_GROUPS = KD / GS;
|
||||||
|
constexpr int OUT_COLS = OUTC;
|
||||||
|
// SWIGLU: the tile's first 32 weight rows are gate rows, the last 32 the
|
||||||
|
// matching up rows, each half loaded by 64 threads from its own row range.
|
||||||
|
using WeightLoader = metal::conditional_t<
|
||||||
|
SWIGLU,
|
||||||
|
QuantizedBlockLoader<T, BN / 2, BK, BK_PAD, 1, 64, GS, BITS>,
|
||||||
|
QuantizedBlockLoader<T, BN, BK, BK_PAD, 1, 128, GS, BITS>>;
|
||||||
|
|
||||||
|
threadgroup T w_tile[BN * BK_PAD];
|
||||||
|
|
||||||
|
// The tile this threadgroup owns, and the one expert whose rows it holds:
|
||||||
|
// the last expert whose first tile is at or before it.
|
||||||
|
const int tile = int(threadgroup_position_in_grid.y);
|
||||||
|
if (tile >= tile_start[ED]) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
int lo = 0;
|
||||||
|
int hi = ED - 1;
|
||||||
|
while (lo < hi) {
|
||||||
|
const int mid = (lo + hi + 1) >> 1;
|
||||||
|
if (tile_start[mid] <= tile) {
|
||||||
|
lo = mid;
|
||||||
|
} else {
|
||||||
|
hi = mid - 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const int expert = lo;
|
||||||
|
const int row0 = row_start[expert] + (tile - tile_start[expert]) * BM;
|
||||||
|
const int tile_rows = min(BM, row_start[expert + 1] - row0);
|
||||||
|
const int col0 = int(threadgroup_position_in_grid.x) * BN;
|
||||||
|
|
||||||
|
const ushort sg = ushort(simdgroup_index_in_threadgroup);
|
||||||
|
const ushort lane = ushort(thread_index_in_simdgroup);
|
||||||
|
const short tm = SM * short(sg / WN);
|
||||||
|
const short tn = SN * short(sg % WN);
|
||||||
|
// Rows of this tile that fall in this simdgroup's 32-row half (0 to 32).
|
||||||
|
const short rows = short(clamp(tile_rows - int(tm), 0, int(SM)));
|
||||||
|
|
||||||
|
ushort load_sg = sg;
|
||||||
|
const device uint8_t* w_src = (const device uint8_t*)w;
|
||||||
|
const device T* s_src = scales;
|
||||||
|
const device T* b_src = biases;
|
||||||
|
size_t w_row = size_t(expert) * ND + size_t(col0);
|
||||||
|
threadgroup T* w_dst = w_tile;
|
||||||
|
if (SWIGLU) {
|
||||||
|
// Output columns [col0 / 2, col0 / 2 + 32): the matching gate rows
|
||||||
|
// of w and up rows of wu (the same array at an offset when the
|
||||||
|
// weight is a fused [gate | up]).
|
||||||
|
const int up_half = int(load_sg / 2);
|
||||||
|
const size_t c = size_t(col0 / 2);
|
||||||
|
if (up_half) {
|
||||||
|
w_src = (const device uint8_t*)wu;
|
||||||
|
s_src = scales_u;
|
||||||
|
b_src = biases_u;
|
||||||
|
w_row = size_t(expert) * UD + size_t(UOFF) + c;
|
||||||
|
} else {
|
||||||
|
w_row = size_t(expert) * ND + c;
|
||||||
|
}
|
||||||
|
w_dst = w_tile + up_half * (BN / 2) * BK_PAD;
|
||||||
|
load_sg = load_sg % 2;
|
||||||
|
}
|
||||||
|
thread WeightLoader loader(
|
||||||
|
w_src + w_row * ROW_BYTES,
|
||||||
|
s_src + w_row * ROW_GROUPS,
|
||||||
|
b_src + w_row * ROW_GROUPS,
|
||||||
|
KD,
|
||||||
|
w_dst,
|
||||||
|
load_sg,
|
||||||
|
lane);
|
||||||
|
|
||||||
|
NAXTile<float, TM, TN> acc;
|
||||||
|
acc.clear();
|
||||||
|
|
||||||
|
// A lane holds rows (y, y + 8) and four consecutive columns from x of every
|
||||||
|
// 16x16 fragment (MLX's fragment coordinates). Its four token rows are
|
||||||
|
// fixed for the whole K loop; rows past the tile read as zero, as MLX's
|
||||||
|
// bounded load does.
|
||||||
|
const short2 coord = BaseNAXFrag::get_coord();
|
||||||
|
const device T* lane_rows[TM][2];
|
||||||
|
bool lane_live[TM][2];
|
||||||
|
for (short i = 0; i < TM; i++) {
|
||||||
|
for (short r = 0; r < 2; r++) {
|
||||||
|
const short local = i * 16 + coord.y + r * 8;
|
||||||
|
lane_live[i][r] = local < rows;
|
||||||
|
const int token = lane_live[i][r] ? int(row_map[row0 + tm + local]) : 0;
|
||||||
|
lane_rows[i][r] = x + size_t(token) * KD + coord.x;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The K loop, compiled twice: for a simdgroup whose 32 rows are all live
|
||||||
|
// (no bounds) and for a partial or empty half.
|
||||||
|
dispatch_bool(rows == SM, [&](auto full) {
|
||||||
|
for (int k = 0; k < KD / BK; k++) {
|
||||||
|
threadgroup_barrier(mem_flags::mem_threadgroup);
|
||||||
|
loader.load_unsafe();
|
||||||
|
threadgroup_barrier(mem_flags::mem_threadgroup);
|
||||||
|
|
||||||
|
STEEL_PRAGMA_NO_UNROLL
|
||||||
|
for (short kk = 0; kk < BK; kk += SK) {
|
||||||
|
if (decltype(full)::value || rows > 0) {
|
||||||
|
NAXTile<T, TM, TK> a;
|
||||||
|
NAXTile<T, TN, TK> b;
|
||||||
|
volatile int keep_order;
|
||||||
|
const int col = k * BK + kk;
|
||||||
|
STEEL_PRAGMA_UNROLL
|
||||||
|
for (short i = 0; i < TM; i++) {
|
||||||
|
STEEL_PRAGMA_UNROLL
|
||||||
|
for (short j = 0; j < TK; j++) {
|
||||||
|
thread auto& frag = a.frag_at(i, j);
|
||||||
|
STEEL_PRAGMA_UNROLL
|
||||||
|
for (short r = 0; r < 2; r++) {
|
||||||
|
const device T* src = lane_rows[i][r] + col + j * 16;
|
||||||
|
STEEL_PRAGMA_UNROLL
|
||||||
|
for (short c = 0; c < 4; c++) {
|
||||||
|
if constexpr (decltype(full)::value) {
|
||||||
|
frag[r * 4 + c] = src[c];
|
||||||
|
} else {
|
||||||
|
frag[r * 4 + c] = lane_live[i][r] ? src[c] : T(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
b.template load<T, BK_PAD, 1>(w_tile + tn * BK_PAD + kk);
|
||||||
|
tile_matmad_nax(
|
||||||
|
acc,
|
||||||
|
a,
|
||||||
|
metal::bool_constant<false>{},
|
||||||
|
b,
|
||||||
|
metal::bool_constant<true>{});
|
||||||
|
(void)keep_order;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
loader.next();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
if constexpr (SWIGLU) {
|
||||||
|
// The up simdgroup of each 32-row half hands its values, rounded to T
|
||||||
|
// as the stock gather's output is, to the gate simdgroup of the same
|
||||||
|
// rows through the (now free) weight tile, lane to lane. The gate
|
||||||
|
// simdgroup writes silu(gate) * up with MLX's own functors, rounding to
|
||||||
|
// T after each op as the stock chain does.
|
||||||
|
constexpr short VALS = TM * TN * 8;
|
||||||
|
static_assert(2 * 32 * VALS <= BN * BK_PAD, "the hand-off must fit the weight tile");
|
||||||
|
threadgroup T* handoff = w_tile + (sg / 2) * (32 * VALS) + lane * VALS;
|
||||||
|
const thread float* vals = acc.elems();
|
||||||
|
threadgroup_barrier(mem_flags::mem_threadgroup);
|
||||||
|
if (tn != 0) {
|
||||||
|
for (short e = 0; e < VALS; e++) {
|
||||||
|
handoff[e] = static_cast<T>(vals[e]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
threadgroup_barrier(mem_flags::mem_threadgroup);
|
||||||
|
if (tn == 0 && rows > 0) {
|
||||||
|
device T* out = y + size_t(row0 + tm) * OUT_COLS + size_t(col0 / 2);
|
||||||
|
for (short i = 0; i < TM; i++) {
|
||||||
|
for (short j = 0; j < TN; j++) {
|
||||||
|
for (short r = 0; r < 2; r++) {
|
||||||
|
const short row = i * 16 + coord.y + r * 8;
|
||||||
|
if (row >= rows) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
for (short c = 0; c < 4; c++) {
|
||||||
|
const short e = (i * TN + j) * 8 + r * 4 + c;
|
||||||
|
const T gate = static_cast<T>(vals[e]);
|
||||||
|
const T act = Multiply()(gate, Sigmoid()(gate));
|
||||||
|
out[size_t(row) * OUT_COLS + j * 16 + coord.x + c] =
|
||||||
|
Multiply()(act, handoff[e]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if (rows > 0) {
|
||||||
|
device T* out = y + size_t(row0 + tm) * ND + size_t(col0 + tn);
|
||||||
|
if (rows == SM) {
|
||||||
|
acc.store(out, ND);
|
||||||
|
} else {
|
||||||
|
acc.store_safe(out, ND, short2(SN, rows));
|
||||||
|
}
|
||||||
|
}
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -41,11 +41,14 @@ mod issue108;
|
|||||||
mod issue109;
|
mod issue109;
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod issue110;
|
mod issue110;
|
||||||
|
#[cfg(test)]
|
||||||
|
mod issue112_bench;
|
||||||
mod linear;
|
mod linear;
|
||||||
mod load;
|
mod load;
|
||||||
mod mlp;
|
mod mlp;
|
||||||
mod model;
|
mod model;
|
||||||
mod moe;
|
mod moe;
|
||||||
|
mod moe_sorted;
|
||||||
mod mtp;
|
mod mtp;
|
||||||
mod ngram;
|
mod ngram;
|
||||||
mod ngram_sidecar;
|
mod ngram_sidecar;
|
||||||
@@ -1200,7 +1203,7 @@ fn gather_bf16_broadcast(
|
|||||||
let lhs_strides = lhs_strides.map(|d| d as i64);
|
let lhs_strides = lhs_strides.map(|d| d as i64);
|
||||||
let rhs_strides = rhs_strides.map(|d| d as i64);
|
let rhs_strides = rhs_strides.map(|d| d as i64);
|
||||||
|
|
||||||
gather_bf16_layout(
|
gather_layout(
|
||||||
[
|
[
|
||||||
tensor(0, w).input(elements[0]),
|
tensor(0, w).input(elements[0]),
|
||||||
tensor(1, s).input(elements[1]),
|
tensor(1, s).input(elements[1]),
|
||||||
@@ -1210,6 +1213,7 @@ fn gather_bf16_broadcast(
|
|||||||
tensor(5, rhs).input(elements[5]),
|
tensor(5, rhs).input(elements[5]),
|
||||||
tensor(6, y).output(elements[6]),
|
tensor(6, y).output(elements[6]),
|
||||||
],
|
],
|
||||||
|
array::Dtype::BF16,
|
||||||
[m, n, k, experts, routes, routes],
|
[m, n, k, experts, routes, routes],
|
||||||
[bits, group],
|
[bits, group],
|
||||||
sorted,
|
sorted,
|
||||||
@@ -1232,8 +1236,10 @@ fn gather_bf16_broadcast(
|
|||||||
|
|
||||||
/// Shared original GatherQMM dispatch; array batch strides are passed verbatim,
|
/// Shared original GatherQMM dispatch; array batch strides are passed verbatim,
|
||||||
/// while the two broadcast index layouts have jointly collapsed dimensions.
|
/// while the two broadcast index layouts have jointly collapsed dimensions.
|
||||||
fn gather_bf16_layout(
|
#[allow(clippy::too_many_arguments)] // Original GatherQMM ABI and array layouts.
|
||||||
|
fn gather_layout(
|
||||||
bindings: [Binding<'_>; 7],
|
bindings: [Binding<'_>; 7],
|
||||||
|
dtype: array::Dtype,
|
||||||
[m, n, k, experts, routes, sorted_rows]: [u32; 6],
|
[m, n, k, experts, routes, sorted_rows]: [u32; 6],
|
||||||
[bits, group]: [u32; 2],
|
[bits, group]: [u32; 2],
|
||||||
sorted: bool,
|
sorted: bool,
|
||||||
@@ -1248,7 +1254,12 @@ fn gather_bf16_layout(
|
|||||||
rhs_strides,
|
rhs_strides,
|
||||||
]: [&[i64]; 6],
|
]: [&[i64]; 6],
|
||||||
) -> Result<(), String> {
|
) -> Result<(), String> {
|
||||||
let suffix = format!("bfloat16_t_gs_{group}_b_{bits}");
|
let ty = match dtype {
|
||||||
|
array::Dtype::BF16 => "bfloat16_t",
|
||||||
|
array::Dtype::F16 => "float16_t",
|
||||||
|
_ => return Err("GatherQMM dtype unsupported".into()),
|
||||||
|
};
|
||||||
|
let suffix = format!("{ty}_gs_{group}_b_{bits}");
|
||||||
if m == 1 && routes >= 16 && sorted && routes / experts >= 4 {
|
if m == 1 && routes >= 16 && sorted && routes / experts >= 4 {
|
||||||
let (bm, bn, bk, wm, wn) = if nax {
|
let (bm, bn, bk, wm, wn) = if nax {
|
||||||
(
|
(
|
||||||
|
|||||||
@@ -438,6 +438,7 @@ impl Bank {
|
|||||||
policy: model::CompilePolicy,
|
policy: model::CompilePolicy,
|
||||||
fused_moe_decode: bool,
|
fused_moe_decode: bool,
|
||||||
fused_moe_verify: bool,
|
fused_moe_verify: bool,
|
||||||
|
sorted_gather: bool,
|
||||||
) -> Result<model::TextModel<'_>, String> {
|
) -> Result<model::TextModel<'_>, String> {
|
||||||
let mut bind = Bind {
|
let mut bind = Bind {
|
||||||
bank: self,
|
bank: self,
|
||||||
@@ -445,6 +446,7 @@ impl Bank {
|
|||||||
order: Vec::new(),
|
order: Vec::new(),
|
||||||
fused_moe_decode,
|
fused_moe_decode,
|
||||||
fused_moe_verify,
|
fused_moe_verify,
|
||||||
|
sorted_gather,
|
||||||
};
|
};
|
||||||
let embedding = bind.linear("language_model.model.embed_tokens")?;
|
let embedding = bind.linear("language_model.model.embed_tokens")?;
|
||||||
let layers = self
|
let layers = self
|
||||||
@@ -503,6 +505,7 @@ struct Bind<'a> {
|
|||||||
order: Vec<String>,
|
order: Vec<String>,
|
||||||
fused_moe_decode: bool,
|
fused_moe_decode: bool,
|
||||||
fused_moe_verify: bool,
|
fused_moe_verify: bool,
|
||||||
|
sorted_gather: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -566,7 +569,12 @@ fn mtplx_installed_module_binding_matches_reference() {
|
|||||||
"no eager parameter evaluation"
|
"no eager parameter evaluation"
|
||||||
);
|
);
|
||||||
let mut model = bank
|
let mut model = bank
|
||||||
.bind(model::CompilePolicy::from_values(None, None), true, true)
|
.bind(
|
||||||
|
model::CompilePolicy::from_values(None, None),
|
||||||
|
true,
|
||||||
|
true,
|
||||||
|
true,
|
||||||
|
)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
assert_eq!(case["mask"], order["mask"]);
|
assert_eq!(case["mask"], order["mask"]);
|
||||||
assert_eq!(case["mtp"], order["mtp"]);
|
assert_eq!(case["mtp"], order["mtp"]);
|
||||||
@@ -687,7 +695,12 @@ fn mtplx_installed_module_binding_matches_reference() {
|
|||||||
.all(|(_, a)| !a.has_data())
|
.all(|(_, a)| !a.has_data())
|
||||||
);
|
);
|
||||||
let mut owned = BoundModel::try_new(bank, |bank| {
|
let mut owned = BoundModel::try_new(bank, |bank| {
|
||||||
bank.bind(model::CompilePolicy::from_values(None, None), true, true)
|
bank.bind(
|
||||||
|
model::CompilePolicy::from_values(None, None),
|
||||||
|
true,
|
||||||
|
true,
|
||||||
|
true,
|
||||||
|
)
|
||||||
})
|
})
|
||||||
.unwrap();
|
.unwrap();
|
||||||
let module_address = std::ptr::from_ref(owned.borrow_dependent()).addr();
|
let module_address = std::ptr::from_ref(owned.borrow_dependent()).addr();
|
||||||
@@ -705,7 +718,12 @@ fn mtplx_installed_module_binding_matches_reference() {
|
|||||||
let extra = bank.parameters["language_model.model.layers.0.linear_attn.A_log"].clone();
|
let extra = bank.parameters["language_model.model.layers.0.linear_attn.A_log"].clone();
|
||||||
bank.parameters.insert("unexpected.weight".into(), extra);
|
bank.parameters.insert("unexpected.weight".into(), extra);
|
||||||
assert!(
|
assert!(
|
||||||
bank.bind(model::CompilePolicy::from_values(None, None), true, true)
|
bank.bind(
|
||||||
|
model::CompilePolicy::from_values(None, None),
|
||||||
|
true,
|
||||||
|
true,
|
||||||
|
true
|
||||||
|
)
|
||||||
.err()
|
.err()
|
||||||
.unwrap()
|
.unwrap()
|
||||||
.contains("unbound checkpoint parameters")
|
.contains("unbound checkpoint parameters")
|
||||||
@@ -714,7 +732,12 @@ fn mtplx_installed_module_binding_matches_reference() {
|
|||||||
bank.parameters
|
bank.parameters
|
||||||
.remove("language_model.model.layers.0.linear_attn.norm.weight");
|
.remove("language_model.model.layers.0.linear_attn.norm.weight");
|
||||||
assert!(
|
assert!(
|
||||||
bank.bind(model::CompilePolicy::from_values(None, None), true, true)
|
bank.bind(
|
||||||
|
model::CompilePolicy::from_values(None, None),
|
||||||
|
true,
|
||||||
|
true,
|
||||||
|
true
|
||||||
|
)
|
||||||
.err()
|
.err()
|
||||||
.unwrap()
|
.unwrap()
|
||||||
.contains("missing parameter")
|
.contains("missing parameter")
|
||||||
@@ -938,6 +961,7 @@ impl<'a> Bind<'a> {
|
|||||||
normalize: c.norm_topk_prob,
|
normalize: c.norm_topk_prob,
|
||||||
fused_decode: self.fused_moe_decode,
|
fused_decode: self.fused_moe_decode,
|
||||||
fused_verify: self.fused_moe_verify,
|
fused_verify: self.fused_moe_verify,
|
||||||
|
sorted_gather: self.sorted_gather,
|
||||||
};
|
};
|
||||||
let attn_hyper = self.hyper(&format!("{p}.attn_hyper_connection"), true)?;
|
let attn_hyper = self.hyper(&format!("{p}.attn_hyper_connection"), true)?;
|
||||||
let mlp_hyper = self.hyper(&format!("{p}.mlp_hyper_connection"), true)?;
|
let mlp_hyper = self.hyper(&format!("{p}.mlp_hyper_connection"), true)?;
|
||||||
|
|||||||
@@ -565,10 +565,10 @@ fn equivalent_operation(a: &Operation, b: &Operation, x: &Array, y: &Array) -> b
|
|||||||
| (AffineQuantize { bits: a, group: b }, AffineQuantize { bits: x, group: y })
|
| (AffineQuantize { bits: a, group: b }, AffineQuantize { bits: x, group: y })
|
||||||
| (AffineDequantize { bits: a, group: b }, AffineDequantize { bits: x, group: y })
|
| (AffineDequantize { bits: a, group: b }, AffineDequantize { bits: x, group: y })
|
||||||
| (
|
| (
|
||||||
GatherQmmBf16 {
|
GatherQmm {
|
||||||
bits: a, group: b, ..
|
bits: a, group: b, ..
|
||||||
},
|
},
|
||||||
GatherQmmBf16 {
|
GatherQmm {
|
||||||
bits: x, group: y, ..
|
bits: x, group: y, ..
|
||||||
},
|
},
|
||||||
) => (a, b) == (x, y),
|
) => (a, b) == (x, y),
|
||||||
|
|||||||
@@ -299,6 +299,7 @@ fn mtplx_decoder_connected_attention_and_moe_advance_caches() {
|
|||||||
shared_gate: quantized(1, 2560),
|
shared_gate: quantized(1, 2560),
|
||||||
topk: 1,
|
topk: 1,
|
||||||
normalize: true,
|
normalize: true,
|
||||||
|
sorted_gather: true,
|
||||||
fused_decode: false,
|
fused_decode: false,
|
||||||
fused_verify: false,
|
fused_verify: false,
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -313,6 +313,7 @@ impl Execution {
|
|||||||
),
|
),
|
||||||
flag("MTPLX_FUSED_MOE_DECODE"),
|
flag("MTPLX_FUSED_MOE_DECODE"),
|
||||||
flag("MTPLX_FUSED_MOE_VERIFY"),
|
flag("MTPLX_FUSED_MOE_VERIFY"),
|
||||||
|
super::policy::sorted_gather_enabled(get),
|
||||||
)
|
)
|
||||||
})?;
|
})?;
|
||||||
if !lazy {
|
if !lazy {
|
||||||
|
|||||||
@@ -44,7 +44,7 @@ fn unit_gather(rows: i32, streams: &Streams) -> Array {
|
|||||||
.unwrap(),
|
.unwrap(),
|
||||||
)
|
)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
linear::guarded_gather_qmm_bf16(
|
linear::guarded_gather_qmm(
|
||||||
&x,
|
&x,
|
||||||
[&w, &s, &b],
|
[&w, &s, &b],
|
||||||
[None, Some(&rhs)],
|
[None, Some(&rhs)],
|
||||||
@@ -141,7 +141,7 @@ fn issue110_actual_projection_rows_match_oracle() {
|
|||||||
)
|
)
|
||||||
.unwrap()
|
.unwrap()
|
||||||
});
|
});
|
||||||
let out = linear::guarded_gather_qmm_bf16(
|
let out = linear::guarded_gather_qmm(
|
||||||
&x,
|
&x,
|
||||||
[&layer[0], &layer[1], &layer[2]],
|
[&layer[0], &layer[1], &layer[2]],
|
||||||
[lhs.as_ref(), Some(&rhs)],
|
[lhs.as_ref(), Some(&rhs)],
|
||||||
@@ -176,7 +176,7 @@ pub(super) fn record(rows: i32, pad: i32, k: i32, n: i32) {
|
|||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
#[ignore = "full installed model and large partial chunks; serial memory supervisor required"]
|
#[ignore = "full installed model and large partial chunks; serial memory supervisor required"]
|
||||||
fn issue110_large_cold_and_warm_prefill() {
|
pub(super) fn issue110_large_cold_and_warm_prefill() {
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
std::env::var("MTPLX_GDN_BOUNDARY_TAIL_INTERVAL").as_deref(),
|
std::env::var("MTPLX_GDN_BOUNDARY_TAIL_INTERVAL").as_deref(),
|
||||||
Ok("0")
|
Ok("0")
|
||||||
|
|||||||
@@ -0,0 +1,180 @@
|
|||||||
|
//! Paired, serial installed-model prefill timings; state reads follow timing.
|
||||||
|
use super::execution::Execution;
|
||||||
|
use super::sampling::SamplerConfig;
|
||||||
|
use super::session_cache::{DecodeOptions, TurnProgress};
|
||||||
|
use std::cell::Cell;
|
||||||
|
use std::time::Instant;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
#[ignore = "full installed model; serial memory supervisor and explicit output directory"]
|
||||||
|
fn issue112_paired_prefill() {
|
||||||
|
let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
|
||||||
|
let directory = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE112_DIR").unwrap());
|
||||||
|
std::fs::create_dir_all(&directory).unwrap();
|
||||||
|
let repeats = std::env::var("DS4_ISSUE112_REPEATS")
|
||||||
|
.ok()
|
||||||
|
.map(|n| n.parse::<usize>().unwrap())
|
||||||
|
.unwrap_or(3);
|
||||||
|
let settings = crate::settings::effective_settings(
|
||||||
|
crate::model::ModelChoice::Qwen38FlashNext,
|
||||||
|
&crate::settings::GenerationPreferences {
|
||||||
|
context_tokens: 16384,
|
||||||
|
max_generated_tokens: 1,
|
||||||
|
temperature: Some(0.),
|
||||||
|
..Default::default()
|
||||||
|
},
|
||||||
|
&crate::settings::RuntimePreferences::default(),
|
||||||
|
root.parent().unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let (mut execution, mut options) = Execution::open(&settings.engine).unwrap();
|
||||||
|
let sampler = SamplerConfig {
|
||||||
|
temperature: 0.,
|
||||||
|
..Default::default()
|
||||||
|
};
|
||||||
|
options.sampler = sampler;
|
||||||
|
options.draft_sampler = sampler;
|
||||||
|
options.prefill.return_hidden = false;
|
||||||
|
options.decode = DecodeOptions::Ar(super::ar::Options {
|
||||||
|
max_tokens: 1,
|
||||||
|
seed: 1,
|
||||||
|
stop_tokens: &[],
|
||||||
|
sync_eval: false,
|
||||||
|
return_hidden: false,
|
||||||
|
capture_final_state: true,
|
||||||
|
});
|
||||||
|
options.store_min_suffix = None;
|
||||||
|
let tokens = |len: usize| {
|
||||||
|
(0..len)
|
||||||
|
.map(|i| [9707, 11, 1075, 374, 264, 1092, 13][i % 7])
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
};
|
||||||
|
for chunk in [2048, 4096] {
|
||||||
|
options.prefill.chunk_size = Some(chunk);
|
||||||
|
for rep in 0..repeats {
|
||||||
|
let mut prompt = tokens(4096);
|
||||||
|
let mut saved = None;
|
||||||
|
for phase in ["cold", "warm"] {
|
||||||
|
let start = Instant::now();
|
||||||
|
let ttft = Cell::new(None);
|
||||||
|
let out = execution
|
||||||
|
.session_turn(
|
||||||
|
&prompt,
|
||||||
|
saved.as_ref(),
|
||||||
|
&options,
|
||||||
|
|event| {
|
||||||
|
if let TurnProgress::Prefill(super::prefill::Progress::Settled {
|
||||||
|
tokens,
|
||||||
|
total,
|
||||||
|
}) = event
|
||||||
|
{
|
||||||
|
eprintln!("#112 {chunk}/{rep}/{phase} {tokens}/{total}");
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
},
|
||||||
|
|_| {
|
||||||
|
if ttft.get().is_none() {
|
||||||
|
ttft.set(Some(start.elapsed().as_secs_f64()));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
},
|
||||||
|
|_| Ok(()),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let elapsed = start.elapsed().as_secs_f64();
|
||||||
|
assert!(out.final_snapshot_error.is_none());
|
||||||
|
let snapshot = out.snapshot.unwrap();
|
||||||
|
let state = super::issue104::state(&execution, &snapshot);
|
||||||
|
let hidden_state=snapshot.hidden.as_ref().map(|a| {
|
||||||
|
let values=execution.values_f32(a).unwrap();
|
||||||
|
assert!(values.iter().all(|v|v.is_finite()));
|
||||||
|
serde_json::json!({"shape":a.layout().shape(),"sha256":super::super::qwen_mtplx_tests::hex_digest(&values.iter().flat_map(|v|v.to_le_bytes()).collect::<Vec<_>>())})
|
||||||
|
});
|
||||||
|
assert!(hidden_state.is_none());
|
||||||
|
let metrics = serde_json::json!({"chunk":chunk,"rep":rep,"phase":phase,"prompt":prompt.len(),"cached":out.cached_tokens,"ttft_seconds":ttft.get().unwrap(),"prefill_seconds":out.prefill_seconds,"prefill_tokens_per_second":(prompt.len()-out.cached_tokens) as f64/out.prefill_seconds,"elapsed_seconds":elapsed});
|
||||||
|
let record =
|
||||||
|
serde_json::json!({"metrics":metrics,"state":state,"hidden":hidden_state});
|
||||||
|
std::fs::write(
|
||||||
|
directory.join(format!("{chunk}-{rep}-{phase}.json")),
|
||||||
|
serde_json::to_vec(&record).unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
println!(
|
||||||
|
"{}",
|
||||||
|
serde_json::json!({"case":"paired_prefill","metrics":metrics})
|
||||||
|
);
|
||||||
|
prompt = snapshot.token_ids.clone();
|
||||||
|
prompt.extend(tokens(2048));
|
||||||
|
saved = Some(snapshot);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
#[ignore = "full installed model hidden-state prefill; serial supervisor required"]
|
||||||
|
fn issue112_hidden_prefill() {
|
||||||
|
let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
|
||||||
|
let directory = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE112_DIR").unwrap());
|
||||||
|
std::fs::create_dir_all(&directory).unwrap();
|
||||||
|
let mut settings = crate::settings::effective_settings(
|
||||||
|
crate::model::ModelChoice::Qwen38FlashNext,
|
||||||
|
&crate::settings::GenerationPreferences {
|
||||||
|
context_tokens: 16384,
|
||||||
|
max_generated_tokens: 1,
|
||||||
|
..Default::default()
|
||||||
|
},
|
||||||
|
&crate::settings::RuntimePreferences::default(),
|
||||||
|
root.parent().unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
settings.engine.speculative.glm_mtp = true;
|
||||||
|
let (mut execution, mut options) = Execution::open(&settings.engine).unwrap();
|
||||||
|
options.decode = DecodeOptions::Ar(super::ar::Options {
|
||||||
|
max_tokens: 1,
|
||||||
|
seed: 1,
|
||||||
|
stop_tokens: &[],
|
||||||
|
sync_eval: false,
|
||||||
|
return_hidden: true,
|
||||||
|
capture_final_state: true,
|
||||||
|
});
|
||||||
|
options.prefill.return_hidden = true;
|
||||||
|
options.store_min_suffix = None;
|
||||||
|
let tokens = |len: usize| {
|
||||||
|
(0..len)
|
||||||
|
.map(|i| [9707, 11, 1075, 374, 264, 1092, 13][i % 7])
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
};
|
||||||
|
for chunk in [2048, 4096] {
|
||||||
|
options.prefill.chunk_size = Some(chunk);
|
||||||
|
let mut saved = None;
|
||||||
|
let mut prompt = tokens(4096);
|
||||||
|
for phase in ["cold", "warm"] {
|
||||||
|
let snapshot = execution
|
||||||
|
.prefill_session(&prompt, saved.as_ref(), &options, |event| {
|
||||||
|
if let super::prefill::Progress::Settled { tokens, total } = event {
|
||||||
|
eprintln!("#112 hidden {chunk}/{phase} {tokens}/{total}");
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
})
|
||||||
|
.unwrap();
|
||||||
|
let state = super::issue104::state(&execution, &snapshot);
|
||||||
|
let hidden = snapshot.hidden.as_ref().unwrap();
|
||||||
|
let values = execution.values_f32(hidden).unwrap();
|
||||||
|
assert!(values.iter().all(|v| v.is_finite()));
|
||||||
|
let hidden = serde_json::json!({"shape":hidden.layout().shape(),"sha256":super::super::qwen_mtplx_tests::hex_digest(&values.iter().flat_map(|v|v.to_le_bytes()).collect::<Vec<_>>())});
|
||||||
|
let record = serde_json::json!({"case":"hidden_prefill","chunk":chunk,"phase":phase,"cached":saved.as_ref().map_or(0,|s|s.token_ids.len()),"state":state,"hidden":hidden});
|
||||||
|
std::fs::write(
|
||||||
|
directory.join(format!("{chunk}-{phase}.json")),
|
||||||
|
serde_json::to_vec(&record).unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
println!(
|
||||||
|
"{}",
|
||||||
|
serde_json::json!({"case":"hidden_prefill","chunk":chunk,"phase":phase,"hidden":hidden})
|
||||||
|
);
|
||||||
|
prompt.extend(tokens(2048));
|
||||||
|
saved = Some(snapshot);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -371,7 +371,7 @@ fn sorted_gather_guard_matches_pinned_call_predicates() {
|
|||||||
|
|
||||||
/// Guard both stock Gate/Up and Down gathers, including future fused-path
|
/// Guard both stock Gate/Up and Down gathers, including future fused-path
|
||||||
/// fallbacks. Keep the underlying runtime primitive unchanged.
|
/// fallbacks. Keep the underlying runtime primitive unchanged.
|
||||||
pub(super) fn guarded_gather_qmm_bf16(
|
pub(super) fn guarded_gather_qmm(
|
||||||
x: &Array,
|
x: &Array,
|
||||||
weights: [&Array; 3],
|
weights: [&Array; 3],
|
||||||
indices: [Option<&Array>; 2],
|
indices: [Option<&Array>; 2],
|
||||||
@@ -390,7 +390,7 @@ pub(super) fn guarded_gather_qmm_bf16(
|
|||||||
super::sparse_attention::nax_available(),
|
super::sparse_attention::nax_available(),
|
||||||
);
|
);
|
||||||
if pad == 0 {
|
if pad == 0 {
|
||||||
return gather_qmm_bf16(x, weights, indices, quant, sorted, streams, stream);
|
return gather_qmm(x, weights, indices, quant, sorted, streams, stream);
|
||||||
}
|
}
|
||||||
let rows = shape[0];
|
let rows = shape[0];
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
@@ -428,7 +428,7 @@ pub(super) fn guarded_gather_qmm_bf16(
|
|||||||
ops::concatenate(&[lhs.clone(), tail], 0, stream)
|
ops::concatenate(&[lhs.clone(), tail], 0, stream)
|
||||||
})
|
})
|
||||||
.transpose()?;
|
.transpose()?;
|
||||||
let out = gather_qmm_bf16(
|
let out = gather_qmm(
|
||||||
&x,
|
&x,
|
||||||
weights,
|
weights,
|
||||||
[lhs.as_ref(), Some(&rhs)],
|
[lhs.as_ref(), Some(&rhs)],
|
||||||
@@ -448,9 +448,9 @@ pub(super) fn guarded_gather_qmm_bf16(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Installed affine/BF16, transposed GatherQMM from ops.cpp. Explicit index
|
/// Affine BF16/F16 transposed GatherQMM from ops.cpp. Explicit index
|
||||||
/// casts use the default stream, as indices_or_default does in the reference.
|
/// casts use the default stream, as indices_or_default does in the reference.
|
||||||
pub(super) fn gather_qmm_bf16(
|
pub(super) fn gather_qmm(
|
||||||
x: &Array,
|
x: &Array,
|
||||||
[w, scales, biases]: [&Array; 3],
|
[w, scales, biases]: [&Array; 3],
|
||||||
indices: [Option<&Array>; 2],
|
indices: [Option<&Array>; 2],
|
||||||
@@ -466,17 +466,19 @@ pub(super) fn gather_qmm_bf16(
|
|||||||
let wl = w.layout();
|
let wl = w.layout();
|
||||||
let sl = scales.layout();
|
let sl = scales.layout();
|
||||||
if stream.device() != Device::Gpu
|
if stream.device() != Device::Gpu
|
||||||
|| xl.dtype() != Dtype::BF16
|
|| !matches!(xl.dtype(), Dtype::BF16 | Dtype::F16)
|
||||||
|| wl.dtype() != Dtype::U32
|
|| wl.dtype() != Dtype::U32
|
||||||
|| sl.dtype() != Dtype::BF16
|
|| sl.dtype() != xl.dtype()
|
||||||
|| biases.layout().dtype() != Dtype::BF16
|
|| biases.layout().dtype() != xl.dtype()
|
||||||
|| xl.shape().len() < 2
|
|| xl.shape().len() < 2
|
||||||
|| wl.shape().len() < 2
|
|| wl.shape().len() < 2
|
||||||
|| wl.shape().len() != sl.shape().len()
|
|| wl.shape().len() != sl.shape().len()
|
||||||
|| !matches!(bits, 4 | 8)
|
|| !matches!(bits, 4 | 8)
|
||||||
|| !matches!(group, 32 | 64)
|
|| !matches!(group, 32 | 64 | 128)
|
||||||
{
|
{
|
||||||
return Err("GatherQMM requires BF16 GPU matrices and U32 affine expert weights".into());
|
return Err(
|
||||||
|
"GatherQMM requires BF16/F16 GPU matrices and U32 affine expert weights".into(),
|
||||||
|
);
|
||||||
}
|
}
|
||||||
let wrank = wl.shape().len();
|
let wrank = wl.shape().len();
|
||||||
let k = *xl.shape().last().unwrap() as u32;
|
let k = *xl.shape().last().unwrap() as u32;
|
||||||
@@ -491,7 +493,9 @@ pub(super) fn gather_qmm_bf16(
|
|||||||
{
|
{
|
||||||
return Err("GatherQMM input/weight/group shapes do not match".into());
|
return Err("GatherQMM input/weight/group shapes do not match".into());
|
||||||
}
|
}
|
||||||
affine_plan([m, n, k], bits, group, true)?;
|
if [m, n, k].iter().any(|&d| d == 0 || d > i32::MAX as u32) || !k.is_multiple_of(group) {
|
||||||
|
return Err("invalid GatherQMM shape".into());
|
||||||
|
}
|
||||||
let index = |a: &Array, indices: Option<&Array>| -> Result<Array, String> {
|
let index = |a: &Array, indices: Option<&Array>| -> Result<Array, String> {
|
||||||
if let Some(indices) = indices {
|
if let Some(indices) = indices {
|
||||||
if !matches!(
|
if !matches!(
|
||||||
@@ -530,7 +534,7 @@ pub(super) fn gather_qmm_bf16(
|
|||||||
)?);
|
)?);
|
||||||
let mut shape = inputs[4].layout().shape().to_vec();
|
let mut shape = inputs[4].layout().shape().to_vec();
|
||||||
shape.extend([m as i32, n as i32]);
|
shape.extend([m as i32, n as i32]);
|
||||||
let layout = Layout::new(&shape, Dtype::BF16)?;
|
let layout = Layout::new(&shape, xl.dtype())?;
|
||||||
if layout.size() > i32::MAX as usize {
|
if layout.size() > i32::MAX as usize {
|
||||||
return Err("GatherQMM output size overflow".into());
|
return Err("GatherQMM output size overflow".into());
|
||||||
}
|
}
|
||||||
@@ -538,7 +542,7 @@ pub(super) fn gather_qmm_bf16(
|
|||||||
stream,
|
stream,
|
||||||
&inputs,
|
&inputs,
|
||||||
layout,
|
layout,
|
||||||
Operation::GatherQmmBf16 {
|
Operation::GatherQmm {
|
||||||
bits,
|
bits,
|
||||||
group,
|
group,
|
||||||
right_sorted: sorted && indices[0].is_none(),
|
right_sorted: sorted && indices[0].is_none(),
|
||||||
@@ -575,7 +579,7 @@ pub(super) fn gather_evaluate(
|
|||||||
if prepared[0].layout().size() / k as usize != prepared[5].layout().size() {
|
if prepared[0].layout().size() / k as usize != prepared[5].layout().size() {
|
||||||
let mut shape = prepared[5].layout().shape().to_vec();
|
let mut shape = prepared[5].layout().shape().to_vec();
|
||||||
shape.extend([m as i32, k as i32]);
|
shape.extend([m as i32, k as i32]);
|
||||||
let broadcast = Array::unallocated(&shape, Dtype::BF16)?;
|
let broadcast = Array::unallocated(&shape, prepared[0].layout().dtype())?;
|
||||||
views::broadcast(&prepared[0], &broadcast)?;
|
views::broadcast(&prepared[0], &broadcast)?;
|
||||||
prepared[0] = broadcast;
|
prepared[0] = broadcast;
|
||||||
}
|
}
|
||||||
@@ -596,7 +600,7 @@ pub(super) fn gather_evaluate(
|
|||||||
views::collapse_joint(ll.shape(), &[ll.strides(), rl.strides()]);
|
views::collapse_joint(ll.shape(), &[ll.strides(), rl.strides()]);
|
||||||
let arrays = [w, s, b, x, lhs, rhs, output];
|
let arrays = [w, s, b, x, lhs, rhs, output];
|
||||||
let buffers = arrays.map(Array::buffer);
|
let buffers = arrays.map(Array::buffer);
|
||||||
super::gather_bf16_layout(
|
super::gather_layout(
|
||||||
std::array::from_fn(|i| {
|
std::array::from_fn(|i| {
|
||||||
let binding = super::tensor(i as u32, &buffers[i]).at_byte_offset(arrays[i].offset());
|
let binding = super::tensor(i as u32, &buffers[i]).at_byte_offset(arrays[i].offset());
|
||||||
if i == 6 {
|
if i == 6 {
|
||||||
@@ -605,6 +609,7 @@ pub(super) fn gather_evaluate(
|
|||||||
binding.input(arrays[i].data_size())
|
binding.input(arrays[i].data_size())
|
||||||
}
|
}
|
||||||
}),
|
}),
|
||||||
|
xl.dtype(),
|
||||||
[m, n, k, experts, batches, sorted_rows],
|
[m, n, k, experts, batches, sorted_rows],
|
||||||
[bits, group],
|
[bits, group],
|
||||||
right_sorted,
|
right_sorted,
|
||||||
@@ -1058,7 +1063,7 @@ fn mtplx_graph_gather_qmm_matches_reference() {
|
|||||||
.unwrap();
|
.unwrap();
|
||||||
}
|
}
|
||||||
let (output, construction) = capture_dispatches(|| {
|
let (output, construction) = capture_dispatches(|| {
|
||||||
gather_qmm_bf16(
|
gather_qmm(
|
||||||
&x,
|
&x,
|
||||||
[&w, &s, &b],
|
[&w, &s, &b],
|
||||||
[left.then_some(&lhs), right.then_some(&rhs)],
|
[left.then_some(&lhs), right.then_some(&rhs)],
|
||||||
|
|||||||
@@ -721,7 +721,7 @@ impl TextModel<'_> {
|
|||||||
O::RmsNorm(_) => "RMSNorm",
|
O::RmsNorm(_) => "RMSNorm",
|
||||||
O::Softmax(_) => "Softmax",
|
O::Softmax(_) => "Softmax",
|
||||||
O::QuantizedLinearBf16 { .. } => "QuantizedMatmul",
|
O::QuantizedLinearBf16 { .. } => "QuantizedMatmul",
|
||||||
O::GatherQmmBf16 { .. } => "GatherQMM",
|
O::GatherQmm { .. } => "GatherQMM",
|
||||||
O::DenseMatmul => "Matmul",
|
O::DenseMatmul => "Matmul",
|
||||||
O::Sum(_) => "Sum",
|
O::Sum(_) => "Sum",
|
||||||
O::Sort {
|
O::Sort {
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ fn gather(
|
|||||||
streams: &Streams,
|
streams: &Streams,
|
||||||
stream: Stream,
|
stream: Stream,
|
||||||
) -> Result<Array, String> {
|
) -> Result<Array, String> {
|
||||||
linear::guarded_gather_qmm_bf16(
|
linear::guarded_gather_qmm(
|
||||||
input,
|
input,
|
||||||
[&layer.weight, &layer.scales, &layer.biases],
|
[&layer.weight, &layer.scales, &layer.biases],
|
||||||
[None, Some(ids)],
|
[None, Some(ids)],
|
||||||
@@ -55,13 +55,19 @@ impl FeedForward {
|
|||||||
&self,
|
&self,
|
||||||
input: &Array,
|
input: &Array,
|
||||||
indices: &Array,
|
indices: &Array,
|
||||||
|
sorted_gather: bool,
|
||||||
streams: &Streams,
|
streams: &Streams,
|
||||||
stream: Stream,
|
stream: Stream,
|
||||||
) -> Result<Array, String> {
|
) -> Result<Array, String> {
|
||||||
let mut x = ops::expand_dims(input, &[-2, -3], stream)?;
|
|
||||||
let do_sort = indices.layout().size() >= 64;
|
let do_sort = indices.layout().size() >= 64;
|
||||||
|
let mut x = if do_sort {
|
||||||
|
input.clone()
|
||||||
|
} else {
|
||||||
|
ops::expand_dims(input, &[-2, -3], stream)?
|
||||||
|
};
|
||||||
let mut ids = indices.clone();
|
let mut ids = indices.clone();
|
||||||
let mut inverse = None;
|
let mut inverse = None;
|
||||||
|
let mut row_map = None;
|
||||||
if do_sort {
|
if do_sort {
|
||||||
let size =
|
let size =
|
||||||
i32::try_from(indices.layout().size()).map_err(|_| "MoE index size overflow")?;
|
i32::try_from(indices.layout().size()).map_err(|_| "MoE index size overflow")?;
|
||||||
@@ -76,15 +82,27 @@ impl FeedForward {
|
|||||||
let k = *input.layout().shape().last().ok_or("MoE input rank")?;
|
let k = *input.layout().shape().last().ok_or("MoE input rank")?;
|
||||||
let flat_rows = i32::try_from(input.layout().size() / k as usize)
|
let flat_rows = i32::try_from(input.layout().size() / k as usize)
|
||||||
.map_err(|_| "MoE row count overflow")?;
|
.map_err(|_| "MoE row count overflow")?;
|
||||||
x = indexing::take(
|
x = ops::reshape(&x, &[flat_rows, 1, k], stream)?;
|
||||||
&ops::reshape(&x, &[flat_rows, 1, k], stream)?,
|
row_map = Some(rows);
|
||||||
&rows,
|
|
||||||
0,
|
|
||||||
stream,
|
|
||||||
)?;
|
|
||||||
ids = indexing::take(&ids, &order, 0, stream)?;
|
ids = indexing::take(&ids, &order, 0, stream)?;
|
||||||
}
|
}
|
||||||
let hidden = match &self.input {
|
let hidden = if let Some(map) = row_map {
|
||||||
|
let (gate, up) = match &self.input {
|
||||||
|
GateUp::Fused(gu) => (gu, None),
|
||||||
|
GateUp::Separate { gate, up } => (gate, Some(up)),
|
||||||
|
};
|
||||||
|
super::moe_sorted::swiglu_rows(
|
||||||
|
&x,
|
||||||
|
&map,
|
||||||
|
gate,
|
||||||
|
up,
|
||||||
|
&ids,
|
||||||
|
sorted_gather,
|
||||||
|
streams,
|
||||||
|
stream,
|
||||||
|
)?
|
||||||
|
} else {
|
||||||
|
match &self.input {
|
||||||
GateUp::Fused(gu) => {
|
GateUp::Fused(gu) => {
|
||||||
let gu = gather(&x, gu, &ids, do_sort, streams, stream)?;
|
let gu = gather(&x, gu, &ids, do_sort, streams, stream)?;
|
||||||
let n = *gu.layout().shape().last().ok_or("MoE gate/up rank")?;
|
let n = *gu.layout().shape().last().ok_or("MoE gate/up rank")?;
|
||||||
@@ -104,6 +122,7 @@ impl FeedForward {
|
|||||||
let gate = gather(&x, gate, &ids, do_sort, streams, stream)?;
|
let gate = gather(&x, gate, &ids, do_sort, streams, stream)?;
|
||||||
mlp::swiglu(&gate, &up, stream)?
|
mlp::swiglu(&gate, &up, stream)?
|
||||||
}
|
}
|
||||||
|
}
|
||||||
};
|
};
|
||||||
let mut output = gather(&hidden, &self.down, &ids, do_sort, streams, stream)?;
|
let mut output = gather(&hidden, &self.down, &ids, do_sort, streams, stream)?;
|
||||||
if let Some(inverse) = inverse {
|
if let Some(inverse) = inverse {
|
||||||
@@ -123,6 +142,7 @@ pub(super) struct SparseMoe {
|
|||||||
pub(super) shared_gate: QuantizedLinear,
|
pub(super) shared_gate: QuantizedLinear,
|
||||||
pub(super) topk: i32,
|
pub(super) topk: i32,
|
||||||
pub(super) normalize: bool,
|
pub(super) normalize: bool,
|
||||||
|
pub(super) sorted_gather: bool,
|
||||||
pub(super) fused_decode: bool,
|
pub(super) fused_decode: bool,
|
||||||
pub(super) fused_verify: bool,
|
pub(super) fused_verify: bool,
|
||||||
}
|
}
|
||||||
@@ -199,7 +219,9 @@ impl SparseMoe {
|
|||||||
stream,
|
stream,
|
||||||
)?
|
)?
|
||||||
} else {
|
} else {
|
||||||
let switched = self.experts.switched(input, &indices, streams, stream)?;
|
let switched =
|
||||||
|
self.experts
|
||||||
|
.switched(input, &indices, self.sorted_gather, streams, stream)?;
|
||||||
let scores = ops::expand_dims(&scores, &[-1], stream)?;
|
let scores = ops::expand_dims(&scores, &[-1], stream)?;
|
||||||
reduce::sum(
|
reduce::sum(
|
||||||
&binary(&switched, &scores, Binary::Multiply, stream)?,
|
&binary(&switched, &scores, Binary::Multiply, stream)?,
|
||||||
@@ -413,6 +435,7 @@ fn mtplx_graph_sparse_moe_matches_reference() {
|
|||||||
shared_gate: mlp::fixture_linear(1, k, 4, 64, 52),
|
shared_gate: mlp::fixture_linear(1, k, 4, 64, 52),
|
||||||
topk: 10,
|
topk: 10,
|
||||||
normalize: row["normalize"].as_bool().unwrap(),
|
normalize: row["normalize"].as_bool().unwrap(),
|
||||||
|
sorted_gather: true,
|
||||||
fused_decode: row["fused"].as_bool().unwrap_or(false),
|
fused_decode: row["fused"].as_bool().unwrap_or(false),
|
||||||
fused_verify: row["fused"].as_bool().unwrap_or(false),
|
fused_verify: row["fused"].as_bool().unwrap_or(false),
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -0,0 +1,839 @@
|
|||||||
|
//! MTPLX 9882703 expert row-map gather, including whole-first-call canaries.
|
||||||
|
use super::array::{Array, Dtype, Layout};
|
||||||
|
use super::binary::{Binary, binary, floor_divide_integer};
|
||||||
|
use super::mlp::{self, QuantizedLinear};
|
||||||
|
use super::stream::{Stream, Streams};
|
||||||
|
use super::{indexing, linear, ops};
|
||||||
|
use std::collections::BTreeMap;
|
||||||
|
use std::sync::{LazyLock, Mutex};
|
||||||
|
|
||||||
|
#[derive(Default, Clone, Copy, Debug)]
|
||||||
|
pub(super) struct Stats {
|
||||||
|
pub(super) calls: u64,
|
||||||
|
pub(super) fallbacks: u64,
|
||||||
|
pub(super) canaries: u64,
|
||||||
|
pub(super) failures: u64,
|
||||||
|
}
|
||||||
|
static CANARY: LazyLock<Mutex<(BTreeMap<String, bool>, Stats)>> =
|
||||||
|
LazyLock::new(|| Mutex::new((BTreeMap::new(), Stats::default())));
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
pub(super) fn stats() -> Stats {
|
||||||
|
CANARY.lock().unwrap().1
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
static FAILURE: std::sync::atomic::AtomicU8 = std::sync::atomic::AtomicU8::new(0);
|
||||||
|
|
||||||
|
#[derive(Clone, Copy, Debug)]
|
||||||
|
pub(super) struct Kernel {
|
||||||
|
dtype: Dtype,
|
||||||
|
bits: u32,
|
||||||
|
group: u32,
|
||||||
|
k: i32,
|
||||||
|
n: i32,
|
||||||
|
experts: i32,
|
||||||
|
swiglu: bool,
|
||||||
|
separate: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Kernel {
|
||||||
|
fn width(self) -> i32 {
|
||||||
|
if self.swiglu && !self.separate {
|
||||||
|
self.n / 2
|
||||||
|
} else {
|
||||||
|
self.n
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fn name(self) -> String {
|
||||||
|
format!(
|
||||||
|
"kernel_qwen_mtplx_sorted_{}_g{}_b{}_k{}_n{}_e{}_s{}_u{}",
|
||||||
|
self.dtype.kernel_name(),
|
||||||
|
self.group,
|
||||||
|
self.bits,
|
||||||
|
self.k,
|
||||||
|
self.n,
|
||||||
|
self.experts,
|
||||||
|
u8::from(self.swiglu),
|
||||||
|
u8::from(self.separate)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
fn source(self) -> String {
|
||||||
|
let ty = if self.dtype == Dtype::BF16 {
|
||||||
|
"bfloat16_t"
|
||||||
|
} else {
|
||||||
|
"float16_t"
|
||||||
|
};
|
||||||
|
let params = [
|
||||||
|
("x", ty),
|
||||||
|
("w", "uint"),
|
||||||
|
("scales", ty),
|
||||||
|
("biases", ty),
|
||||||
|
("wu", "uint"),
|
||||||
|
("scales_u", ty),
|
||||||
|
("biases_u", ty),
|
||||||
|
("tile_start", "int"),
|
||||||
|
("row_start", "int"),
|
||||||
|
("row_map", "uint"),
|
||||||
|
]
|
||||||
|
.iter()
|
||||||
|
.enumerate()
|
||||||
|
.map(|(i, (name, ty))| format!("const device {ty}* {name} [[buffer({i})]]"))
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join(",\n");
|
||||||
|
format!(
|
||||||
|
"{}\nusing T={ty};\nconstant int GS={}, BITS={}, KD={}, ND={}, UD={}, UOFF={}, OUTC={}, ED={};\nconstant bool SWIGLU={};\nkernel void {}({params}, device T* y [[buffer(10)]], uint3 threadgroup_position_in_grid [[threadgroup_position_in_grid]], uint simdgroup_index_in_threadgroup [[simdgroup_index_in_threadgroup]], uint thread_index_in_simdgroup [[thread_index_in_simdgroup]]) {{\n{}\n}}",
|
||||||
|
include_str!("../../../../metal/mtplx-moe-sorted-header.metal"),
|
||||||
|
self.group,
|
||||||
|
self.bits,
|
||||||
|
self.k,
|
||||||
|
self.n,
|
||||||
|
self.n,
|
||||||
|
if self.swiglu && !self.separate {
|
||||||
|
self.n / 2
|
||||||
|
} else {
|
||||||
|
0
|
||||||
|
},
|
||||||
|
self.width(),
|
||||||
|
self.experts,
|
||||||
|
self.swiglu,
|
||||||
|
self.name(),
|
||||||
|
include_str!("../../../../metal/mtplx-moe-sorted-body.metal")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
pub(super) fn evaluate(self, inputs: &[Array], outputs: &[Array]) -> Result<(), String> {
|
||||||
|
super::compile_custom(&self.name(), || {
|
||||||
|
#[cfg(test)]
|
||||||
|
if FAILURE.load(std::sync::atomic::Ordering::Relaxed) == 1 {
|
||||||
|
return Ok("injected invalid Metal source".into());
|
||||||
|
}
|
||||||
|
Ok(self.source())
|
||||||
|
})?;
|
||||||
|
let rows = outputs[0].layout().shape()[0] as u32;
|
||||||
|
let cols = if self.swiglu {
|
||||||
|
self.width() * 2
|
||||||
|
} else {
|
||||||
|
self.n
|
||||||
|
} as u32;
|
||||||
|
super::gdn::custom_evaluate(inputs, outputs, |bindings| {
|
||||||
|
super::dispatch_geometry(
|
||||||
|
&self.name(),
|
||||||
|
bindings,
|
||||||
|
&[],
|
||||||
|
[
|
||||||
|
(cols / 64) * 128,
|
||||||
|
rows.div_ceil(64) + self.experts as u32,
|
||||||
|
1,
|
||||||
|
],
|
||||||
|
[128, 1, 1],
|
||||||
|
true,
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Applies mirrors the affine tensor/layout predicates. QuantizedLinear always
|
||||||
|
// carries biases and affine mode; separate packs must also share quantization.
|
||||||
|
pub(super) fn applies(
|
||||||
|
tokens: &Array,
|
||||||
|
map: &Array,
|
||||||
|
gate: &QuantizedLinear,
|
||||||
|
up: Option<&QuantizedLinear>,
|
||||||
|
ids: &Array,
|
||||||
|
enabled: bool,
|
||||||
|
nax: bool,
|
||||||
|
) -> bool {
|
||||||
|
if let Some(up) = up
|
||||||
|
&& (gate.bits != up.bits
|
||||||
|
|| gate.group != up.group
|
||||||
|
|| [&gate.weight, &gate.scales, &gate.biases]
|
||||||
|
.iter()
|
||||||
|
.zip([&up.weight, &up.scales, &up.biases])
|
||||||
|
.any(|(a, b)| {
|
||||||
|
a.layout().shape() != b.layout().shape()
|
||||||
|
|| a.layout().dtype() != b.layout().dtype()
|
||||||
|
}))
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
let x = tokens.layout();
|
||||||
|
let w = gate.weight.layout();
|
||||||
|
let s = gate.scales.layout();
|
||||||
|
let b = gate.biases.layout();
|
||||||
|
let r = map.layout();
|
||||||
|
let i = ids.layout();
|
||||||
|
enabled
|
||||||
|
&& nax
|
||||||
|
&& matches!(gate.bits, 4 | 8)
|
||||||
|
&& matches!(gate.group, 32 | 64 | 128)
|
||||||
|
&& matches!(x.dtype(), Dtype::BF16 | Dtype::F16)
|
||||||
|
&& x.shape().len() == 3
|
||||||
|
&& x.shape()[1] == 1
|
||||||
|
&& s.dtype() == x.dtype()
|
||||||
|
&& b.dtype() == x.dtype()
|
||||||
|
&& i.shape().len() == 1
|
||||||
|
&& r.shape() == i.shape()
|
||||||
|
&& i.size() >= 4096
|
||||||
|
&& r.dtype() == Dtype::U32
|
||||||
|
&& w.shape().len() == 3
|
||||||
|
&& w.dtype() == Dtype::U32
|
||||||
|
&& w.shape()[1] % 64 == 0
|
||||||
|
&& (w.shape()[2] * 32 / gate.bits as i32) % 64 == 0
|
||||||
|
&& x.shape()[2] == w.shape()[2] * 32 / gate.bits as i32
|
||||||
|
&& x.shape()[2] % gate.group as i32 == 0
|
||||||
|
}
|
||||||
|
|
||||||
|
fn scalar(n: i32) -> Result<Array, String> {
|
||||||
|
Array::new(&[], Dtype::I32, super::scalar_buffer(&n.to_le_bytes())?)
|
||||||
|
}
|
||||||
|
fn schedule(ids: &Array, experts: i32, stream: Stream) -> Result<(Array, Array), String> {
|
||||||
|
if !matches!(
|
||||||
|
ids.layout().dtype(),
|
||||||
|
Dtype::U8
|
||||||
|
| Dtype::U16
|
||||||
|
| Dtype::U32
|
||||||
|
| Dtype::U64
|
||||||
|
| Dtype::I8
|
||||||
|
| Dtype::I16
|
||||||
|
| Dtype::I32
|
||||||
|
| Dtype::I64
|
||||||
|
) {
|
||||||
|
return Err("MoE expert indices must be integral".into());
|
||||||
|
}
|
||||||
|
let ids = ops::contiguous(&ops::astype(ids, Dtype::U32, false, stream)?, false, stream)?;
|
||||||
|
let zero = ops::zeros(&[experts], Dtype::I32, stream)?;
|
||||||
|
let counts = Array::make_operation_with_inputs(
|
||||||
|
stream,
|
||||||
|
&[zero, ids.clone()],
|
||||||
|
Layout::new(&[experts], Dtype::I32)?,
|
||||||
|
ops::Operation::MoeSortedCounts,
|
||||||
|
)?;
|
||||||
|
let row_start = ops::concatenate(
|
||||||
|
&[
|
||||||
|
ops::zeros(&[1], Dtype::I32, stream)?,
|
||||||
|
indexing::cumsum_last_i32(&counts, stream)?,
|
||||||
|
],
|
||||||
|
0,
|
||||||
|
stream,
|
||||||
|
)?;
|
||||||
|
let tiles = floor_divide_integer(
|
||||||
|
&binary(&counts, &scalar(63)?, Binary::Add, stream)?,
|
||||||
|
&scalar(64)?,
|
||||||
|
stream,
|
||||||
|
)?;
|
||||||
|
let tile_start = ops::concatenate(
|
||||||
|
&[
|
||||||
|
ops::zeros(&[1], Dtype::I32, stream)?,
|
||||||
|
indexing::cumsum_last_i32(&tiles, stream)?,
|
||||||
|
],
|
||||||
|
0,
|
||||||
|
stream,
|
||||||
|
)?;
|
||||||
|
Ok((tile_start, row_start))
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(super) fn counts_evaluate(inputs: &[Array], output: &Array) -> Result<(), String> {
|
||||||
|
// Same zeros.at[ids].add(1) schedule, with integer atomic sums on the GPU.
|
||||||
|
// The pinned library does not export this integer scatter specialization.
|
||||||
|
const NAME: &str = "kernel_qwen_mtplx_sorted_counts";
|
||||||
|
super::compile_custom(NAME, || {
|
||||||
|
Ok("#include <metal_stdlib>\nusing namespace metal;\nkernel void kernel_qwen_mtplx_sorted_counts(const device uint* ids [[buffer(0)]], device atomic_int* counts [[buffer(1)]], constant uint& size [[buffer(2)]],uint i [[thread_position_in_grid]]) { if(i<size) atomic_fetch_add_explicit(counts+ids[i],1,memory_order_relaxed); }".into())
|
||||||
|
})?;
|
||||||
|
super::views::copy_gpu(&inputs[0], output, true)?;
|
||||||
|
let ids = &inputs[1];
|
||||||
|
let n = ids.layout().size() as u32;
|
||||||
|
super::dispatch_geometry(
|
||||||
|
NAME,
|
||||||
|
&[
|
||||||
|
super::tensor(0, &ids.buffer())
|
||||||
|
.at_byte_offset(ids.offset())
|
||||||
|
.input(ids.data_size()),
|
||||||
|
super::tensor(1, &output.buffer()).output(output.data_size()),
|
||||||
|
super::bytes(2, &n),
|
||||||
|
],
|
||||||
|
&[],
|
||||||
|
[n, 1, 1],
|
||||||
|
[n.min(1024), 1, 1],
|
||||||
|
true,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
fn launch(
|
||||||
|
tokens: &Array,
|
||||||
|
map: &Array,
|
||||||
|
gate: &QuantizedLinear,
|
||||||
|
up: Option<&QuantizedLinear>,
|
||||||
|
ids: &Array,
|
||||||
|
kernel: Kernel,
|
||||||
|
stream: Stream,
|
||||||
|
) -> Result<Array, String> {
|
||||||
|
let (tiles, rows) = schedule(ids, kernel.experts, stream)?;
|
||||||
|
let u = up.unwrap_or(gate);
|
||||||
|
let input = ops::reshape(tokens, &[-1, kernel.k], stream)?;
|
||||||
|
let y = Array::make_operation_with_inputs(
|
||||||
|
stream,
|
||||||
|
&[
|
||||||
|
input,
|
||||||
|
gate.weight.clone(),
|
||||||
|
gate.scales.clone(),
|
||||||
|
gate.biases.clone(),
|
||||||
|
u.weight.clone(),
|
||||||
|
u.scales.clone(),
|
||||||
|
u.biases.clone(),
|
||||||
|
tiles,
|
||||||
|
rows,
|
||||||
|
map.clone(),
|
||||||
|
],
|
||||||
|
Layout::new(&[ids.layout().shape()[0], kernel.width()], kernel.dtype)?,
|
||||||
|
ops::Operation::MoeSortedGather(kernel),
|
||||||
|
)?;
|
||||||
|
ops::reshape(&y, &[ids.layout().shape()[0], 1, kernel.width()], stream)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn gather(
|
||||||
|
x: &Array,
|
||||||
|
q: &QuantizedLinear,
|
||||||
|
ids: &Array,
|
||||||
|
streams: &Streams,
|
||||||
|
stream: Stream,
|
||||||
|
) -> Result<Array, String> {
|
||||||
|
linear::guarded_gather_qmm(
|
||||||
|
x,
|
||||||
|
[&q.weight, &q.scales, &q.biases],
|
||||||
|
[None, Some(ids)],
|
||||||
|
[q.bits, q.group],
|
||||||
|
true,
|
||||||
|
streams,
|
||||||
|
stream,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn activation(gate: &Array, up: &Array, separate: bool, stream: Stream) -> Result<Array, String> {
|
||||||
|
if gate.layout().dtype() == Dtype::BF16 {
|
||||||
|
return if separate {
|
||||||
|
mlp::swiglu(gate, up, stream)
|
||||||
|
} else {
|
||||||
|
binary(&mlp::silu(gate, stream)?, up, Binary::Multiply, stream)
|
||||||
|
};
|
||||||
|
}
|
||||||
|
let graph = super::compiled::Replay::trace_compiled(&[gate.clone(), up.clone()], |a| {
|
||||||
|
let sig = super::unary::unary(&a[0], super::unary::Unary::Sigmoid, stream)?;
|
||||||
|
let silu = binary(&a[0], &sig, Binary::Multiply, stream)?;
|
||||||
|
Ok(vec![if separate {
|
||||||
|
binary(&silu, &a[1], Binary::Multiply, stream)?
|
||||||
|
} else {
|
||||||
|
silu
|
||||||
|
}])
|
||||||
|
})?;
|
||||||
|
let result = graph.run(&[gate.clone(), up.clone()])?.remove(0);
|
||||||
|
if separate {
|
||||||
|
Ok(result)
|
||||||
|
} else {
|
||||||
|
binary(&result, up, Binary::Multiply, stream)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
fn stock(
|
||||||
|
tokens: &Array,
|
||||||
|
map: &Array,
|
||||||
|
gate: &QuantizedLinear,
|
||||||
|
up: Option<&QuantizedLinear>,
|
||||||
|
ids: &Array,
|
||||||
|
swiglu: bool,
|
||||||
|
streams: &Streams,
|
||||||
|
stream: Stream,
|
||||||
|
) -> Result<Array, String> {
|
||||||
|
let x = indexing::take(tokens, map, 0, stream)?;
|
||||||
|
let y = gather(&x, gate, ids, streams, stream)?;
|
||||||
|
if !swiglu {
|
||||||
|
return Ok(y);
|
||||||
|
}
|
||||||
|
if let Some(up) = up {
|
||||||
|
activation(&y, &gather(&x, up, ids, streams, stream)?, true, stream)
|
||||||
|
} else {
|
||||||
|
let pair = ops::split(&y, &[y.layout().dim(-1)? / 2], -1, stream)?;
|
||||||
|
activation(&pair[0], &pair[1], false, stream)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
fn checked(
|
||||||
|
tokens: &Array,
|
||||||
|
map: &Array,
|
||||||
|
gate: &QuantizedLinear,
|
||||||
|
up: Option<&QuantizedLinear>,
|
||||||
|
ids: &Array,
|
||||||
|
swiglu: bool,
|
||||||
|
streams: &Streams,
|
||||||
|
stream: Stream,
|
||||||
|
) -> Result<Option<Array>, String> {
|
||||||
|
let kernel = Kernel {
|
||||||
|
dtype: tokens.layout().dtype(),
|
||||||
|
bits: gate.bits,
|
||||||
|
group: gate.group,
|
||||||
|
k: tokens.layout().dim(-1)?,
|
||||||
|
n: gate.weight.layout().dim(-2)?,
|
||||||
|
experts: gate.weight.layout().shape()[0],
|
||||||
|
swiglu,
|
||||||
|
separate: up.is_some(),
|
||||||
|
};
|
||||||
|
let key = format!(
|
||||||
|
"{}_{}",
|
||||||
|
kernel.name(),
|
||||||
|
if ids.layout().size() >= 64 * kernel.experts as usize {
|
||||||
|
"wide"
|
||||||
|
} else {
|
||||||
|
"narrow"
|
||||||
|
}
|
||||||
|
);
|
||||||
|
let mut state = CANARY.lock().map_err(|_| "MoE canary lock poisoned")?;
|
||||||
|
if state.0.get(&key) == Some(&false) {
|
||||||
|
state.1.fallbacks += 1;
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
if state.0.get(&key) == Some(&true) {
|
||||||
|
state.1.calls += 1;
|
||||||
|
drop(state);
|
||||||
|
return launch(tokens, map, gate, up, ids, kernel, stream).map(Some);
|
||||||
|
}
|
||||||
|
state.1.canaries += 1;
|
||||||
|
let probe: Result<Array, String> = (|| {
|
||||||
|
let ours = launch(tokens, map, gate, up, ids, kernel, stream)?;
|
||||||
|
let reference = stock(tokens, map, gate, up, ids, swiglu, streams, stream)?;
|
||||||
|
ops::evaluate(streams, &[ours.clone(), reference.clone()], stream, false)?;
|
||||||
|
#[cfg(test)]
|
||||||
|
if FAILURE.load(std::sync::atomic::Ordering::Relaxed) == 2 {
|
||||||
|
let mut bits = [0u8; 2];
|
||||||
|
ours.buffer().read(ours.offset(), &mut bits)?;
|
||||||
|
bits[1] ^= 0x80;
|
||||||
|
ours.buffer().write(ours.offset(), &bits)?;
|
||||||
|
}
|
||||||
|
#[cfg(test)]
|
||||||
|
if FAILURE.load(std::sync::atomic::Ordering::Relaxed) == 3 {
|
||||||
|
ours.buffer().write(ours.offset(), &[0, 128])?;
|
||||||
|
reference.buffer().write(reference.offset(), &[0, 0])?;
|
||||||
|
}
|
||||||
|
let equal = binary(
|
||||||
|
&ops::view(&ours, Dtype::U16, stream)?,
|
||||||
|
&ops::view(&reference, Dtype::U16, stream)?,
|
||||||
|
Binary::Equal,
|
||||||
|
stream,
|
||||||
|
)?;
|
||||||
|
let different = super::unary::unary(&equal, super::unary::Unary::LogicalNot, stream)?;
|
||||||
|
let mismatch = super::reduce::any(&different, &[0, 1, 2], false, stream)?;
|
||||||
|
ops::evaluate(streams, std::slice::from_ref(&mismatch), stream, false)?;
|
||||||
|
let mut failed = [0u8];
|
||||||
|
mismatch.buffer().read(mismatch.offset(), &mut failed)?;
|
||||||
|
if failed[0] != 0 {
|
||||||
|
return Err(
|
||||||
|
"first whole-call output differs bit for bit from the stock sorted gather".into(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
Ok(ours)
|
||||||
|
})();
|
||||||
|
match probe {
|
||||||
|
Ok(ours) => {
|
||||||
|
state.0.insert(key.clone(), true);
|
||||||
|
state.1.calls += 1;
|
||||||
|
eprintln!(
|
||||||
|
"[moe-sorted-gather] tensor-unit kernel on for {key}: first call equals stock bit for bit"
|
||||||
|
);
|
||||||
|
Ok(Some(ours))
|
||||||
|
}
|
||||||
|
Err(reason) => {
|
||||||
|
state.0.insert(key.clone(), false);
|
||||||
|
state.1.failures += 1;
|
||||||
|
state.1.fallbacks += 1;
|
||||||
|
eprintln!(
|
||||||
|
"[moe-sorted-gather] tensor-unit kernel off for {key}: {reason}; stock sorted gather runs instead (canary_failures={})",
|
||||||
|
state.1.failures
|
||||||
|
);
|
||||||
|
Ok(None)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
pub(super) fn swiglu_rows(
|
||||||
|
tokens: &Array,
|
||||||
|
map: &Array,
|
||||||
|
gate: &QuantizedLinear,
|
||||||
|
up: Option<&QuantizedLinear>,
|
||||||
|
ids: &Array,
|
||||||
|
enabled: bool,
|
||||||
|
streams: &Streams,
|
||||||
|
stream: Stream,
|
||||||
|
) -> Result<Array, String> {
|
||||||
|
let eligible = applies(
|
||||||
|
tokens,
|
||||||
|
map,
|
||||||
|
gate,
|
||||||
|
up,
|
||||||
|
ids,
|
||||||
|
enabled,
|
||||||
|
super::sparse_attention::nax_available(),
|
||||||
|
);
|
||||||
|
if eligible && let Some(y) = checked(tokens, map, gate, up, ids, true, streams, stream)? {
|
||||||
|
return Ok(y);
|
||||||
|
}
|
||||||
|
// The fused-weight wrapper retries its plain row-map gather if the SwiGLU
|
||||||
|
// canary failed. Separate SwitchGLU instead returns to its original chain.
|
||||||
|
if eligible
|
||||||
|
&& up.is_none()
|
||||||
|
&& let Some(gu) = checked(tokens, map, gate, None, ids, false, streams, stream)?
|
||||||
|
{
|
||||||
|
let pair = ops::split(&gu, &[gu.layout().dim(-1)? / 2], -1, stream)?;
|
||||||
|
return activation(&pair[0], &pair[1], false, stream);
|
||||||
|
}
|
||||||
|
if let Some(up) = up {
|
||||||
|
let x = indexing::take(tokens, map, 0, stream)?;
|
||||||
|
let up = gather(&x, up, ids, streams, stream)?;
|
||||||
|
let gate = gather(&x, gate, ids, streams, stream)?;
|
||||||
|
return activation(&gate, &up, true, stream);
|
||||||
|
}
|
||||||
|
stock(tokens, map, gate, None, ids, true, streams, stream)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
#[ignore = "Metal synthetic all supported row-map variants; external supervisor"]
|
||||||
|
fn issue112_all_kernel_variants() {
|
||||||
|
use super::super::{
|
||||||
|
gpu::Context,
|
||||||
|
qwen_mtplx_tests::{hex_digest, pattern},
|
||||||
|
};
|
||||||
|
super::configure_sources().unwrap();
|
||||||
|
let _context = Context::open_qwen(0).unwrap();
|
||||||
|
let streams = Streams::new(Some(super::allocator::Allocator::new().unwrap()));
|
||||||
|
let gpu = streams.default_stream(super::stream::Device::Gpu).unwrap();
|
||||||
|
let e = 128;
|
||||||
|
let n = 256;
|
||||||
|
let k = 256;
|
||||||
|
let tokens = 512;
|
||||||
|
let read = |a: &Array| {
|
||||||
|
ops::evaluate(&streams, std::slice::from_ref(a), gpu, false).unwrap();
|
||||||
|
let mut v = vec![0; a.layout().nbytes()];
|
||||||
|
a.buffer().read(a.offset(), &mut v).unwrap();
|
||||||
|
v
|
||||||
|
};
|
||||||
|
let u32s = |v: Vec<u32>| {
|
||||||
|
Array::new(
|
||||||
|
&[v.len() as i32],
|
||||||
|
Dtype::U32,
|
||||||
|
super::scalar_buffer(&v.into_iter().flat_map(u32::to_le_bytes).collect::<Vec<_>>())
|
||||||
|
.unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap()
|
||||||
|
};
|
||||||
|
for dtype in [Dtype::BF16, Dtype::F16] {
|
||||||
|
for bits in [4, 8] {
|
||||||
|
for group in [32, 64, 128] {
|
||||||
|
let pack = |salt| {
|
||||||
|
let mut q = mlp::fixture_linear(e * n, k, bits, group, salt);
|
||||||
|
for a in [&mut q.weight, &mut q.scales, &mut q.biases] {
|
||||||
|
let width = a.layout().dim(-1).unwrap();
|
||||||
|
*a = ops::reshape(a, &[e as i32, n as i32, width], gpu).unwrap();
|
||||||
|
if a.layout().dtype() == Dtype::BF16 {
|
||||||
|
*a = ops::astype(a, dtype, false, gpu).unwrap();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
q
|
||||||
|
};
|
||||||
|
let gate = pack(7);
|
||||||
|
let up = pack(9);
|
||||||
|
let x = ops::astype(
|
||||||
|
&Array::new(
|
||||||
|
&[tokens, 1, k as i32],
|
||||||
|
Dtype::BF16,
|
||||||
|
pattern(tokens as u32 * k, 8),
|
||||||
|
)
|
||||||
|
.unwrap(),
|
||||||
|
dtype,
|
||||||
|
false,
|
||||||
|
gpu,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
for separate in [false, true] {
|
||||||
|
for rows in [4096, 8193, 32769] {
|
||||||
|
eprintln!(
|
||||||
|
"row-map {dtype:?} b{bits}/g{group} separate={separate} rows={rows}"
|
||||||
|
);
|
||||||
|
let ids = u32s(
|
||||||
|
(0..rows)
|
||||||
|
.map(|r| (r * (e - 3) / rows) / 2 * 2 + 1)
|
||||||
|
.collect(),
|
||||||
|
);
|
||||||
|
let map = u32s((0..rows).map(|r| (r * 13) % tokens as u32).collect());
|
||||||
|
let ids = if rows == 32769 {
|
||||||
|
ops::astype(&ids, Dtype::I64, false, gpu).unwrap()
|
||||||
|
} else {
|
||||||
|
ids
|
||||||
|
};
|
||||||
|
let map = if rows == 8193 {
|
||||||
|
let repeated = ops::repeat(&map, 2, 0, gpu).unwrap();
|
||||||
|
ops::slice(&repeated, &[0], &[(rows * 2) as i32], &[2], gpu).unwrap()
|
||||||
|
} else {
|
||||||
|
map
|
||||||
|
};
|
||||||
|
let up = separate.then_some(&up);
|
||||||
|
assert!(applies(&x, &map, &gate, up, &ids, true, true));
|
||||||
|
assert!(!applies(&x, &map, &gate, up, &ids, false, true));
|
||||||
|
assert!(!applies(&x, &map, &gate, up, &ids, true, false));
|
||||||
|
let before = stats();
|
||||||
|
let y =
|
||||||
|
swiglu_rows(&x, &map, &gate, up, &ids, true, &streams, gpu).unwrap();
|
||||||
|
let got = read(&y);
|
||||||
|
let reference =
|
||||||
|
read(&stock(&x, &map, &gate, up, &ids, true, &streams, gpu).unwrap());
|
||||||
|
assert_eq!(got, reference);
|
||||||
|
let after = stats();
|
||||||
|
assert_eq!(after.failures, before.failures, "{after:?}");
|
||||||
|
assert_eq!(after.calls, before.calls + 1);
|
||||||
|
println!(
|
||||||
|
"{}",
|
||||||
|
serde_json::json!({"case":"synthetic","dtype":format!("{dtype:?}"),"bits":bits,"group":group,"separate":separate,"rows":rows,"sha256":hex_digest(&got),"canaries":after.canaries,"calls":after.calls})
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
#[ignore = "Metal canary rejection and sticky fallback; external supervisor"]
|
||||||
|
fn issue112_canary_failures() {
|
||||||
|
use super::super::{gpu::Context, qwen_mtplx_tests::pattern};
|
||||||
|
super::configure_sources().unwrap();
|
||||||
|
let _context = Context::open_qwen(0).unwrap();
|
||||||
|
let streams = Streams::new(Some(super::allocator::Allocator::new().unwrap()));
|
||||||
|
let gpu = streams.default_stream(super::stream::Device::Gpu).unwrap();
|
||||||
|
let rows = 4096;
|
||||||
|
let e = 16;
|
||||||
|
let n = 128;
|
||||||
|
let k = 256;
|
||||||
|
let u32s = |v: Vec<u32>| {
|
||||||
|
Array::new(
|
||||||
|
&[v.len() as i32],
|
||||||
|
Dtype::U32,
|
||||||
|
super::scalar_buffer(&v.into_iter().flat_map(u32::to_le_bytes).collect::<Vec<_>>())
|
||||||
|
.unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap()
|
||||||
|
};
|
||||||
|
let x = Array::new(&[1, 1, k], Dtype::BF16, pattern(k as u32, 8)).unwrap();
|
||||||
|
let ids = u32s((0..rows).map(|r| r * e / rows).collect());
|
||||||
|
let map = u32s(vec![0; rows as usize]);
|
||||||
|
let read = |a: &Array| {
|
||||||
|
ops::evaluate(&streams, std::slice::from_ref(a), gpu, false).unwrap();
|
||||||
|
let mut v = vec![0; a.layout().nbytes()];
|
||||||
|
a.buffer().read(a.offset(), &mut v).unwrap();
|
||||||
|
v
|
||||||
|
};
|
||||||
|
for (failure, group) in [(1, 32), (2, 64), (3, 128)] {
|
||||||
|
for separate in [false, true] {
|
||||||
|
let mut q = mlp::fixture_linear(e * n, k as u32, 4, group, 7);
|
||||||
|
for a in [&mut q.weight, &mut q.scales, &mut q.biases] {
|
||||||
|
let width = a.layout().dim(-1).unwrap();
|
||||||
|
*a = ops::reshape(a, &[e as i32, n as i32, width], gpu).unwrap();
|
||||||
|
}
|
||||||
|
let up = separate.then_some(&q);
|
||||||
|
let reference = read(&stock(&x, &map, &q, up, &ids, true, &streams, gpu).unwrap());
|
||||||
|
let before = stats();
|
||||||
|
FAILURE.store(failure, std::sync::atomic::Ordering::Relaxed);
|
||||||
|
let actual = read(&swiglu_rows(&x, &map, &q, up, &ids, true, &streams, gpu).unwrap());
|
||||||
|
FAILURE.store(0, std::sync::atomic::Ordering::Relaxed);
|
||||||
|
assert_eq!(actual, reference);
|
||||||
|
let rejected = stats();
|
||||||
|
assert_eq!(
|
||||||
|
rejected.failures - before.failures,
|
||||||
|
if separate { 1 } else { 2 }
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
read(&swiglu_rows(&x, &map, &q, up, &ids, true, &streams, gpu).unwrap()),
|
||||||
|
reference
|
||||||
|
);
|
||||||
|
let after = stats();
|
||||||
|
assert_eq!(after.canaries, rejected.canaries);
|
||||||
|
assert_eq!(after.calls, before.calls);
|
||||||
|
assert_eq!(after.failures, rejected.failures);
|
||||||
|
println!(
|
||||||
|
"{}",
|
||||||
|
serde_json::json!({"case":"sticky_fallback","failure":failure,"separate":separate,"canaries":after.canaries,"failures":after.failures,"fallbacks":after.fallbacks,"exact":true})
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn row_map_applicability() {
|
||||||
|
let a = |shape: &[i32], dtype| Array::unallocated(shape, dtype).unwrap();
|
||||||
|
let q = QuantizedLinear {
|
||||||
|
weight: a(&[128, 256, 32], Dtype::U32),
|
||||||
|
scales: a(&[128, 256, 4], Dtype::BF16),
|
||||||
|
biases: a(&[128, 256, 4], Dtype::BF16),
|
||||||
|
bits: 4,
|
||||||
|
group: 64,
|
||||||
|
};
|
||||||
|
let x = a(&[512, 1, 256], Dtype::BF16);
|
||||||
|
for rows in [1, 40, 63, 64, 4095, 4096, 32769] {
|
||||||
|
let map = a(&[rows], Dtype::U32);
|
||||||
|
let ids = a(&[rows], Dtype::U32);
|
||||||
|
assert_eq!(applies(&x, &map, &q, None, &ids, true, true), rows >= 4096);
|
||||||
|
assert!(!applies(&x, &map, &q, None, &ids, false, true));
|
||||||
|
assert!(!applies(&x, &map, &q, None, &ids, true, false));
|
||||||
|
}
|
||||||
|
let ids = a(&[4096], Dtype::U32);
|
||||||
|
for map in [
|
||||||
|
a(&[4096], Dtype::I32),
|
||||||
|
a(&[1, 4096], Dtype::U32),
|
||||||
|
a(&[4095], Dtype::U32),
|
||||||
|
] {
|
||||||
|
assert!(!applies(&x, &map, &q, None, &ids, true, true));
|
||||||
|
}
|
||||||
|
for x in [
|
||||||
|
a(&[512, 1, 256], Dtype::F32),
|
||||||
|
a(&[512, 256], Dtype::BF16),
|
||||||
|
a(&[512, 2, 256], Dtype::BF16),
|
||||||
|
a(&[512, 1, 128], Dtype::BF16),
|
||||||
|
] {
|
||||||
|
assert!(!applies(&x, &ids, &q, None, &ids, true, true));
|
||||||
|
}
|
||||||
|
let mut bad = q.clone();
|
||||||
|
bad.group = 16;
|
||||||
|
assert!(!applies(&x, &ids, &bad, None, &ids, true, true));
|
||||||
|
bad = q.clone();
|
||||||
|
bad.bits = 2;
|
||||||
|
assert!(!applies(&x, &ids, &bad, None, &ids, true, true));
|
||||||
|
bad = q.clone();
|
||||||
|
bad.scales = a(&[128, 256, 4], Dtype::F16);
|
||||||
|
assert!(!applies(&x, &ids, &bad, None, &ids, true, true));
|
||||||
|
bad = q.clone();
|
||||||
|
bad.weight = a(&[128, 255, 32], Dtype::U32);
|
||||||
|
assert!(!applies(&x, &ids, &bad, None, &ids, true, true));
|
||||||
|
assert!(!applies(&x, &ids, &q, Some(&bad), &ids, true, true));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
#[ignore = "installed projection weights; external supervisor required"]
|
||||||
|
fn issue112_installed_gate_up() {
|
||||||
|
use super::super::{
|
||||||
|
gpu::Context,
|
||||||
|
qwen_mtplx_tests::{hex_digest, pattern},
|
||||||
|
};
|
||||||
|
super::configure_sources().unwrap();
|
||||||
|
let _context = Context::open_qwen(0).unwrap();
|
||||||
|
let streams = Streams::new(Some(super::allocator::Allocator::new().unwrap()));
|
||||||
|
let gpu = streams.default_stream(super::stream::Device::Gpu).unwrap();
|
||||||
|
let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
|
||||||
|
let weights = super::weights::load_shards(&root, &streams).unwrap();
|
||||||
|
let parts = |name: &str| {
|
||||||
|
["weight", "scales", "biases"].map(|p| {
|
||||||
|
weights[&format!("language_model.model.layers.0.mlp.switch_mlp.{name}.{p}")].clone()
|
||||||
|
})
|
||||||
|
};
|
||||||
|
let gate = parts("gate_proj");
|
||||||
|
let up = parts("up_proj");
|
||||||
|
let gu = std::array::from_fn(|i| {
|
||||||
|
ops::concatenate(&[gate[i].clone(), up[i].clone()], 1, gpu).unwrap()
|
||||||
|
});
|
||||||
|
let pack = |[weight, scales, biases]: [Array; 3]| QuantizedLinear {
|
||||||
|
weight,
|
||||||
|
scales,
|
||||||
|
biases,
|
||||||
|
bits: 4,
|
||||||
|
group: 64,
|
||||||
|
};
|
||||||
|
let fused = pack(gu);
|
||||||
|
let gate = pack(gate);
|
||||||
|
let up = pack(up);
|
||||||
|
let u32s = |v: Vec<u32>| {
|
||||||
|
Array::new(
|
||||||
|
&[v.len() as i32],
|
||||||
|
Dtype::U32,
|
||||||
|
super::scalar_buffer(&v.into_iter().flat_map(u32::to_le_bytes).collect::<Vec<_>>())
|
||||||
|
.unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap()
|
||||||
|
};
|
||||||
|
let read = |a: &Array| {
|
||||||
|
ops::evaluate(&streams, std::slice::from_ref(a), gpu, false).unwrap();
|
||||||
|
let mut v = vec![0; a.layout().nbytes()];
|
||||||
|
a.buffer().read(a.offset(), &mut v).unwrap();
|
||||||
|
v
|
||||||
|
};
|
||||||
|
for tokens in [410, 2048, 3277, 4095, 4096, 8192] {
|
||||||
|
let rows = tokens * 10;
|
||||||
|
let x = Array::new(
|
||||||
|
&[tokens as i32, 1, 2560],
|
||||||
|
Dtype::BF16,
|
||||||
|
pattern(tokens * 2560, 7),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let ids = u32s((0..rows).map(|r| r * 512 / rows).collect());
|
||||||
|
let map = u32s((0..rows).map(|r| r % tokens).collect());
|
||||||
|
let mut packed = None;
|
||||||
|
for separate in [false, true] {
|
||||||
|
let (g, u) = if separate {
|
||||||
|
(&gate, Some(&up))
|
||||||
|
} else {
|
||||||
|
(&fused, None)
|
||||||
|
};
|
||||||
|
let before = stats();
|
||||||
|
let y = swiglu_rows(&x, &map, g, u, &ids, true, &streams, gpu).unwrap();
|
||||||
|
let got = read(&y);
|
||||||
|
assert_eq!(
|
||||||
|
got,
|
||||||
|
read(&stock(&x, &map, g, u, &ids, true, &streams, gpu).unwrap())
|
||||||
|
);
|
||||||
|
let hash = hex_digest(&got);
|
||||||
|
if let Some(packed) = &packed {
|
||||||
|
assert_eq!(&hash, packed);
|
||||||
|
} else {
|
||||||
|
packed = Some(hash.clone());
|
||||||
|
}
|
||||||
|
let after = stats();
|
||||||
|
assert_eq!(after.failures, before.failures);
|
||||||
|
assert_eq!(after.calls, before.calls + 1);
|
||||||
|
println!(
|
||||||
|
"{}",
|
||||||
|
serde_json::json!({"case":"installed_gate_up","tokens":tokens,"rows":rows,"separate":separate,"sha256":hash,"canaries":after.canaries,"calls":after.calls,"exact":true})
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
#[ignore = "full installed model; validates all cold/warm state receipts against the pre-port oracle"]
|
||||||
|
fn issue112_full_model_state() {
|
||||||
|
let before = stats();
|
||||||
|
super::issue110::issue110_large_cold_and_warm_prefill();
|
||||||
|
let after = stats();
|
||||||
|
assert!(after.calls > before.calls);
|
||||||
|
assert_eq!(after.failures, before.failures);
|
||||||
|
let directory = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE110_DIR").unwrap());
|
||||||
|
for chunk in [2048, 4096] {
|
||||||
|
for phase in ["cold", "warm"] {
|
||||||
|
let actual: serde_json::Value = serde_json::from_slice(
|
||||||
|
&std::fs::read(directory.join(format!("{chunk}-{phase}.json"))).unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let expected: serde_json::Value = serde_json::from_slice(
|
||||||
|
&std::fs::read(
|
||||||
|
std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(format!(
|
||||||
|
"docs/issue110-sorted-gather/{chunk}-{phase}-state.json"
|
||||||
|
)),
|
||||||
|
)
|
||||||
|
.unwrap(),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(actual["state"], expected["state"]);
|
||||||
|
assert_eq!(actual["cached"], expected["cached"]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
println!(
|
||||||
|
"{}",
|
||||||
|
serde_json::json!({"case":"model_state","calls":after.calls,"canaries":after.canaries,"failures":after.failures,"cold_warm_exact":true})
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -6,6 +6,8 @@ use super::stream::{Device, Stream, Streams};
|
|||||||
use super::{eval, views};
|
use super::{eval, views};
|
||||||
|
|
||||||
pub(super) enum Operation {
|
pub(super) enum Operation {
|
||||||
|
MoeSortedGather(super::moe_sorted::Kernel),
|
||||||
|
MoeSortedCounts,
|
||||||
Vision(super::vision_kernels::Kernel),
|
Vision(super::vision_kernels::Kernel),
|
||||||
Compiled(super::fused::Expression),
|
Compiled(super::fused::Expression),
|
||||||
Load(super::load::Load),
|
Load(super::load::Load),
|
||||||
@@ -60,7 +62,7 @@ pub(super) enum Operation {
|
|||||||
bits: u32,
|
bits: u32,
|
||||||
group: u32,
|
group: u32,
|
||||||
},
|
},
|
||||||
GatherQmmBf16 {
|
GatherQmm {
|
||||||
bits: u32,
|
bits: u32,
|
||||||
group: u32,
|
group: u32,
|
||||||
right_sorted: bool,
|
right_sorted: bool,
|
||||||
@@ -134,6 +136,9 @@ pub(super) enum Operation {
|
|||||||
impl Operation {
|
impl Operation {
|
||||||
fn evaluate(&self, input: &Array, output: &Array, stream: Stream) -> Result<(), String> {
|
fn evaluate(&self, input: &Array, output: &Array, stream: Stream) -> Result<(), String> {
|
||||||
match self {
|
match self {
|
||||||
|
Self::MoeSortedGather(_) | Self::MoeSortedCounts => {
|
||||||
|
Err("sorted MoE requires captured inputs".into())
|
||||||
|
}
|
||||||
Self::Compiled(_) => Err("compiled expression requires all inputs/outputs".into()),
|
Self::Compiled(_) => Err("compiled expression requires all inputs/outputs".into()),
|
||||||
Self::QsaCompiledScalar(_) => {
|
Self::QsaCompiledScalar(_) => {
|
||||||
Err("compiled scalar requires its captured inputs".into())
|
Err("compiled scalar requires its captured inputs".into())
|
||||||
@@ -215,7 +220,7 @@ impl Operation {
|
|||||||
| Self::ArangeI64 => Err("invalid indexing arity".into()),
|
| Self::ArangeI64 => Err("invalid indexing arity".into()),
|
||||||
Self::Vision(_) | Self::RmsNorm(_) => Err("RMSNorm requires two inputs".into()),
|
Self::Vision(_) | Self::RmsNorm(_) => Err("RMSNorm requires two inputs".into()),
|
||||||
Self::QuantizedLinearBf16 { .. } => Err("quantized linear requires four inputs".into()),
|
Self::QuantizedLinearBf16 { .. } => Err("quantized linear requires four inputs".into()),
|
||||||
Self::GatherQmmBf16 { .. } => Err("GatherQMM requires six inputs".into()),
|
Self::GatherQmm { .. } => Err("GatherQMM requires six inputs".into()),
|
||||||
Self::AffineQuantize { .. } => Err("quantize requires three sibling outputs".into()),
|
Self::AffineQuantize { .. } => Err("quantize requires three sibling outputs".into()),
|
||||||
Self::AffineDequantize { .. } => Err("dequantize requires three inputs".into()),
|
Self::AffineDequantize { .. } => Err("dequantize requires three inputs".into()),
|
||||||
Self::HyperRead(_) | Self::HyperV3R1 | Self::HyperV3R2 => {
|
Self::HyperRead(_) | Self::HyperV3R1 | Self::HyperV3R2 => {
|
||||||
@@ -262,6 +267,12 @@ pub(super) fn evaluate(
|
|||||||
.operation()
|
.operation()
|
||||||
.ok_or("array has no executable primitive")?;
|
.ok_or("array has no executable primitive")?;
|
||||||
match &*operation {
|
match &*operation {
|
||||||
|
Operation::MoeSortedGather(kernel) => {
|
||||||
|
return kernel.evaluate(inputs, outputs);
|
||||||
|
}
|
||||||
|
Operation::MoeSortedCounts => {
|
||||||
|
return super::moe_sorted::counts_evaluate(inputs, &outputs[0]);
|
||||||
|
}
|
||||||
Operation::Vision(k) => {
|
Operation::Vision(k) => {
|
||||||
return super::vision_kernels::evaluate(*k, inputs, &outputs[0]);
|
return super::vision_kernels::evaluate(*k, inputs, &outputs[0]);
|
||||||
}
|
}
|
||||||
@@ -526,7 +537,7 @@ pub(super) fn evaluate(
|
|||||||
}
|
}
|
||||||
return super::linear::evaluate(inputs, &outputs[0], *bits, *group);
|
return super::linear::evaluate(inputs, &outputs[0], *bits, *group);
|
||||||
}
|
}
|
||||||
if let Operation::GatherQmmBf16 {
|
if let Operation::GatherQmm {
|
||||||
bits,
|
bits,
|
||||||
group,
|
group,
|
||||||
right_sorted,
|
right_sorted,
|
||||||
|
|||||||
@@ -25,6 +25,11 @@ fn falsey(value: Option<&str>) -> bool {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub(super) fn sorted_gather_enabled(get: impl Fn(&'static str) -> Option<String>) -> bool {
|
||||||
|
!falsey(get("MTPLX_MOE_SORTED_GATHER_KERNEL").as_deref())
|
||||||
|
&& !truthy(get("MTPLX_FORCE_GPU_FAMILY_FALLBACK").as_deref())
|
||||||
|
}
|
||||||
|
|
||||||
fn strict_bool(name: &str, raw: Option<String>) -> Result<bool, String> {
|
fn strict_bool(name: &str, raw: Option<String>) -> Result<bool, String> {
|
||||||
match raw
|
match raw
|
||||||
.as_deref()
|
.as_deref()
|
||||||
@@ -323,3 +328,19 @@ fn qwen_server_policy_matches_original_functions() {
|
|||||||
}
|
}
|
||||||
assert_eq!(count, 268);
|
assert_eq!(count, 268);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn sorted_moe_switch_and_family_rehearsal() {
|
||||||
|
assert!(sorted_gather_enabled(|_| None));
|
||||||
|
for off in ["0", " false ", "NO", "off"] {
|
||||||
|
assert!(!sorted_gather_enabled(|key| (key
|
||||||
|
== "MTPLX_MOE_SORTED_GATHER_KERNEL")
|
||||||
|
.then(|| off.to_owned())));
|
||||||
|
}
|
||||||
|
for force in ["1", " true ", "YES", "on"] {
|
||||||
|
assert!(!sorted_gather_enabled(|key| (key
|
||||||
|
== "MTPLX_FORCE_GPU_FAMILY_FALLBACK")
|
||||||
|
.then(|| force.to_owned())));
|
||||||
|
}
|
||||||
|
assert!(sorted_gather_enabled(|_| Some("unexpected".into())));
|
||||||
|
}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
"""Regenerate only the vendored GPU shader closure from the pinned oracle."""
|
||||||
|
import subprocess,sys
|
||||||
|
from pathlib import Path
|
||||||
|
checkout=Path(sys.argv[1])
|
||||||
|
assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
|
||||||
|
sys.path.insert(0,str(checkout))
|
||||||
|
import mlx.core as mx
|
||||||
|
from mtplx.kernels import moe_sorted_gather_nax as k
|
||||||
|
assert mx.__version__=='0.32.2'
|
||||||
|
root=k._include_root();seen=set();out=[]
|
||||||
|
for rel in (k._PREAMBLE_HEADER,*k._KERNEL_HEADERS):k._inline(root,rel,seen,out)
|
||||||
|
for rel,name in [('mlx/backend/metal/kernels/unary_ops.h','Sigmoid'),('mlx/backend/metal/kernels/binary_ops.h','Multiply')]:
|
||||||
|
functor=k._functor(root,rel,name)
|
||||||
|
assert functor is not None
|
||||||
|
out.append(functor)
|
||||||
|
metal=Path(__file__).resolve().parents[1]/'metal'
|
||||||
|
preamble='// MLX 0.32.2 GPU support, Copyright Apple Inc., MIT; see MLX-LICENSE.txt.\n// Flattened only the GPU header closure used by MTPLX 9882703 row-map gather.\n'
|
||||||
|
(metal/'mtplx-moe-sorted-header.metal').write_text(preamble+'\n'.join(out)+'\nusing namespace mlx::steel;\n')
|
||||||
|
(metal/'mtplx-moe-sorted-body.metal').write_text('// MTPLX 9882703, MIT; see MTPLX-LICENSE.txt.\n'+k._SOURCE)
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
"""Installed gate/up row-map oracle, using the identical native fixture inputs."""
|
||||||
|
import hashlib,json,subprocess,sys
|
||||||
|
from pathlib import Path
|
||||||
|
checkout,root,native=map(Path,sys.argv[1:4])
|
||||||
|
assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
|
||||||
|
sys.path.insert(0,str(checkout))
|
||||||
|
import mlx.core as mx
|
||||||
|
import numpy as np
|
||||||
|
from mtplx.kernels import moe_sorted_gather_nax as kernel
|
||||||
|
assert mx.__version__=='0.32.2'
|
||||||
|
index=json.loads((root/'model.safetensors.index.json').read_text())['weight_map']
|
||||||
|
base='language_model.model.layers.0.mlp.switch_mlp.'
|
||||||
|
weights=mx.load(str(root/index[base+'gate_proj.weight']))
|
||||||
|
def parts(proj):return [weights[base+proj+'.'+part] for part in ['weight','scales','biases']]
|
||||||
|
gate=parts('gate_proj');up=parts('up_proj')
|
||||||
|
gu=[mx.concatenate(a,axis=1) for a in zip(gate,up)]
|
||||||
|
mx.eval(*gu,*gate,*up)
|
||||||
|
def digest(a):
|
||||||
|
mx.eval(a)
|
||||||
|
return hashlib.sha256(np.asarray(a.view(mx.uint16)).tobytes()).hexdigest()
|
||||||
|
count=0
|
||||||
|
for line in native.read_text().splitlines():
|
||||||
|
if '"case":"installed_gate_up"' not in line:continue
|
||||||
|
case=json.loads(line[line.index('{'):]);count+=1
|
||||||
|
tokens,rows=case['tokens'],case['rows']
|
||||||
|
v=((np.arange(tokens*2560,dtype=np.uint32)*17+7*13)%257).astype(np.int32)-128
|
||||||
|
x=mx.array((v.astype(np.float32)/128).reshape(tokens,1,2560)).astype(mx.bfloat16)
|
||||||
|
row_map=mx.arange(rows,dtype=mx.uint32)%tokens
|
||||||
|
ids=mx.array(np.arange(rows,dtype=np.uint32)*512//rows)
|
||||||
|
g,u=(gate,up) if case['separate'] else (gu,None)
|
||||||
|
kw=dict(group_size=64,bits=4,swiglu=True,up=u)
|
||||||
|
y=kernel.gather_rows_qmm(x,row_map,*g,ids,**kw)
|
||||||
|
stock=kernel._stock(x,row_map,*g,ids,**kw)
|
||||||
|
assert y is not None and digest(y)==digest(stock)==case['sha256'],case
|
||||||
|
print(json.dumps(dict(case='installed_oracle',native=case,exact=True,kernel=kernel.stats())),flush=True)
|
||||||
|
del x,row_map,ids,y,stock
|
||||||
|
mx.clear_cache()
|
||||||
|
assert count==12,count
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
"""Pinned full-model cold/warm prefill, including explicit hidden-state capture."""
|
||||||
|
import argparse,hashlib,json,os,subprocess,sys
|
||||||
|
from pathlib import Path
|
||||||
|
parser=argparse.ArgumentParser(description=__doc__)
|
||||||
|
for name in ['checkout','root','native','out']:parser.add_argument(name,type=Path)
|
||||||
|
parser.add_argument('--hidden',action='store_true')
|
||||||
|
a=parser.parse_args()
|
||||||
|
assert subprocess.check_output(['git','-C',str(a.checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
|
||||||
|
sys.path.insert(0,str(a.checkout))
|
||||||
|
from mtplx.server.openai import _server_runtime_env_overrides
|
||||||
|
os.environ.update(_server_runtime_env_overrides(argparse.Namespace(model=str(a.root),generation_mode='mtp' if a.hidden else 'ar'),None))
|
||||||
|
os.environ['MTPLX_SUSTAINED_PREFILL']='1'
|
||||||
|
import mlx.core as mx
|
||||||
|
import numpy as np
|
||||||
|
from mlx_lm.utils import load_model
|
||||||
|
from mtplx.models import qwen4_exp
|
||||||
|
from mtplx.runtime import MTPLXRuntime,_load_tokenizer_resilient
|
||||||
|
from mtplx.sampling import SamplerConfig
|
||||||
|
from mtplx.generation import generate_ar,restore_or_prefill_prompt_state
|
||||||
|
from mtplx.cache_state import snapshot_cache,snapshot_cache_lazy_hybrid
|
||||||
|
from mtplx.session_bank import SessionBank,_lazy_snapshot_enabled
|
||||||
|
from mtplx.kernels import moe_sorted_gather_nax as kernel
|
||||||
|
assert mx.__version__=='0.32.2'
|
||||||
|
a.out.mkdir(parents=True,exist_ok=True)
|
||||||
|
model,config=load_model(a.root,lazy=False,strict=True,get_model_classes=lambda **_:(qwen4_exp.Model,qwen4_exp.ModelArgs))
|
||||||
|
model.post_weight_load(a.root)
|
||||||
|
tokenizer=_load_tokenizer_resilient(a.root,config)
|
||||||
|
from mtplx.mtp_patch import MTPContract
|
||||||
|
if a.hidden: assert model.attach_mtp(a.root)
|
||||||
|
rt=MTPLXRuntime(model,tokenizer,a.root,a.hidden,MTPContract() if a.hidden else None)
|
||||||
|
def fingerprint(v):
|
||||||
|
values=np.asarray(v.astype(mx.float32)).astype('<f4')
|
||||||
|
assert np.isfinite(values).all()
|
||||||
|
return dict(shape=list(v.shape),sha256=hashlib.sha256(values.tobytes()).hexdigest())
|
||||||
|
for chunk in [2048,4096]:
|
||||||
|
os.environ['MTPLX_PREFILL_CHUNK_SIZE']=str(chunk)
|
||||||
|
bank=SessionBank()
|
||||||
|
for phase in ['cold','warm']:
|
||||||
|
name=f'{chunk}-{phase}' if a.hidden else f'{chunk}-0-{phase}'
|
||||||
|
expected=json.loads((a.native/(name+'.json')).read_text())
|
||||||
|
prompt=expected['state']['ids'] if a.hidden else expected['state']['ids'][:expected['metrics']['prompt']]
|
||||||
|
if a.hidden:
|
||||||
|
result=restore_or_prefill_prompt_state(rt,prompt,mtp_history_policy='cycle',session_bank=bank,session_id='issue112',capture_hidden=True,store_prefix_snapshot=False,prefill_callback=lambda event:print(json.dumps(dict(case='progress',chunk=chunk,phase=phase,event=event),default=str),flush=True))
|
||||||
|
cache,logits,hidden,ids,cached=result.trunk_cache,result.logits,result.hidden,prompt,result.cached_tokens
|
||||||
|
else:
|
||||||
|
result=generate_ar(rt,prompt,max_tokens=1,sampler=SamplerConfig(temperature=0.,top_p=.95,top_k=20),seed=1,stop_token_ids=set(),capture_final_state=True,session_bank=bank,session_id='issue112',store_prefix_snapshot=False,token_callback=lambda ids:print(json.dumps(dict(case='tokens',chunk=chunk,phase=phase,ids=ids)),flush=True))
|
||||||
|
final=result.final_state
|
||||||
|
assert final.safe_to_commit
|
||||||
|
cache,logits,hidden,ids,cached=final.final_trunk_cache,final.final_logits,final.final_hidden,prompt+result.tokens,result.stats.cached_tokens
|
||||||
|
states=[[None if v is None else fingerprint(v) for v in state] if state is not None else None for state in (snapshot_cache_lazy_hybrid(cache) if _lazy_snapshot_enabled() else snapshot_cache(cache)).states]
|
||||||
|
record=dict(state=dict(ids=ids,logits=fingerprint(logits),states=states),hidden=None if hidden is None else fingerprint(hidden))
|
||||||
|
(a.out/(name+'.json')).write_text(json.dumps(record))
|
||||||
|
assert record['state']==expected['state'],(chunk,phase,'state differs')
|
||||||
|
assert record['hidden']==expected['hidden'],(chunk,phase,'hidden differs')
|
||||||
|
assert cached==(expected['cached'] if a.hidden else expected['metrics']['cached'])
|
||||||
|
print(json.dumps(dict(case='model_oracle',chunk=chunk,phase=phase,hidden=a.hidden,exact=True,cached=cached,kernel=kernel.stats())),flush=True)
|
||||||
|
if phase=='cold':
|
||||||
|
assert bank.put(runtime=rt,token_ids=ids,cache=cache,logits=logits,hidden=hidden,hidden_variant='post_norm',session_id='issue112',mtp_history_policy='cycle',snapshot_epoch=len(ids)) is not None
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
"""Pinned row-map kernel, guarded stock chain and native synthetic bit receipts."""
|
||||||
|
import hashlib,json,subprocess,sys
|
||||||
|
from pathlib import Path
|
||||||
|
checkout,native=map(Path,sys.argv[1:3])
|
||||||
|
assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
|
||||||
|
sys.path.insert(0,str(checkout))
|
||||||
|
import mlx.core as mx
|
||||||
|
import numpy as np
|
||||||
|
from mtplx.kernels import moe_sorted_gather_nax as kernel
|
||||||
|
assert mx.__version__=='0.32.2'
|
||||||
|
def pattern(count,salt,scale=1):
|
||||||
|
a=((np.arange(count,dtype=np.uint32)*17+salt*13)%257).astype(np.int32)-128
|
||||||
|
return mx.array(a.astype(np.float32)/128*scale).astype(mx.bfloat16)
|
||||||
|
def pack(e,n,k,bits,group,salt,dtype):
|
||||||
|
words=np.arange(e*n*k//32*bits,dtype=np.uint32)*np.uint32(2654435761)+np.uint32(12345)
|
||||||
|
return (mx.array(words.reshape(e,n,k*bits//32)),pattern(e*n*k//group,salt,1/64).reshape(e,n,k//group).astype(dtype),pattern(e*n*k//group,salt+1,1/64).reshape(e,n,k//group).astype(dtype))
|
||||||
|
def digest(a):
|
||||||
|
mx.eval(a)
|
||||||
|
return hashlib.sha256(np.asarray(a.view(mx.uint16)).tobytes()).hexdigest()
|
||||||
|
count=0
|
||||||
|
for line in native.read_text().splitlines():
|
||||||
|
if '"case":"synthetic"' not in line:continue
|
||||||
|
case=json.loads(line[line.index('{'):])
|
||||||
|
count+=1
|
||||||
|
dtype={'BF16':mx.bfloat16,'F16':mx.float16}[case['dtype']]
|
||||||
|
bits,group,rows=case['bits'],case['group'],case['rows']
|
||||||
|
gate=pack(128,256,256,bits,group,7,dtype)
|
||||||
|
up=pack(128,256,256,bits,group,9,dtype) if case['separate'] else None
|
||||||
|
x=pattern(512*256,8).reshape(512,1,256).astype(dtype)
|
||||||
|
r=np.arange(rows,dtype=np.uint32)
|
||||||
|
ids=mx.array((r*125//rows)//2*2+1)
|
||||||
|
row_map=mx.array((r*13)%512)
|
||||||
|
assert kernel.applies(x,row_map,*gate,ids,group_size=group,bits=bits,mode='affine',up=up)
|
||||||
|
kwargs=dict(group_size=group,bits=bits,swiglu=True,up=up)
|
||||||
|
before=kernel.stats()
|
||||||
|
y=kernel.gather_rows_qmm(x,row_map,*gate,ids,**kwargs)
|
||||||
|
assert y is not None,kernel.stats()
|
||||||
|
stock=kernel._stock(x,row_map,*gate,ids,**kwargs)
|
||||||
|
actual=digest(y)
|
||||||
|
assert actual==digest(stock)==case['sha256'],(case,actual,digest(stock))
|
||||||
|
print(json.dumps(dict(case='synthetic_oracle',native=case,exact=True,kernel_before=before,kernel_after=kernel.stats())),flush=True)
|
||||||
|
del x,row_map,ids,gate,up,y,stock
|
||||||
|
mx.clear_cache()
|
||||||
|
|
||||||
|
assert count==72,count
|
||||||
Reference in new issue
Block a user