diff --git a/docs/issue103-analysis/README.md b/docs/issue103-analysis/README.md new file mode 100644 index 0000000..6da37e1 --- /dev/null +++ b/docs/issue103-analysis/README.md @@ -0,0 +1,101 @@ +# MTPLX 2.12.2 review handoff for #103 + +Analysis only, 2026-10-03. Eight independently reproduced native defects were filed as bugs. No production behavior, model revision, runtime shader, default tuning or tokenizer asset was changed. Implementation requires the user's review and subsequent instruction. Some requested stress/branch matrices remain unmeasured; the inventory distinguishes them from confirmed failures. + +## Immutable comparison and inputs + +- Affected DS4Server: `cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983`. +- Previous oracle: MTPLX 2.10.2 `e652d55e2652137a4abcf1312357abbf3eb9d692`. +- Refreshed HEAD and peeled v2.12.2: `9882703f3105363ddc37eca9f97aa09a1d387112`. The range contains 1,077 commits, including merges; applicability was narrowed using source, tests and release notes rather than treating every change as relevant to this backend. +- [Immutable release notes](https://github.com/youssofal/mtplx/blob/9882703f3105363ddc37eca9f97aa09a1d387112/CHANGELOG.md), [runtime pin](https://github.com/youssofal/mtplx/commit/0d56bc6d1940906884fc1ec40f97c180a48ed43b). +- Unchanged `Youssofal/Qwen3.8-Flash-Next-MTPLX-Bare-Speed`, revision `74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`, MLX exactly 0.32.2. Model/config/tokenizer/template/vision/MTP/PLE files were streamed through SHA256 and every checked file matches its stored digest. [environment.json](environment.json) contains actual hashes, sizes, runtime/compiler, hardware/OS, prompt digest and effective model-test options. +- M5 Max/Mac17,7/128GiB, macOS26.6.2, Metal32023.883. Reference environment: mlx-lm0.31.3, transformers5.8.0, tokenizers0.22.2. Both oracle checkouts were retained separately; Python is used only for isolated reference probes. +- Fresh model execution with SSD PLE; each model case restores the same initial prompt snapshot. This is not a claim of cold SSD pages. Tests add synchronization/readbacks and a watchdog; their timings cannot establish product speed. Synthetic cases use explicit dtypes/shapes/values in the test, no image/template or API context-admission setting. + +## Confirmed bugs and diagnostic ownership + +| Area | Issue | Observed failure | Owned receipt cases | +|---|---|---|---| +| A1 | [#104](https://git.rfc1437.de/hugo/DS4Server/issues/104) | Pipeline top-p normalizes only top-k. Minimal expected support {0,1} becomes {0}; official .6/.95/k20 also loses valid candidates. | `sampling_law`, `sampling_matrix` in [native-sampling.jsonl](native-sampling.jsonl) | +| A2 | [#105](https://git.rfc1437.de/hugo/DS4Server/issues/105) | NaN/+inf/all-inf rows fabricate token IDs; finite-winner -inf masks remain legitimate. | `invalid_logits`, `invalid_logits_lazy`, same file | +| B | [#106](https://git.rfc1437.de/hugo/DS4Server/issues/106) | First-primary stop/limit banks a four-token key over three-token trunk state; next-turn 247,643 logits differ. | All [native-model.jsonl](native-model.jsonl) cases | +| D1 | [#107](https://git.rfc1437.de/hugo/DS4Server/issues/107) | Vision presence discards sparse QSA selection, changing actual Attention output with identical positions. Image-aware indexer threading is missing in source. | `vision_sparse_qsa` in [native-vision.jsonl](native-vision.jsonl) | +| D2 | [#108](https://git.rfc1437.de/hugo/DS4Server/issues/108) | A straddling chunk applies plain RoPE to its table-covered image rows; two wrong rows, inside/outside controls exact. | `vision_positions`, same file | +| E | [#109](https://git.rfc1437.de/hugo/DS4Server/issues/109) | Actual BF16 attention gate differs from latest compiled verify law at -6.84375. | `active_attention_gate`, same file; `bf16_gate` in [native-numeric.jsonl](native-numeric.jsonl) | +| F | [#110](https://git.rfc1437.de/hugo/DS4Server/issues/110) | M5 sorted expert gather leaves 32/1,280/8,192 rows unwritten at 3,277/3,404/4,095 tokens; 3,276 and 4,096 controls pass. | [native-gather.jsonl](native-gather.jsonl) | +| I | [#111](https://git.rfc1437.de/hugo/DS4Server/issues/111) | Qwen native pretokenizer misclassifies Unicode number categories, changing exact IDs for circled/Roman/superscript/mathematical numerals. | [native-tokenizer.jsonl](native-tokenizer.jsonl) | + +Each issue contains caller/trigger, actual vs expected, immutable upstream links, severity, native scope, limitations, regression ownership and product acceptance. Existing issues were searched in all states; #93–99 own the completed integrations, not these subsequently reproduced regressions. #100/#102 and optional paging #77 have different roots. No duplicate was created for an existing root cause. + +## Applicability and evidence inventory A–J + +| Area | Disposition and active-route evidence | What is still unmeasured | +|---|---|---| +| A | Two confirmed bugs above. `policy::server_overrides` → `ar::decode` enables pipeline AR when eligible; ordinary target/draft support uses full vocabulary. Latest a2d0b4a and mask tests distinguish invalid from legal masked rows. p=1 control preserves sampling law. | HTTP error injection, invalid-session cleanup and valid retry; MTP invalid target/draft end-to-end. These are explicit #105 acceptance, not observed cleanup failures. | +| B | Confirmed first-primary stop and max-token exits, depths1/2/3, greedy and native .6 sampler, real model. `pending.take`/early break bypasses final commit; final_arrays/capture trusts completeness. Trunk offset3, MTP2 versus bank key4. Warm teacher-force comparison independently confirms missing commit. | Fresh-primary/correction/bonus/accepted-draft exits, cancellation and final-capture failure. GDN/PLE remain at prefill because no final forward runs; individual tensor hashes for each site were not measured. #106 owns the full matrix. | +| C | Source-verified non-applicability of upstream tiny-gap tolerance: `request::LiveSession` requires token-exact complete `starts_with` plus content identity; `owner::align_prompt` resets mismatching prompts. `restore_session` rejects non-prefix before replacing caches. Outer checkpoint requires exact model/context/vocabulary/MTP identity; interrupted/incompatible/corrupt restore tests pass. No attention-only trimming of a later recurrent state was found. B can still make a token-exact bank lie about its state. | Product edits1/4/8/9/40 tokens before end; alternate tokenization/reasoning echo, whitespace, tool-schema ordering/choice/argument/system-message matrices; restart/truncated history. A safe miss is not a confirmed bug. Do not infer all warm reuse correct from checkpoint tests. | +| D | Two confirmed active Attention defects. `qsa_indexer` and qsa_prepare use plain query/block-start positions; latest 03a0a0c/18087e7/8172196 establishes corrected vision contract. Native table builder/rendering already exists. `request::LiveSession` gates on qwen_vision::same_prefix content/whole-image identity; whole-image policy is coarse, with no demonstrated incorrect restore. | Real dim128/topk512 image indexer selection and full long-image logits; multiple-image edit/content/cancel matrices and proof of all restore-span boundaries. Source indexer omission is within #107 scope but has no independent real-image receipt yet. | +| E | BF16 attention verification mismatch is confirmed. PLE/hyper/shared-expert gate lowering must be reviewed by phase; their independent active-site failures have not been asserted. A separate generic compiler probe demonstrates seven-digit constants changing sqrt128 and YaRN factor4 bits. **No scalar-format bug issue:** `qsa_core` uses `qsa_compile::Body.trace`/`Replay::trace` with exact captured operands; QSA prepare/select passes scales separately; whole model `runs` compiles only linear GDN spans without PLE. Full QSA/PLE fixed-M4 compiler is not installed. `qsa_graph` is test-only. | Real 128-dim/512-block near-cutoff IDs/order under actual routes; static YaRN factor4 ordinary/vision; PLE/hyper/shared gate site reproduction and full-model verify/logit comparison. `qsa_divisor`/`yarn_amplitude` receipts are a dormant compiler hazard owned by this report, not proof of active model divergence. | +| F | Confirmed same-dispatcher row defect under M5/0.32.2/large supported chunks; default2048 is below trigger. Unaligned >32767 sorted routes + BM64 produces unwritten sentinel rows. Upstream b36b106 provides pad/slice contract. Pinned MTP header has58 tensors with explicit switch_mlp gate/up/down and scales/biases. `binding::Bank.load` includes mtp.* and binds decoder/feed-forward/linear strictly; full native MTP model load succeeds. [mtp-binding.json](mtp-binding.json) records header shapes/dtypes. Qwen3.5/3.6 fused naming fix13128e2 is not automatically applicable to these keys. | Full model larger-chunk text output and both gate/up/down real geometries. #110 owns those tests; no loader bug was observed. | +| G | Source-verified dynamic reserve-before-update in cache::KvCache/QsaCache; existing native KV/QSA growth/trim/snapshot controls pass. `copy_round` feeds ordinary model forward, not the complete upstream fixed-M4/paged replay owner. Presence of compiled primitives does not establish that lane. No new cache-capacity defect has been demonstrated outside B. | Accepted/rejected copy windows/final pending writes on full-model growth boundaries; verifier width transitions; OOM atomicity/recovery; 64K/128K/near-native-max memory/head chunking and snapshot copy peaks. Existing finite cache controls do not discharge these stress obligations. | +| H | Normal policy keeps lazy_history=false; append_committed evaluates MTP history even without later draft use. eval::evaluate detaches evaluated graph nodes/events; native eval failure/retry, donation, tracer topology, siblings/continuations, graph-event/fence and checkpoint ownership tests pass. Allocator already tracks active/cached/peak/resources, reclaims cached buffers and limits memory to min(1.5×recommended,.95×RAM). Product owner drops snapshots before context and publishes checkpoint only after successful validation. | Long streamed/non-streamed event marathon, many-session pending persistence, interrupted request memory/admission retry and whole-Mac reclaimable/compressor peaks. Short tests and one ~73.9GB model peak do not prove absence of a leak. Latest admission accounting is a policy/measurement candidate until a concrete local defect is reproduced. 2.12.2 dense-model/small-Mac repair is not evidence of a Flash-Next128GiB failure. | +| I | Confirmed native Unicode category bug; Hindi/Thai/vowelled Arabic combining marks and English/code/Chinese/mixed punctuation match exact trained IDs. Upstream transformers loader repair is not our implementation, but its corrected regex/oracle exposes local category error. Both old and refreshed resilient tokenizer agree on those IDs. | Wider Unicode category coverage and rendered API warm/context accounting belong to #111 acceptance. | +| J | Optimization candidates only. New MTPLX kernel/default engagement reviewed against native model/MoE/QSA/history routes. No optional optimization was filed as a bug without measured supported-contract violation. | Paired serial Bare-Speed workloads for TTFT, prefill/decode, accepted tokens/depth, dispatch/readback, memory/thermal/variance. No new product performance regression or 60tok/s/5%/75.9coding acceptance result was measured. | + +Important upstream follow-ups: [long-context verify head chunking](https://github.com/youssofal/mtplx/commit/57fa9174f9a44f4e8cd0e56ef0aaa23b556963bb), [lazy write lifetime](https://github.com/youssofal/mtplx/commit/73628baa68eb0175bb0e7189ade0151e4d4cb26f), [history scheduling](https://github.com/youssofal/mtplx/commit/011f23f9d0883c98c2654cdae139afc8384576e2). Their upstream failures alone are not local bug proof. + +## Optimization opportunities retained for separate authorization + +- M5 4096 prefill/sparse-from16K needs #110 resolved and exact output/state validation before tuning. +- Cache-direct sparse prefill, simdgroup row selectors, runtime block counts and less recompilation. +- Fused hyper read/write, GDN prework/norm, expert combine and M5 dequantize/GEMM; actual current compiled spans exclude QSA/PLE. +- Cache-only MTP history, reduced readbacks, PLE read-ahead/overlap. +- Full fixed-M4 model replay, verify-width pooled/QSA/HC fusion and routed-down reduction; installing that owner makes presently dormant capacity/scalar obligations active. +- 8192 capacity buckets, one-copy/per-layer growth, recurrent boundary retention and content-exact image reuse. + +Validate semantics first and then paired performance on unchanged Bare-Speed/M5 Max. Published Optimized-Speed numbers and kernel-only measurements do not satisfy this repository's same-machine contract. + +## Reproduce without this chat + +[issue103_analysis.rs](../../src/engine/metal/qwen_mtplx/issue103_analysis.rs) is cfg(test) and ignored normally. It deliberately pins/prints broken behavior on the affected revision; a green diagnostic means confirmed reproduction. Future fixes must turn owned assertions into correct-contract regression tests, not weaken the oracle. Six tests share minimal primitives and actual native Attention/model paths. No upstream production code/test suite was copied into production. + +```sh +cargo test --all-features --bin ds4-server issue103_ --no-run +cargo build --bin test-supervisor +export DS4_QWEN38_ARTIFACTS='/path/to/the/unchanged/qwen3.8-flash-next' +export DS4_ISSUE103_ORACLE="$PWD/docs/issue103-analysis/latest-oracle.jsonl" +# Use the ds4_server test executable printed by cargo (not cargo itself). +# Full model requires enough RAM for SSD-PLE Bare-Speed; limit is 98GiB. +target/debug/test-supervisor 100352 60 120 --command /path/to/test-binary issue103_ --ignored --nocapture --test-threads=1 +``` + +Model-free sampling/numeric/vision/gather cases can be selected individually under a4096MiB watchdog. Tokenizer additionally needs the artifact and oracle environment variables. `issue103_mtp_final_receipts` is the only full-model test. On a different Metal device, don't treat failure to select the M5 NAX kernel as reproducing #110. + +For the independent Python oracle, install each exact MTPLX revision in a separate checkout. Use MLX0.32.2 and the recorded tokenizer library versions; in the existing analysis environment the command was: + +```sh +PYTHONPATH=/path/to/mtplx-9882703f /path/to/reference-python tools/issue103-reference.py "$DS4_QWEN38_ARTIFACTS" +# Repeat with the e652d55e checkout; do not change tokenizer assets. +``` + +The probe reads only tokenizer/config/template/merges; no model tensors. [latest-oracle.jsonl](latest-oracle.jsonl) and [old-oracle.jsonl](old-oracle.jsonl) are preserved even if the external scratch environments disappear. `bf16_verify_gate` is unavailable on the old reference and recorded explicitly. Bad-row oracle exceptions are deliberate; warnings are not harness failures. + +Targeted upstream test files: non_finite_logits_and_lazy_sampler_law, mtp_final_state_covers_committed_tokens, qwen4_vision_mrope, float32_operand, qwen4_yarn_amplitude, moe_sorted_gather, qwen4_moe_sorted_rows, tokenizer_pretokenizer_repair; additionally attention_gate_exactness and qwen4_verify_gate_sites. See [upstream-tests.log](upstream-tests.log) and [upstream-gates.log](upstream-gates.log). One installed27B asset-dependent tokenizer case was skipped; native exact Bare-Speed tokenizer was separately exercised. + +Existing native controls were run serially with their own watchdogs: mtplx_kv_cache_graph_matches_reference, mtplx_qsa_cache_graph_matches_reference, graph-event/fence tests and qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state. [native-controls.log](native-controls.log) records exact filters/results; [native-eval-controls.log](native-eval-controls.log) covers four eval/lifetime/retry tests. These test their finite scenarios, not unrestricted long-run resource behavior. + +## Delivery checks and unresolved verification + +The durable evidence commit contains only cfg(test) diagnostics, an isolated reference script and these reports/receipts. All confirmed failures have owning issues; the generic scalar-format experiment belongs to this report with the dormant-route disposition above. + +The broad branch/stress cases in the inventory still need evidence before claiming those paths correct. They are explicit review/acceptance or follow-up investigation obligations, not additional confirmed bugs and not authorization for production changes. #103 remains open to keep that unfinished verification visible. + +Validation completed: + +- `cargo fmt --all -- --check`: passed. +- `cargo clippy --all-targets --all-features -- -D warnings`: passed ([log](clippy.log)). +- `make bundle`: passed including app packaging/ad-hoc signature ([log](bundle.log)). +- `cargo test --all-features`: first parallel run had two unrelated existing test failures ([log](tests.log)): a generated seven-digit all-numeric git prefix was parsed as a YAML integer; process-wide CPU accounting exceeded its5ms concurrent tolerance. No unrelated code was changed. Justified serial rerun `cargo test --all-features -- --test-threads=1`: passed307 main tests,210 ignored, plus8 tool tests ([log](tests-serial.log)). +- Explicit ignored native diagnostics: six passed; final sampler matrix and tokenizer assertions separately passed after their additions. Existing ignored native controls all passed. Targeted official upstream gates:12 passed; other selected upstream tests passed with one installed27B tokenizer asset case skipped. Ignored full-model tests outside this analysis were not broadly executed. + +Review the remaining inventory obligations before claiming exhaustive branch, long-context, resource or performance coverage. diff --git a/docs/issue103-analysis/bundle.log b/docs/issue103-analysis/bundle.log new file mode 100644 index 0000000..def1986 --- /dev/null +++ b/docs/issue103-analysis/bundle.log @@ -0,0 +1,8 @@ +cargo build --release --all-targets --all-features + Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server) + Finished `release` profile [optimized] target(s) in 31.28s +warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6 +note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1` +cargo packager --release --formats app +codesign --force --deep --sign - --timestamp=none target/release/DS4Server.app +target/release/DS4Server.app: replacing existing signature diff --git a/docs/issue103-analysis/clippy.log b/docs/issue103-analysis/clippy.log new file mode 100644 index 0000000..604dc1d --- /dev/null +++ b/docs/issue103-analysis/clippy.log @@ -0,0 +1,4 @@ + Checking ds4-server v0.1.0 (/Users/gb/Projects/DS4Server) + Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.79s +warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6 +note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1` diff --git a/docs/issue103-analysis/environment.json b/docs/issue103-analysis/environment.json new file mode 100644 index 0000000..868d750 --- /dev/null +++ b/docs/issue103-analysis/environment.json @@ -0,0 +1,256 @@ +{ + "analysis_date": "2026-10-03", + "affected_revision": "cea8fc6f7c7aa7aeb97ebc303594ae4088ef6983", + "old_mtplx": "e652d55e2652137a4abcf1312357abbf3eb9d692", + "latest_mtplx": "9882703f3105363ddc37eca9f97aa09a1d387112", + "latest_release": "2.12.2", + "model": "Youssofal/Qwen3.8-Flash-Next-MTPLX-Bare-Speed", + "model_revision": "74559cdf34fbfc0b593de72d17e93f37fd4f9ea7", + "model_root": "/Users/gb/Library/Application Support/de.rfc1437.ds4server/models/qwen3.8-flash-next", + "hardware": "Apple M5 Max, Mac17,7, 128 GiB", + "os": "ProductName:\t\tmacOS\nProductVersion:\t\t26.6.2\nBuildVersion:\t\t25G83", + "rustc": "rustc 1.97.1 (8bab26f4f 2026-07-14)", + "cargo": "cargo 1.97.1 (c980f4866 2026-06-30)", + "mlx": "0.32.2", + "artifact_files": { + "LICENSE": { + "bytes": 3235, + "sha256": "a0dc422560841fd68e06d974907f8b4c709bca44a67daad2b528437bdf676c08", + "stored_sha256": "a0dc422560841fd68e06d974907f8b4c709bca44a67daad2b528437bdf676c08", + "matches_stored": true + }, + "chat_template.jinja": { + "bytes": 8952, + "sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041", + "stored_sha256": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041", + "matches_stored": true + }, + "config.json": { + "bytes": 227092, + "sha256": "826302af9ff3293cfc9975c9a5a96546fa0aa9f0808505adfc30a62fef806965", + "stored_sha256": "826302af9ff3293cfc9975c9a5a96546fa0aa9f0808505adfc30a62fef806965", + "matches_stored": true + }, + "generation_config.json": { + "bytes": 202, + "sha256": "e70c136c1b78ddc1fb0905bac8e733a4dc448d4f852a5dd75143fffc70be550e", + "stored_sha256": "e70c136c1b78ddc1fb0905bac8e733a4dc448d4f852a5dd75143fffc70be550e", + "matches_stored": true + }, + "merges.txt": { + "bytes": 3353259, + "sha256": "a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d", + "stored_sha256": "a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d", + "matches_stored": true + }, + "model-00001-of-00017.safetensors": { + "bytes": 4666167150, + "sha256": "a27232c9434b9d8961f198cf36f44e228b3f23a5f8d874ba4ca8af32a1b23ffc", + "stored_sha256": "a27232c9434b9d8961f198cf36f44e228b3f23a5f8d874ba4ca8af32a1b23ffc", + "matches_stored": true + }, + "model-00002-of-00017.safetensors": { + "bytes": 4299165712, + "sha256": "6268ae80950f6b935fca672fae5aa455eb3ea4f139730e54e6e5f8581faa94dc", + "stored_sha256": "6268ae80950f6b935fca672fae5aa455eb3ea4f139730e54e6e5f8581faa94dc", + "matches_stored": true + }, + "model-00003-of-00017.safetensors": { + "bytes": 4666167168, + "sha256": "71a9abbd420dce567f2c06593bfee2d6cea681989d51eaef1b1c5dffd65a844b", + "stored_sha256": "71a9abbd420dce567f2c06593bfee2d6cea681989d51eaef1b1c5dffd65a844b", + "matches_stored": true + }, + "model-00004-of-00017.safetensors": { + "bytes": 4299165702, + "sha256": "c8112da735bb1c8dbc4757926ab8ec1eba7a707a8767c7e43c6150586cf63a4c", + "stored_sha256": "c8112da735bb1c8dbc4757926ab8ec1eba7a707a8767c7e43c6150586cf63a4c", + "matches_stored": true + }, + "model-00005-of-00017.safetensors": { + "bytes": 4666167169, + "sha256": "48258b931e7596c1b43c8b38c044060787ef6d85cca2c1935852bc962801f792", + "stored_sha256": "48258b931e7596c1b43c8b38c044060787ef6d85cca2c1935852bc962801f792", + "matches_stored": true + }, + "model-00006-of-00017.safetensors": { + "bytes": 4299165708, + "sha256": "f357d2d66f7b768ebb19ba464556bbe46c572cf3cd1f8226faddf9216b150d0c", + "stored_sha256": "f357d2d66f7b768ebb19ba464556bbe46c572cf3cd1f8226faddf9216b150d0c", + "matches_stored": true + }, + "model-00007-of-00017.safetensors": { + "bytes": 4666167168, + "sha256": "d3152e808d013e3afa4a9c391f6d8088e74a10303c89d45fb61dad7e457160f6", + "stored_sha256": "d3152e808d013e3afa4a9c391f6d8088e74a10303c89d45fb61dad7e457160f6", + "matches_stored": true + }, + "model-00008-of-00017.safetensors": { + "bytes": 4299165697, + "sha256": "1d93bd4872f338fa19416bac64ebeb05aedf2721a07ac48e37c53921252c95c1", + "stored_sha256": "1d93bd4872f338fa19416bac64ebeb05aedf2721a07ac48e37c53921252c95c1", + "matches_stored": true + }, + "model-00009-of-00017.safetensors": { + "bytes": 4666167162, + "sha256": "d22d4be580296e3d4fa5f4fe5902c8fc448d8204b6dcde4641f87b9c675930ca", + "stored_sha256": "d22d4be580296e3d4fa5f4fe5902c8fc448d8204b6dcde4641f87b9c675930ca", + "matches_stored": true + }, + "model-00010-of-00017.safetensors": { + "bytes": 4299165708, + "sha256": "111d0166c888cabb7d6fc19c4bb7eb42e2b51946f71b4983dd750b031755637e", + "stored_sha256": "111d0166c888cabb7d6fc19c4bb7eb42e2b51946f71b4983dd750b031755637e", + "matches_stored": true + }, + "model-00011-of-00017.safetensors": { + "bytes": 4666167159, + "sha256": "73dad882cf94a5486cfe0dcda876b0f740556bc469608d2ced7a07540e699803", + "stored_sha256": "73dad882cf94a5486cfe0dcda876b0f740556bc469608d2ced7a07540e699803", + "matches_stored": true + }, + "model-00012-of-00017.safetensors": { + "bytes": 4299165706, + "sha256": "a6057d5a06276461760a4f4e88991550a43abdc5f8afacef06e860f5e7843393", + "stored_sha256": "a6057d5a06276461760a4f4e88991550a43abdc5f8afacef06e860f5e7843393", + "matches_stored": true + }, + "model-00013-of-00017.safetensors": { + "bytes": 4666167174, + "sha256": "ff25dc6e1c250c9b9ece83d59111a4c4055dad1b1f160546c614dcc17395b152", + "stored_sha256": "ff25dc6e1c250c9b9ece83d59111a4c4055dad1b1f160546c614dcc17395b152", + "matches_stored": true + }, + "model-00014-of-00017.safetensors": { + "bytes": 4299165704, + "sha256": "d8c9de3ae28abccc8e3da09dc92fbbed4039d6d65dabb2657edebc11b2b09652", + "stored_sha256": "d8c9de3ae28abccc8e3da09dc92fbbed4039d6d65dabb2657edebc11b2b09652", + "matches_stored": true + }, + "model-00015-of-00017.safetensors": { + "bytes": 4666167146, + "sha256": "a5438d468074c709febdeadb9f6a2db54fa9fb30a37c33de78584f28d6fa1191", + "stored_sha256": "a5438d468074c709febdeadb9f6a2db54fa9fb30a37c33de78584f28d6fa1191", + "matches_stored": true + }, + "model-00016-of-00017.safetensors": { + "bytes": 4300831084, + "sha256": "4a07409bd568cd1435641495d90c805dcb0d0e5d8dc64409273d604a7d06b342", + "stored_sha256": "4a07409bd568cd1435641495d90c805dcb0d0e5d8dc64409273d604a7d06b342", + "matches_stored": true + }, + "model-00017-of-00017.safetensors": { + "bytes": 18354282, + "sha256": "9c064e0b8dc0054e2b453fe2b8450c287e1ede6e74bb74ecf5aaed1ef9271913", + "stored_sha256": "9c064e0b8dc0054e2b453fe2b8450c287e1ede6e74bb74ecf5aaed1ef9271913", + "matches_stored": true + }, + "model-vision.safetensors": { + "bytes": 897899145, + "sha256": "5aff6983f36f52acbecc1f61f703ae7b927e74525bf3b1133ba276bb7a5b52b1", + "stored_sha256": "5aff6983f36f52acbecc1f61f703ae7b927e74525bf3b1133ba276bb7a5b52b1", + "matches_stored": true + }, + "model.safetensors.index.json": { + "bytes": 284868, + "sha256": "81e77f5b25bb29e25e239ae2f9f324c9cada379ee600441d6c229f1d95a13443", + "stored_sha256": "81e77f5b25bb29e25e239ae2f9f324c9cada379ee600441d6c229f1d95a13443", + "matches_stored": true + }, + "mtp.safetensors": { + "bytes": 1672575532, + "sha256": "1bd4b2fc20a93b53f22b4cab0b14d68bc0fe666bb64e4e2f898872c30d7d26ae", + "stored_sha256": "1bd4b2fc20a93b53f22b4cab0b14d68bc0fe666bb64e4e2f898872c30d7d26ae", + "matches_stored": true + }, + "mtplx_runtime.json": { + "bytes": 2676, + "sha256": "381c7666872f878542494a48a874bb0784c72a955d18cc077324d3c9e74b8cdb", + "stored_sha256": "381c7666872f878542494a48a874bb0784c72a955d18cc077324d3c9e74b8cdb", + "matches_stored": true + }, + "ngram-table.safetensors": { + "bytes": 32000154008, + "sha256": "af7f0880c1798e60353b07cc6b0faa0d41f78734818d858ff013ab7e3bce9de5", + "stored_sha256": "af7f0880c1798e60353b07cc6b0faa0d41f78734818d858ff013ab7e3bce9de5", + "matches_stored": true + }, + "preprocessor_config.json": { + "bytes": 390, + "sha256": "27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516", + "stored_sha256": "27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516", + "matches_stored": true + }, + "processor_config.json": { + "bytes": 367, + "sha256": "dcc7ce0121d6d1c514804e68e5673e04c239ad2547339ea6330289d272a748f8", + "stored_sha256": "dcc7ce0121d6d1c514804e68e5673e04c239ad2547339ea6330289d272a748f8", + "matches_stored": true + }, + "tokenizer.json": { + "bytes": 12809320, + "sha256": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3", + "stored_sha256": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3", + "matches_stored": true + }, + "tokenizer_config.json": { + "bytes": 17928, + "sha256": "b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27", + "stored_sha256": "b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27", + "matches_stored": true + }, + "video_preprocessor_config.json": { + "bytes": 385, + "sha256": "7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13", + "stored_sha256": "7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13", + "matches_stored": true + }, + "vocab.json": { + "bytes": 6722759, + "sha256": "ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003", + "stored_sha256": "ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003", + "matches_stored": true + } + }, + "production_changes": false, + "model_repro": { + "prompt_ids": [ + 9707, + 11, + 1075 + ], + "prompt_ids_i32le_sha256": "cf43b34fcb65a553b909fced3d7a1741c2f8bdd373c3928ccd0a948f2ee3ffc0", + "template": null, + "images": null, + "chunk_size": 2048, + "context": "direct Execution, not API context admission; three-token prompt", + "seed": 0, + "temperature": [ + 0.0, + 0.6 + ], + "depth": [ + 1, + 2, + 3 + ], + "max_tokens": [ + 1, + 8 + ], + "stop_ids": [ + 11, + 314 + ], + "cold_warm": "fresh model execution; restored identical prompt snapshot for every case; OS file cache not flushed", + "memory_limit_mib": 100352, + "ple": "SSD streamed; MTPLX_NGRAM_RESIDENT unset; resident_policy asserted false", + "sampler": { + "top_p": 0.95, + "top_k": 20, + "presence_penalty": 0, + "frequency_penalty": 0, + "same_target_and_draft": true + } + } +} diff --git a/docs/issue103-analysis/issues.json b/docs/issue103-analysis/issues.json new file mode 100644 index 0000000..de5a074 --- /dev/null +++ b/docs/issue103-analysis/issues.json @@ -0,0 +1,74 @@ +[ + { + "slug": "sampling-law", + "title": "Use full-vocabulary top-p law in Qwen AR pipeline", + "area": "A1", + "test": "issue103_sampling_receipts", + "upstream": "a2d0b4a493923f9c3b13178713e9f78413a1d2d7", + "issue": 104, + "url": "https://git.rfc1437.de/hugo/DS4Server/issues/104" + }, + { + "slug": "invalid-logits", + "title": "Reject invalid Qwen logits instead of emitting fabricated tokens", + "area": "A2", + "test": "issue103_sampling_receipts", + "upstream": "a2d0b4a493923f9c3b13178713e9f78413a1d2d7", + "issue": 105, + "url": "https://git.rfc1437.de/hugo/DS4Server/issues/105" + }, + { + "slug": "mtp-final", + "title": "Commit the final MTP primary before banking warm session state", + "area": "B", + "test": "issue103_mtp_final_receipts", + "upstream": "1a977b65b002d7efcdfeed69c209d2129abc2b67", + "issue": 106, + "url": "https://git.rfc1437.de/hugo/DS4Server/issues/106" + }, + { + "slug": "vision-sparse", + "title": "Preserve sparse QSA and image-aware indexer positions for vision turns", + "area": "D1", + "test": "issue103_vision_position_receipts", + "upstream": "03a0a0c3727cb8d7d2957f97dedabc50ff65194f", + "issue": 107, + "url": "https://git.rfc1437.de/hugo/DS4Server/issues/107" + }, + { + "slug": "vision-straddle", + "title": "Apply M-RoPE row by row across the vision position-table boundary", + "area": "D2", + "test": "issue103_vision_position_receipts", + "upstream": "81721967881d77974f11cdd355f6127c92558081", + "issue": 108, + "url": "https://git.rfc1437.de/hugo/DS4Server/issues/108" + }, + { + "slug": "bf16-gate", + "title": "Preserve compiled BF16 attention-gate math in MTP verification", + "area": "E", + "test": "issue103_vision_position_receipts", + "upstream": "a075e78b5a685d974d64984e4a50267b35f75054", + "issue": 109, + "url": "https://git.rfc1437.de/hugo/DS4Server/issues/109" + }, + { + "slug": "sorted-gather", + "title": "Prevent uninitialized expert outputs for large unaligned M5 prefill chunks", + "area": "F", + "test": "issue103_sorted_gather_receipts", + "upstream": "b36b106f1aea936072d71aa6cfaa53094da0e340", + "issue": 110, + "url": "https://git.rfc1437.de/hugo/DS4Server/issues/110" + }, + { + "slug": "tokenizer-number", + "title": "Tokenize all Unicode number categories with the Qwen3 contract", + "area": "I", + "test": "issue103_tokenizer_receipts", + "upstream": "54b1063a6bd98e077eea2421bfed7e634cf5f459", + "issue": 111, + "url": "https://git.rfc1437.de/hugo/DS4Server/issues/111" + } +] diff --git a/docs/issue103-analysis/latest-oracle.jsonl b/docs/issue103-analysis/latest-oracle.jsonl new file mode 100644 index 0000000..8ed610a --- /dev/null +++ b/docs/issue103-analysis/latest-oracle.jsonl @@ -0,0 +1,35 @@ +{"case": "environment", "mlx": "0.32.2", "device": {"device_name": "Apple M5 Max", "max_recommended_working_set_size": 115448725504, "memory_size": 137438953472, "architecture": "applegpu_g17s", "max_buffer_length": 86586540032, "resource_limit": 499000}, "hashes": {"config.json": "826302af9ff3293cfc9975c9a5a96546fa0aa9f0808505adfc30a62fef806965", "tokenizer.json": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3", "merges.txt": "a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d", "chat_template.jinja": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}} +{"case": "sampling_law", "probabilities": [0.5714285675041357, 0.4285714324958642]} +{"case": "sampling_law_lazy", "counts": [76, 52]} +{"case": "sampling_matrix", "temperature": 0.6, "top_k": 20, "top_p": 0.95, "ordinary_probabilities": [0.5081967376594639, 0.3811475366342391, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "lazy_counts": [255, 201, 2, 4, 2, 3, 3, 4, 5, 5, 2, 4, 3, 2, 2, 3, 2, 3, 2, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]} +{"case": "sampling_matrix", "temperature": 0.2, "top_k": 8, "top_p": 0.8, "ordinary_probabilities": [0.5486725619186343, 0.4115044341651371, 0.006637167319371426, 0.006637167319371426, 0.006637167319371426, 0.006637167319371426, 0.006637167319371426, 0.006637167319371426, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "lazy_counts": [285, 210, 1, 4, 2, 1, 4, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]} +{"case": "sampling_matrix", "temperature": 1.0, "top_k": 2, "top_p": 1.0, "ordinary_probabilities": [0.5714285675041357, 0.4285714324958642, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "lazy_counts": [299, 213, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]} +{"case": "invalid_logits", "row": "nan", "temperature": 0.0, "error": "NonFiniteLogitsError"} +{"case": "invalid_logits", "row": "nan", "temperature": 0.6, "error": "NonFiniteLogitsError"} +{"case": "invalid_logits", "row": "nan", "temperature": 1.0, "error": "NonFiniteLogitsError"} +{"case": "invalid_logits_lazy", "row": "nan", "token": -1} +{"case": "invalid_logits", "row": "positive_inf", "temperature": 0.0, "error": "NonFiniteLogitsError"} +{"case": "invalid_logits", "row": "positive_inf", "temperature": 0.6, "error": "NonFiniteLogitsError"} +{"case": "invalid_logits", "row": "positive_inf", "temperature": 1.0, "error": "NonFiniteLogitsError"} +{"case": "invalid_logits_lazy", "row": "positive_inf", "token": -1} +{"case": "invalid_logits", "row": "all_negative_inf", "temperature": 0.0, "error": "NonFiniteLogitsError"} +{"case": "invalid_logits", "row": "all_negative_inf", "temperature": 0.6, "error": "NonFiniteLogitsError"} +{"case": "invalid_logits", "row": "all_negative_inf", "temperature": 1.0, "error": "NonFiniteLogitsError"} +{"case": "invalid_logits_lazy", "row": "all_negative_inf", "token": -1} +{"case": "invalid_logits", "row": "valid_mask", "temperature": 0.0, "token": 1} +{"case": "invalid_logits", "row": "valid_mask", "temperature": 0.6, "token": 1} +{"case": "invalid_logits", "row": "valid_mask", "temperature": 1.0, "token": 1} +{"case": "invalid_logits_lazy", "row": "valid_mask", "token": 3} +{"case": "tokenizer", "text": "भारत एक विशाल देश है जिसकी संस्कृति बहुत पुरानी है।", "ids": [149755, 148504, 76519, 171549, 165997, 151224, 197734, 170046, 184864, 160357, 172281, 237627, 158827, 153570, 199638, 80752, 151859, 181117, 170046, 154845]} +{"case": "tokenizer", "text": "ประเทศไทยมีวัฒนธรรมที่เก่าแก่และสวยงามมาก", "ids": [156150, 148450, 186319, 148285, 178118, 155545, 149119, 162566, 149196]} +{"case": "tokenizer", "text": "إِنَّ اللُّغَةَ الْعَرَبِيَّةَ جَمِيلَةٌ", "ids": [88792, 164372, 148784, 159613, 78958, 191556, 205629, 153458, 150765, 151003, 178402, 27004, 79333, 152397, 216808, 233790]} +{"case": "tokenizer", "text": "The quick brown fox jumps over the lazy dog near the river bank.", "ids": [760, 3841, 13477, 37550, 33075, 888, 279, 15217, 5388, 3043, 279, 14367, 5883, 13]} +{"case": "tokenizer", "text": "def foo(x):\n return x + 1\n", "ids": [727, 14785, 2007, 1590, 198, 262, 460, 830, 478, 220, 16, 198]} +{"case": "tokenizer", "text": "中国的文化历史悠久。", "ids": [108733, 96655, 125188, 1710]} +{"case": "tokenizer", "text": "a🙂b ©c ∑d —e …f €g ™h\n", "ids": [64, 169171, 65, 7013, 66, 11654, 239, 67, 1892, 68, 4444, 69, 12612, 70, 27717, 95, 71, 198]} +{"case": "tokenizer", "text": "x́y १२३ ١٢٣ ①Ⅷ²‍Z", "ids": [87, 52033, 88, 220, 12256, 100, 12256, 101, 12256, 102, 220, 149, 94, 149, 95, 149, 96, 220, 46701, 254, 68086, 100, 28495, 373, 235, 57]} +{"case": "tokenizer", "text": " ①", "ids": [220, 46701, 254]} +{"case": "tokenizer", "text": " Ⅷ", "ids": [220, 68086, 100]} +{"case": "tokenizer", "text": " ²", "ids": [220, 28495]} +{"case": "tokenizer", "text": " 𝟘", "ids": [220, 54362, 253, 246]} +{"case": "bf16_verify_gate", "values": [0.001068115234375, 0.001068115234375, 0.001068115234375, 0.001068115234375]} diff --git a/docs/issue103-analysis/mtp-binding.json b/docs/issue103-analysis/mtp-binding.json new file mode 100644 index 0000000..b663d9f --- /dev/null +++ b/docs/issue103-analysis/mtp-binding.json @@ -0,0 +1,641 @@ +{ + "__metadata__": null, + "mtp.fc_embedding.weight": { + "data_offsets": [ + 620910240, + 634017440 + ], + "dtype": "BF16", + "shape": [ + 2560, + 2560 + ] + }, + "mtp.fc_hidden.weight": { + "data_offsets": [ + 1179738016, + 1192845216 + ], + "dtype": "BF16", + "shape": [ + 2560, + 2560 + ] + }, + "mtp.hyper_connection_mixer.hc_norm.weight": { + "data_offsets": [ + 640571040, + 640591520 + ], + "dtype": "BF16", + "shape": [ + 10240 + ] + }, + "mtp.hyper_connection_mixer.input_mix_weight_down.weight": { + "data_offsets": [ + 746319776, + 752873376 + ], + "dtype": "BF16", + "shape": [ + 320, + 10240 + ] + }, + "mtp.hyper_connection_mixer.input_mix_weight_up.weight": { + "data_offsets": [ + 614304160, + 620857760 + ], + "dtype": "BF16", + "shape": [ + 10240, + 320 + ] + }, + "mtp.layers.0.attn_hyper_connection.block_inject_weight.weight": { + "data_offsets": [ + 80973824, + 81055744 + ], + "dtype": "BF16", + "shape": [ + 4, + 10240 + ] + }, + "mtp.layers.0.attn_hyper_connection.hc_norm.weight": { + "data_offsets": [ + 614283680, + 614304160 + ], + "dtype": "BF16", + "shape": [ + 10240 + ] + }, + "mtp.layers.0.attn_hyper_connection.input_mix_weight_down.weight": { + "data_offsets": [ + 753754016, + 760307616 + ], + "dtype": "BF16", + "shape": [ + 320, + 10240 + ] + }, + "mtp.layers.0.attn_hyper_connection.input_mix_weight_up.weight": { + "data_offsets": [ + 634017440, + 640571040 + ], + "dtype": "BF16", + "shape": [ + 10240, + 320 + ] + }, + "mtp.layers.0.mlp.gate.biases": { + "data_offsets": [ + 194699680, + 194740640 + ], + "dtype": "BF16", + "shape": [ + 512, + 40 + ] + }, + "mtp.layers.0.mlp.gate.scales": { + "data_offsets": [ + 194740640, + 194781600 + ], + "dtype": "BF16", + "shape": [ + 512, + 40 + ] + }, + "mtp.layers.0.mlp.gate.weight": { + "data_offsets": [ + 193388960, + 194699680 + ], + "dtype": "U32", + "shape": [ + 512, + 640 + ] + }, + "mtp.layers.0.mlp.shared_expert.down_proj.biases": { + "data_offsets": [ + 140908960, + 140960160 + ], + "dtype": "BF16", + "shape": [ + 2560, + 10 + ] + }, + "mtp.layers.0.mlp.shared_expert.down_proj.scales": { + "data_offsets": [ + 614232480, + 614283680 + ], + "dtype": "BF16", + "shape": [ + 2560, + 10 + ] + }, + "mtp.layers.0.mlp.shared_expert.down_proj.weight": { + "data_offsets": [ + 640642720, + 641461920 + ], + "dtype": "U32", + "shape": [ + 2560, + 80 + ] + }, + "mtp.layers.0.mlp.shared_expert.gate_proj.biases": { + "data_offsets": [ + 640591520, + 640642720 + ], + "dtype": "BF16", + "shape": [ + 640, + 40 + ] + }, + "mtp.layers.0.mlp.shared_expert.gate_proj.scales": { + "data_offsets": [ + 19134464, + 19185664 + ], + "dtype": "BF16", + "shape": [ + 640, + 40 + ] + }, + "mtp.layers.0.mlp.shared_expert.gate_proj.weight": { + "data_offsets": [ + 752934816, + 753754016 + ], + "dtype": "U32", + "shape": [ + 640, + 320 + ] + }, + "mtp.layers.0.mlp.shared_expert.up_proj.biases": { + "data_offsets": [ + 140038560, + 140089760 + ], + "dtype": "BF16", + "shape": [ + 640, + 40 + ] + }, + "mtp.layers.0.mlp.shared_expert.up_proj.scales": { + "data_offsets": [ + 620857760, + 620908960 + ], + "dtype": "BF16", + "shape": [ + 640, + 40 + ] + }, + "mtp.layers.0.mlp.shared_expert.up_proj.weight": { + "data_offsets": [ + 140089760, + 140908960 + ], + "dtype": "U32", + "shape": [ + 640, + 320 + ] + }, + "mtp.layers.0.mlp.shared_expert_gate.biases": { + "data_offsets": [ + 140038400, + 140038480 + ], + "dtype": "BF16", + "shape": [ + 1, + 40 + ] + }, + "mtp.layers.0.mlp.shared_expert_gate.scales": { + "data_offsets": [ + 140038480, + 140038560 + ], + "dtype": "BF16", + "shape": [ + 1, + 40 + ] + }, + "mtp.layers.0.mlp.shared_expert_gate.weight": { + "data_offsets": [ + 620908960, + 620910240 + ], + "dtype": "U32", + "shape": [ + 1, + 320 + ] + }, + "mtp.layers.0.mlp.switch_mlp.down_proj.biases": { + "data_offsets": [ + 140960160, + 193388960 + ], + "dtype": "BF16", + "shape": [ + 512, + 2560, + 20 + ] + }, + "mtp.layers.0.mlp.switch_mlp.down_proj.scales": { + "data_offsets": [ + 641461920, + 693890720 + ], + "dtype": "BF16", + "shape": [ + 512, + 2560, + 20 + ] + }, + "mtp.layers.0.mlp.switch_mlp.down_proj.weight": { + "data_offsets": [ + 1253138336, + 1672568736 + ], + "dtype": "U32", + "shape": [ + 512, + 2560, + 80 + ] + }, + "mtp.layers.0.mlp.switch_mlp.gate_proj.biases": { + "data_offsets": [ + 693890976, + 746319776 + ], + "dtype": "BF16", + "shape": [ + 512, + 640, + 80 + ] + }, + "mtp.layers.0.mlp.switch_mlp.gate_proj.scales": { + "data_offsets": [ + 20271104, + 72699904 + ], + "dtype": "BF16", + "shape": [ + 512, + 640, + 80 + ] + }, + "mtp.layers.0.mlp.switch_mlp.gate_proj.weight": { + "data_offsets": [ + 760307616, + 1179738016 + ], + "dtype": "U32", + "shape": [ + 512, + 640, + 320 + ] + }, + "mtp.layers.0.mlp.switch_mlp.up_proj.biases": { + "data_offsets": [ + 87609600, + 140038400 + ], + "dtype": "BF16", + "shape": [ + 512, + 640, + 80 + ] + }, + "mtp.layers.0.mlp.switch_mlp.up_proj.scales": { + "data_offsets": [ + 1192845216, + 1245274016 + ], + "dtype": "BF16", + "shape": [ + 512, + 640, + 80 + ] + }, + "mtp.layers.0.mlp.switch_mlp.up_proj.weight": { + "data_offsets": [ + 194802080, + 614232480 + ], + "dtype": "U32", + "shape": [ + 512, + 640, + 320 + ] + }, + "mtp.layers.0.mlp_hyper_connection.block_inject_weight.weight": { + "data_offsets": [ + 74338304, + 74420224 + ], + "dtype": "BF16", + "shape": [ + 4, + 10240 + ] + }, + "mtp.layers.0.mlp_hyper_connection.hc_norm.weight": { + "data_offsets": [ + 752914336, + 752934816 + ], + "dtype": "BF16", + "shape": [ + 10240 + ] + }, + "mtp.layers.0.mlp_hyper_connection.input_mix_weight_down.weight": { + "data_offsets": [ + 81055744, + 87609344 + ], + "dtype": "BF16", + "shape": [ + 320, + 10240 + ] + }, + "mtp.layers.0.mlp_hyper_connection.input_mix_weight_up.weight": { + "data_offsets": [ + 74420224, + 80973824 + ], + "dtype": "BF16", + "shape": [ + 10240, + 320 + ] + }, + "mtp.layers.0.self_attn.indexer.index_qk_proj.biases": { + "data_offsets": [ + 19185664, + 19236864 + ], + "dtype": "BF16", + "shape": [ + 640, + 40 + ] + }, + "mtp.layers.0.self_attn.indexer.index_qk_proj.scales": { + "data_offsets": [ + 19236864, + 19288064 + ], + "dtype": "BF16", + "shape": [ + 640, + 40 + ] + }, + "mtp.layers.0.self_attn.indexer.index_qk_proj.weight": { + "data_offsets": [ + 72699904, + 74338304 + ], + "dtype": "U32", + "shape": [ + 640, + 640 + ] + }, + "mtp.layers.0.self_attn.indexer.k_layernorm.weight": { + "data_offsets": [ + 693890720, + 693890976 + ], + "dtype": "BF16", + "shape": [ + 128 + ] + }, + "mtp.layers.0.self_attn.indexer.q_layernorm.weight": { + "data_offsets": [ + 87609344, + 87609600 + ], + "dtype": "BF16", + "shape": [ + 128 + ] + }, + "mtp.layers.0.self_attn.k_norm.weight": { + "data_offsets": [ + 19133952, + 19134464 + ], + "dtype": "BF16", + "shape": [ + 256 + ] + }, + "mtp.layers.0.self_attn.k_proj.biases": { + "data_offsets": [ + 752873376, + 752914336 + ], + "dtype": "BF16", + "shape": [ + 512, + 40 + ] + }, + "mtp.layers.0.self_attn.k_proj.scales": { + "data_offsets": [ + 18396672, + 18437632 + ], + "dtype": "BF16", + "shape": [ + 512, + 40 + ] + }, + "mtp.layers.0.self_attn.k_proj.weight": { + "data_offsets": [ + 18437632, + 19092992 + ], + "dtype": "U32", + "shape": [ + 512, + 320 + ] + }, + "mtp.layers.0.self_attn.o_proj.biases": { + "data_offsets": [ + 17413632, + 17905152 + ], + "dtype": "BF16", + "shape": [ + 2560, + 96 + ] + }, + "mtp.layers.0.self_attn.o_proj.scales": { + "data_offsets": [ + 17905152, + 18396672 + ], + "dtype": "BF16", + "shape": [ + 2560, + 96 + ] + }, + "mtp.layers.0.self_attn.o_proj.weight": { + "data_offsets": [ + 1245274016, + 1253138336 + ], + "dtype": "U32", + "shape": [ + 2560, + 768 + ] + }, + "mtp.layers.0.self_attn.q_norm.weight": { + "data_offsets": [ + 17413120, + 17413632 + ], + "dtype": "BF16", + "shape": [ + 256 + ] + }, + "mtp.layers.0.self_attn.q_proj.biases": { + "data_offsets": [ + 701440, + 1684480 + ], + "dtype": "BF16", + "shape": [ + 12288, + 40 + ] + }, + "mtp.layers.0.self_attn.q_proj.scales": { + "data_offsets": [ + 19288064, + 20271104 + ], + "dtype": "BF16", + "shape": [ + 12288, + 40 + ] + }, + "mtp.layers.0.self_attn.q_proj.weight": { + "data_offsets": [ + 1684480, + 17413120 + ], + "dtype": "U32", + "shape": [ + 12288, + 320 + ] + }, + "mtp.layers.0.self_attn.v_proj.biases": { + "data_offsets": [ + 19092992, + 19133952 + ], + "dtype": "BF16", + "shape": [ + 512, + 40 + ] + }, + "mtp.layers.0.self_attn.v_proj.scales": { + "data_offsets": [ + 5120, + 46080 + ], + "dtype": "BF16", + "shape": [ + 512, + 40 + ] + }, + "mtp.layers.0.self_attn.v_proj.weight": { + "data_offsets": [ + 46080, + 701440 + ], + "dtype": "U32", + "shape": [ + 512, + 320 + ] + }, + "mtp.pre_fc_norm_embedding.weight": { + "data_offsets": [ + 0, + 5120 + ], + "dtype": "BF16", + "shape": [ + 2560 + ] + }, + "mtp.pre_fc_norm_hidden.weight": { + "data_offsets": [ + 194781600, + 194802080 + ], + "dtype": "BF16", + "shape": [ + 10240 + ] + } +} diff --git a/docs/issue103-analysis/native-controls.log b/docs/issue103-analysis/native-controls.log new file mode 100644 index 0000000..07ea621 --- /dev/null +++ b/docs/issue103-analysis/native-controls.log @@ -0,0 +1,340 @@ +COMMAND FILTER: mtplx_graph_event_snapshots_preserve_values_streams_and_shared_errors +{"event":"test_supervisor_started","pid":94185,"limit_bytes":4294967296,"start_seconds":30.0,"idle_seconds":60.0,"sample_ms":100} +{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":94875,"resident_bytes":458752,"physical_bytes":180296,"peak_physical_bytes":180296,"disk_read_bytes":0,"disk_write_bytes":0}} + +running 1 test +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_event_snapshots_preserve_values_streams_and_shared_errors ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out; finished in 0.06s + +{"event":"test_resource_summary","elapsed_ms":101,"peak":{"cpu_time_ns":94875,"resident_bytes":458752,"physical_bytes":180296,"peak_physical_bytes":180296,"disk_read_bytes":0,"disk_write_bytes":0},"error":null} + +COMMAND FILTER: mtplx_graph_fence_coherence_preserves_empty_and_partial_word_spans +{"event":"test_supervisor_started","pid":94188,"limit_bytes":4294967296,"start_seconds":30.0,"idle_seconds":60.0,"sample_ms":100} +{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":111000,"resident_bytes":524288,"physical_bytes":180296,"peak_physical_bytes":180296,"disk_read_bytes":0,"disk_write_bytes":0}} + +running 1 test +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fence_coherence_preserves_empty_and_partial_word_spans ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out; finished in 0.07s + +{"event":"test_resource_summary","elapsed_ms":110,"peak":{"cpu_time_ns":111000,"resident_bytes":524288,"physical_bytes":180296,"peak_physical_bytes":180296,"disk_read_bytes":0,"disk_write_bytes":0},"error":null} + +COMMAND FILTER: mtplx_graph_fences_transfer_cpu_gpu_gpu_cpu_in_both_modes +{"event":"test_supervisor_started","pid":94190,"limit_bytes":4294967296,"start_seconds":30.0,"idle_seconds":60.0,"sample_ms":100} +{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":86500,"resident_bytes":458752,"physical_bytes":180296,"peak_physical_bytes":180296,"disk_read_bytes":0,"disk_write_bytes":0}} + +running 1 test +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fences_transfer_cpu_gpu_gpu_cpu_in_both_modes ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out; finished in 0.07s + +{"event":"test_resource_summary","elapsed_ms":110,"peak":{"cpu_time_ns":86500,"resident_bytes":458752,"physical_bytes":180296,"peak_physical_bytes":180296,"disk_read_bytes":0,"disk_write_bytes":0},"error":null} + +COMMAND FILTER: mtplx_kv_cache_graph_matches_reference +{"event":"test_supervisor_started","pid":94111,"limit_bytes":4294967296,"start_seconds":30.0,"idle_seconds":60.0,"sample_ms":100} +{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":117416,"resident_bytes":638976,"physical_bytes":229496,"peak_physical_bytes":229496,"disk_read_bytes":0,"disk_write_bytes":0}} + +running 1 test +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +{"event":"test_resource_sample","elapsed_ms":110,"usage":{"cpu_time_ns":88250500,"resident_bytes":75284480,"physical_bytes":164119368,"peak_physical_bytes":164119368,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":220,"usage":{"cpu_time_ns":194730833,"resident_bytes":82493440,"physical_bytes":164184904,"peak_physical_bytes":164184904,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_0_0: exact +{"event":"test_resource_sample","elapsed_ms":330,"usage":{"cpu_time_ns":307565875,"resident_bytes":85590016,"physical_bytes":165364552,"peak_physical_bytes":165364552,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_0_1: exact +{"event":"test_resource_sample","elapsed_ms":440,"usage":{"cpu_time_ns":415202708,"resident_bytes":85032960,"physical_bytes":171230024,"peak_physical_bytes":171230024,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":550,"usage":{"cpu_time_ns":525225333,"resident_bytes":88260608,"physical_bytes":171246408,"peak_physical_bytes":171246408,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_1_0: exact +{"event":"test_resource_sample","elapsed_ms":660,"usage":{"cpu_time_ns":630136375,"resident_bytes":92045312,"physical_bytes":171803464,"peak_physical_bytes":171803464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":771,"usage":{"cpu_time_ns":740170791,"resident_bytes":93126656,"physical_bytes":171803464,"peak_physical_bytes":171803464,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_1_1: exact +{"event":"test_resource_sample","elapsed_ms":881,"usage":{"cpu_time_ns":849862333,"resident_bytes":92635136,"physical_bytes":180732744,"peak_physical_bytes":180732744,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":987,"usage":{"cpu_time_ns":957918083,"resident_bytes":92635136,"physical_bytes":180732744,"peak_physical_bytes":180732744,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1097,"usage":{"cpu_time_ns":1065293916,"resident_bytes":95961088,"physical_bytes":180912968,"peak_physical_bytes":180912968,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_255_0: exact +{"event":"test_resource_sample","elapsed_ms":1203,"usage":{"cpu_time_ns":1173627166,"resident_bytes":100253696,"physical_bytes":182059848,"peak_physical_bytes":182059848,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1308,"usage":{"cpu_time_ns":1278186125,"resident_bytes":100253696,"physical_bytes":182059848,"peak_physical_bytes":182059848,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1418,"usage":{"cpu_time_ns":1388219666,"resident_bytes":99172352,"physical_bytes":180978504,"peak_physical_bytes":182059848,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_255_1: exact +{"event":"test_resource_sample","elapsed_ms":1529,"usage":{"cpu_time_ns":1498530208,"resident_bytes":99434496,"physical_bytes":181240648,"peak_physical_bytes":182059848,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1639,"usage":{"cpu_time_ns":1604771916,"resident_bytes":100761600,"physical_bytes":182551368,"peak_physical_bytes":183108424,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1749,"usage":{"cpu_time_ns":1715970125,"resident_bytes":100761600,"physical_bytes":182551368,"peak_physical_bytes":183108424,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1859,"usage":{"cpu_time_ns":1820459958,"resident_bytes":100761600,"physical_bytes":182551368,"peak_physical_bytes":183108424,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_256_0: exact +{"event":"test_resource_sample","elapsed_ms":1969,"usage":{"cpu_time_ns":1934084250,"resident_bytes":101335040,"physical_bytes":183108424,"peak_physical_bytes":183632712,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2070,"usage":{"cpu_time_ns":2035977750,"resident_bytes":101335040,"physical_bytes":183108424,"peak_physical_bytes":183632712,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2180,"usage":{"cpu_time_ns":2143599916,"resident_bytes":100777984,"physical_bytes":182551368,"peak_physical_bytes":183632712,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_256_1: exact +{"event":"test_resource_sample","elapsed_ms":2290,"usage":{"cpu_time_ns":2251819083,"resident_bytes":100253696,"physical_bytes":191595336,"peak_physical_bytes":191595336,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2400,"usage":{"cpu_time_ns":2363792041,"resident_bytes":100253696,"physical_bytes":191595336,"peak_physical_bytes":191595336,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2510,"usage":{"cpu_time_ns":2473765875,"resident_bytes":103448576,"physical_bytes":191628128,"peak_physical_bytes":191628128,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_257_0: exact +{"event":"test_resource_sample","elapsed_ms":2621,"usage":{"cpu_time_ns":2577761833,"resident_bytes":107741184,"physical_bytes":192742240,"peak_physical_bytes":192742240,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2731,"usage":{"cpu_time_ns":2687792500,"resident_bytes":107741184,"physical_bytes":192742240,"peak_physical_bytes":192742240,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2841,"usage":{"cpu_time_ns":2802667250,"resident_bytes":107741184,"physical_bytes":192742240,"peak_physical_bytes":192742240,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2941,"usage":{"cpu_time_ns":2897877291,"resident_bytes":106627072,"physical_bytes":191628128,"peak_physical_bytes":192742240,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_257_1: exact +{"event":"test_resource_sample","elapsed_ms":3051,"usage":{"cpu_time_ns":3012734250,"resident_bytes":111247360,"physical_bytes":200967008,"peak_physical_bytes":200967008,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3161,"usage":{"cpu_time_ns":3117988291,"resident_bytes":119685120,"physical_bytes":320160656,"peak_physical_bytes":320160656,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3271,"usage":{"cpu_time_ns":3230044208,"resident_bytes":128073728,"physical_bytes":320160656,"peak_physical_bytes":320160656,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3381,"usage":{"cpu_time_ns":3338052208,"resident_bytes":132268032,"physical_bytes":320160656,"peak_physical_bytes":320160656,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3491,"usage":{"cpu_time_ns":3448081333,"resident_bytes":144916480,"physical_bytes":324404136,"peak_physical_bytes":324404136,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3602,"usage":{"cpu_time_ns":3558101375,"resident_bytes":154370048,"physical_bytes":329155520,"peak_physical_bytes":329155520,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3712,"usage":{"cpu_time_ns":3670183291,"resident_bytes":163299328,"physical_bytes":329155520,"peak_physical_bytes":329155520,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3820,"usage":{"cpu_time_ns":3778458958,"resident_bytes":167510016,"physical_bytes":329155520,"peak_physical_bytes":329155520,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3930,"usage":{"cpu_time_ns":3888190250,"resident_bytes":176947200,"physical_bytes":329155520,"peak_physical_bytes":329155520,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4040,"usage":{"cpu_time_ns":3998219333,"resident_bytes":186433536,"physical_bytes":333923288,"peak_physical_bytes":333923288,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4150,"usage":{"cpu_time_ns":4108298833,"resident_bytes":191168512,"physical_bytes":216204248,"peak_physical_bytes":333923288,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4260,"usage":{"cpu_time_ns":4218533083,"resident_bytes":201670656,"physical_bytes":221463512,"peak_physical_bytes":333923288,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4371,"usage":{"cpu_time_ns":4328360166,"resident_bytes":212156416,"physical_bytes":221463512,"peak_physical_bytes":333923288,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4481,"usage":{"cpu_time_ns":4438522500,"resident_bytes":217399296,"physical_bytes":221463512,"peak_physical_bytes":333923288,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_2048_0: exact +{"event":"test_resource_sample","elapsed_ms":4591,"usage":{"cpu_time_ns":4539142916,"resident_bytes":215302144,"physical_bytes":219366360,"peak_physical_bytes":333923288,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4701,"usage":{"cpu_time_ns":4647523833,"resident_bytes":217399296,"physical_bytes":339182552,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4811,"usage":{"cpu_time_ns":4757552666,"resident_bytes":217399296,"physical_bytes":339182552,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4921,"usage":{"cpu_time_ns":4867581250,"resident_bytes":217399296,"physical_bytes":339182552,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5031,"usage":{"cpu_time_ns":4986780458,"resident_bytes":217399296,"physical_bytes":339182552,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5141,"usage":{"cpu_time_ns":5087669208,"resident_bytes":217399296,"physical_bytes":339182552,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5251,"usage":{"cpu_time_ns":5197718458,"resident_bytes":217399296,"physical_bytes":339182552,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5362,"usage":{"cpu_time_ns":5307811625,"resident_bytes":217382912,"physical_bytes":339166168,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5470,"usage":{"cpu_time_ns":5425356416,"resident_bytes":217382912,"physical_bytes":339166168,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5580,"usage":{"cpu_time_ns":5527873750,"resident_bytes":217382912,"physical_bytes":339166168,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5690,"usage":{"cpu_time_ns":5637914958,"resident_bytes":217382912,"physical_bytes":339166168,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5800,"usage":{"cpu_time_ns":5748029583,"resident_bytes":217382912,"physical_bytes":339166168,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5911,"usage":{"cpu_time_ns":5858060333,"resident_bytes":217382912,"physical_bytes":221447128,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6021,"usage":{"cpu_time_ns":5968091500,"resident_bytes":217382912,"physical_bytes":221447128,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical kv_cache_graph_2048_1: exact +test engine::metal::qwen_mtplx::cache::mtplx_kv_cache_graph_matches_reference ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out; finished in 6.10s + +{"event":"test_resource_summary","elapsed_ms":6131,"peak":{"cpu_time_ns":5968091500,"resident_bytes":217399296,"physical_bytes":339182552,"peak_physical_bytes":339739608,"disk_read_bytes":53248,"disk_write_bytes":0},"error":null} + +COMMAND FILTER: mtplx_qsa_cache_graph_matches_reference +{"event":"test_supervisor_started","pid":94135,"limit_bytes":4294967296,"start_seconds":30.0,"idle_seconds":60.0,"sample_ms":100} +{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":81958,"resident_bytes":458752,"physical_bytes":180296,"peak_physical_bytes":180296,"disk_read_bytes":0,"disk_write_bytes":0}} + +running 1 test +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +{"event":"test_resource_sample","elapsed_ms":110,"usage":{"cpu_time_ns":85030000,"resident_bytes":76906496,"physical_bytes":189711152,"peak_physical_bytes":189711152,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":220,"usage":{"cpu_time_ns":191719416,"resident_bytes":83836928,"physical_bytes":189874992,"peak_physical_bytes":189874992,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":330,"usage":{"cpu_time_ns":301747375,"resident_bytes":90013696,"physical_bytes":189907760,"peak_physical_bytes":189907760,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":440,"usage":{"cpu_time_ns":411788041,"resident_bytes":96550912,"physical_bytes":189940528,"peak_physical_bytes":189940528,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":550,"usage":{"cpu_time_ns":521820500,"resident_bytes":102727680,"physical_bytes":189956936,"peak_physical_bytes":189956936,"disk_read_bytes":0,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_0_4_128_bfloat16_0: exact +{"event":"test_resource_sample","elapsed_ms":660,"usage":{"cpu_time_ns":618800333,"resident_bytes":106741760,"physical_bytes":190890824,"peak_physical_bytes":190890824,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":771,"usage":{"cpu_time_ns":738428125,"resident_bytes":108183552,"physical_bytes":190235464,"peak_physical_bytes":190890824,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":881,"usage":{"cpu_time_ns":848480333,"resident_bytes":108855296,"physical_bytes":190235464,"peak_physical_bytes":190890824,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":991,"usage":{"cpu_time_ns":955045000,"resident_bytes":109789184,"physical_bytes":190235464,"peak_physical_bytes":190890824,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1101,"usage":{"cpu_time_ns":1061199583,"resident_bytes":110182400,"physical_bytes":190235464,"peak_physical_bytes":190890824,"disk_read_bytes":0,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_0_4_128_bfloat16_1: exact +{"event":"test_resource_sample","elapsed_ms":1211,"usage":{"cpu_time_ns":1157441958,"resident_bytes":110329856,"physical_bytes":208503624,"peak_physical_bytes":208503624,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1321,"usage":{"cpu_time_ns":1258831250,"resident_bytes":111378432,"physical_bytes":208503624,"peak_physical_bytes":208503624,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1431,"usage":{"cpu_time_ns":1377514291,"resident_bytes":113770496,"physical_bytes":208503624,"peak_physical_bytes":208503624,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1542,"usage":{"cpu_time_ns":1487564208,"resident_bytes":118652928,"physical_bytes":208520008,"peak_physical_bytes":208520008,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1648,"usage":{"cpu_time_ns":1588599041,"resident_bytes":124239872,"physical_bytes":208520008,"peak_physical_bytes":208520008,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1759,"usage":{"cpu_time_ns":1704448791,"resident_bytes":127975424,"physical_bytes":208520008,"peak_physical_bytes":208520008,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_1_4_128_bfloat16_0: exact +{"event":"test_resource_sample","elapsed_ms":1869,"usage":{"cpu_time_ns":1797765750,"resident_bytes":128172032,"physical_bytes":208864072,"peak_physical_bytes":209519432,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1979,"usage":{"cpu_time_ns":1910374458,"resident_bytes":128172032,"physical_bytes":208864072,"peak_physical_bytes":209519432,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2089,"usage":{"cpu_time_ns":2016464625,"resident_bytes":128172032,"physical_bytes":208864072,"peak_physical_bytes":209519432,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2199,"usage":{"cpu_time_ns":2129422708,"resident_bytes":129269760,"physical_bytes":208880456,"peak_physical_bytes":209519432,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2309,"usage":{"cpu_time_ns":2238014833,"resident_bytes":129269760,"physical_bytes":208880456,"peak_physical_bytes":209519432,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_1_4_128_bfloat16_1: exact +{"event":"test_resource_sample","elapsed_ms":2419,"usage":{"cpu_time_ns":2349364500,"resident_bytes":129384448,"physical_bytes":209453896,"peak_physical_bytes":210027336,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2529,"usage":{"cpu_time_ns":2442658416,"resident_bytes":129712128,"physical_bytes":238502728,"peak_physical_bytes":238502728,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2639,"usage":{"cpu_time_ns":2548477916,"resident_bytes":130105344,"physical_bytes":238519136,"peak_physical_bytes":238519136,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2750,"usage":{"cpu_time_ns":2653148708,"resident_bytes":131579904,"physical_bytes":238519136,"peak_physical_bytes":238519136,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2860,"usage":{"cpu_time_ns":2770746291,"resident_bytes":132694016,"physical_bytes":238519136,"peak_physical_bytes":238519136,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2970,"usage":{"cpu_time_ns":2882695083,"resident_bytes":133775360,"physical_bytes":238519136,"peak_physical_bytes":238519136,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3080,"usage":{"cpu_time_ns":2992729291,"resident_bytes":139542528,"physical_bytes":238519136,"peak_physical_bytes":238519136,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3190,"usage":{"cpu_time_ns":3102797583,"resident_bytes":144621568,"physical_bytes":238224224,"peak_physical_bytes":238519136,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3300,"usage":{"cpu_time_ns":3203441541,"resident_bytes":149471232,"physical_bytes":238224224,"peak_physical_bytes":238519136,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3410,"usage":{"cpu_time_ns":3313476583,"resident_bytes":154320896,"physical_bytes":238224224,"peak_physical_bytes":238519136,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_255_4_128_bfloat16_0: exact +{"event":"test_resource_sample","elapsed_ms":3520,"usage":{"cpu_time_ns":3426261750,"resident_bytes":155992064,"physical_bytes":238666592,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3630,"usage":{"cpu_time_ns":3524886000,"resident_bytes":156008448,"physical_bytes":239223648,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3740,"usage":{"cpu_time_ns":3636675083,"resident_bytes":156008448,"physical_bytes":239223648,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3851,"usage":{"cpu_time_ns":3744869541,"resident_bytes":156172288,"physical_bytes":239223648,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3961,"usage":{"cpu_time_ns":3853888416,"resident_bytes":156188672,"physical_bytes":239223648,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4071,"usage":{"cpu_time_ns":3963919041,"resident_bytes":156188672,"physical_bytes":239223648,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4181,"usage":{"cpu_time_ns":4076741083,"resident_bytes":156745728,"physical_bytes":239223648,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4282,"usage":{"cpu_time_ns":4174021083,"resident_bytes":156811264,"physical_bytes":239223648,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4392,"usage":{"cpu_time_ns":4285809416,"resident_bytes":156811264,"physical_bytes":239223648,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4498,"usage":{"cpu_time_ns":4384093958,"resident_bytes":156811264,"physical_bytes":239223648,"peak_physical_bytes":240026464,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_255_4_128_bfloat16_1: exact +{"event":"test_resource_sample","elapsed_ms":4608,"usage":{"cpu_time_ns":4492246375,"resident_bytes":157368320,"physical_bytes":240436064,"peak_physical_bytes":240616288,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4718,"usage":{"cpu_time_ns":4595147916,"resident_bytes":157384704,"physical_bytes":241501024,"peak_physical_bytes":241501024,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4828,"usage":{"cpu_time_ns":4704549666,"resident_bytes":157384704,"physical_bytes":241501024,"peak_physical_bytes":241501024,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4938,"usage":{"cpu_time_ns":4807775583,"resident_bytes":157384704,"physical_bytes":241501024,"peak_physical_bytes":241501024,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5048,"usage":{"cpu_time_ns":4919855791,"resident_bytes":157712384,"physical_bytes":241501024,"peak_physical_bytes":241501024,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5158,"usage":{"cpu_time_ns":5035460583,"resident_bytes":157876224,"physical_bytes":241501024,"peak_physical_bytes":241501024,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5269,"usage":{"cpu_time_ns":5141733250,"resident_bytes":157876224,"physical_bytes":241156960,"peak_physical_bytes":241501024,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5379,"usage":{"cpu_time_ns":5249028625,"resident_bytes":158154752,"physical_bytes":241156960,"peak_physical_bytes":241501024,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5489,"usage":{"cpu_time_ns":5358076125,"resident_bytes":158941184,"physical_bytes":241156960,"peak_physical_bytes":241501024,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5599,"usage":{"cpu_time_ns":5468123833,"resident_bytes":158941184,"physical_bytes":241156960,"peak_physical_bytes":241501024,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_256_4_128_bfloat16_0: exact +{"event":"test_resource_sample","elapsed_ms":5709,"usage":{"cpu_time_ns":5569979166,"resident_bytes":158973952,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5819,"usage":{"cpu_time_ns":5680006583,"resident_bytes":158973952,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5929,"usage":{"cpu_time_ns":5790035083,"resident_bytes":158973952,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6032,"usage":{"cpu_time_ns":5893730000,"resident_bytes":158973952,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6142,"usage":{"cpu_time_ns":6003691750,"resident_bytes":158973952,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6248,"usage":{"cpu_time_ns":6110202791,"resident_bytes":158990336,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6359,"usage":{"cpu_time_ns":6220211000,"resident_bytes":158990336,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6469,"usage":{"cpu_time_ns":6330218000,"resident_bytes":158990336,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6579,"usage":{"cpu_time_ns":6430296833,"resident_bytes":158990336,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6689,"usage":{"cpu_time_ns":6549628583,"resident_bytes":158990336,"physical_bytes":241451872,"peak_physical_bytes":253084512,"disk_read_bytes":53248,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_256_4_128_bfloat16_1: exact +{"event":"test_resource_sample","elapsed_ms":6799,"usage":{"cpu_time_ns":6638902166,"resident_bytes":158760960,"physical_bytes":270107488,"peak_physical_bytes":270107488,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6900,"usage":{"cpu_time_ns":6740583041,"resident_bytes":158908416,"physical_bytes":270123872,"peak_physical_bytes":270123872,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7010,"usage":{"cpu_time_ns":6850611750,"resident_bytes":159285248,"physical_bytes":270140280,"peak_physical_bytes":270140280,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7120,"usage":{"cpu_time_ns":6960917958,"resident_bytes":159596544,"physical_bytes":270140280,"peak_physical_bytes":270140280,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7230,"usage":{"cpu_time_ns":7070782125,"resident_bytes":160940032,"physical_bytes":269632376,"peak_physical_bytes":270140280,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7332,"usage":{"cpu_time_ns":7172244125,"resident_bytes":166641664,"physical_bytes":269632376,"peak_physical_bytes":270140280,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7442,"usage":{"cpu_time_ns":7280835500,"resident_bytes":171638784,"physical_bytes":269632376,"peak_physical_bytes":270140280,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7552,"usage":{"cpu_time_ns":7390864208,"resident_bytes":178274304,"physical_bytes":269632376,"peak_physical_bytes":270140280,"disk_read_bytes":204800,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_257_1_17_float16_0: exact +{"event":"test_resource_sample","elapsed_ms":7662,"usage":{"cpu_time_ns":7502277125,"resident_bytes":182648832,"physical_bytes":270828408,"peak_physical_bytes":270828408,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7772,"usage":{"cpu_time_ns":7578124833,"resident_bytes":181534720,"physical_bytes":270238584,"peak_physical_bytes":271418232,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7882,"usage":{"cpu_time_ns":7688153000,"resident_bytes":181534720,"physical_bytes":270238584,"peak_physical_bytes":271418232,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7992,"usage":{"cpu_time_ns":7805849375,"resident_bytes":181534720,"physical_bytes":270238584,"peak_physical_bytes":271418232,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8098,"usage":{"cpu_time_ns":7913138666,"resident_bytes":181534720,"physical_bytes":270238584,"peak_physical_bytes":271418232,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8209,"usage":{"cpu_time_ns":8022787625,"resident_bytes":181518336,"physical_bytes":270222200,"peak_physical_bytes":271418232,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8319,"usage":{"cpu_time_ns":8133149291,"resident_bytes":181518336,"physical_bytes":270222200,"peak_physical_bytes":271418232,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8429,"usage":{"cpu_time_ns":8238365250,"resident_bytes":181518336,"physical_bytes":270222200,"peak_physical_bytes":271418232,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8539,"usage":{"cpu_time_ns":8348408166,"resident_bytes":181518336,"physical_bytes":270222200,"peak_physical_bytes":271418232,"disk_read_bytes":204800,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_257_1_17_float16_1: exact +{"event":"test_resource_sample","elapsed_ms":8649,"usage":{"cpu_time_ns":8420579041,"resident_bytes":181518336,"physical_bytes":273073016,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8759,"usage":{"cpu_time_ns":8530604041,"resident_bytes":181534720,"physical_bytes":273073016,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8869,"usage":{"cpu_time_ns":8645606375,"resident_bytes":181534720,"physical_bytes":273073016,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8979,"usage":{"cpu_time_ns":8750662250,"resident_bytes":181534720,"physical_bytes":273073016,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9089,"usage":{"cpu_time_ns":8860687541,"resident_bytes":181682176,"physical_bytes":273073016,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9199,"usage":{"cpu_time_ns":8974670125,"resident_bytes":181829632,"physical_bytes":272483192,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9310,"usage":{"cpu_time_ns":9085671416,"resident_bytes":182272000,"physical_bytes":272483192,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9420,"usage":{"cpu_time_ns":9190859416,"resident_bytes":184188928,"physical_bytes":272483192,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9530,"usage":{"cpu_time_ns":9300904833,"resident_bytes":184188928,"physical_bytes":272483192,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9640,"usage":{"cpu_time_ns":9414251000,"resident_bytes":184188928,"physical_bytes":272483192,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_257_4_128_bfloat16_0: exact +{"event":"test_resource_sample","elapsed_ms":9750,"usage":{"cpu_time_ns":9525822500,"resident_bytes":184369152,"physical_bytes":272663416,"peak_physical_bytes":273073016,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9860,"usage":{"cpu_time_ns":9616863375,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9961,"usage":{"cpu_time_ns":9716881416,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10071,"usage":{"cpu_time_ns":9831244000,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10181,"usage":{"cpu_time_ns":9937503000,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10291,"usage":{"cpu_time_ns":10048181250,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10401,"usage":{"cpu_time_ns":10162843375,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10512,"usage":{"cpu_time_ns":10271299583,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10622,"usage":{"cpu_time_ns":10378854833,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10732,"usage":{"cpu_time_ns":10487534000,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10842,"usage":{"cpu_time_ns":10597252166,"resident_bytes":184188928,"physical_bytes":273073016,"peak_physical_bytes":274465656,"disk_read_bytes":204800,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_257_4_128_bfloat16_1: exact +{"event":"test_resource_sample","elapsed_ms":10952,"usage":{"cpu_time_ns":10705043708,"resident_bytes":184303616,"physical_bytes":273957752,"peak_physical_bytes":274547576,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11062,"usage":{"cpu_time_ns":10807611166,"resident_bytes":184418304,"physical_bytes":273957752,"peak_physical_bytes":274547576,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11172,"usage":{"cpu_time_ns":10917708916,"resident_bytes":184696832,"physical_bytes":273613688,"peak_physical_bytes":274547576,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11282,"usage":{"cpu_time_ns":11025373791,"resident_bytes":184713216,"physical_bytes":273613688,"peak_physical_bytes":274547576,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11392,"usage":{"cpu_time_ns":11133196250,"resident_bytes":184713216,"physical_bytes":273613688,"peak_physical_bytes":274547576,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11502,"usage":{"cpu_time_ns":11244008041,"resident_bytes":184745984,"physical_bytes":273613688,"peak_physical_bytes":274547576,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11613,"usage":{"cpu_time_ns":11359384666,"resident_bytes":184926208,"physical_bytes":273613688,"peak_physical_bytes":274547576,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11723,"usage":{"cpu_time_ns":11462623458,"resident_bytes":184926208,"physical_bytes":273613688,"peak_physical_bytes":274547576,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11833,"usage":{"cpu_time_ns":11576774083,"resident_bytes":184926208,"physical_bytes":273613688,"peak_physical_bytes":274547576,"disk_read_bytes":368640,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_257_7_32_float32_0: exact +{"event":"test_resource_sample","elapsed_ms":11943,"usage":{"cpu_time_ns":11677132625,"resident_bytes":184942592,"physical_bytes":273711992,"peak_physical_bytes":274908024,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":12053,"usage":{"cpu_time_ns":11789965500,"resident_bytes":184942592,"physical_bytes":273711992,"peak_physical_bytes":274908024,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":12163,"usage":{"cpu_time_ns":11899624125,"resident_bytes":184942592,"physical_bytes":273711992,"peak_physical_bytes":274908024,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":12273,"usage":{"cpu_time_ns":12007109208,"resident_bytes":184942592,"physical_bytes":273711992,"peak_physical_bytes":274908024,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":12383,"usage":{"cpu_time_ns":12117324000,"resident_bytes":184942592,"physical_bytes":273711992,"peak_physical_bytes":274908024,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":12493,"usage":{"cpu_time_ns":12222104125,"resident_bytes":184942592,"physical_bytes":273711992,"peak_physical_bytes":274908024,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":12598,"usage":{"cpu_time_ns":12335053708,"resident_bytes":185024512,"physical_bytes":273711992,"peak_physical_bytes":274908024,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":12709,"usage":{"cpu_time_ns":12441000250,"resident_bytes":185024512,"physical_bytes":273711992,"peak_physical_bytes":274908024,"disk_read_bytes":368640,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_257_7_32_float32_1: exact +{"event":"test_resource_sample","elapsed_ms":12819,"usage":{"cpu_time_ns":12553580458,"resident_bytes":186761216,"physical_bytes":275448696,"peak_physical_bytes":275448696,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":12929,"usage":{"cpu_time_ns":12639358000,"resident_bytes":190332928,"physical_bytes":410862480,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":13039,"usage":{"cpu_time_ns":12749381333,"resident_bytes":195723264,"physical_bytes":410862480,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":13149,"usage":{"cpu_time_ns":12868286291,"resident_bytes":203079680,"physical_bytes":410862480,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":13259,"usage":{"cpu_time_ns":12969484083,"resident_bytes":207962112,"physical_bytes":410518416,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":13369,"usage":{"cpu_time_ns":13079515333,"resident_bytes":213204992,"physical_bytes":410518416,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":13479,"usage":{"cpu_time_ns":13189545708,"resident_bytes":217432064,"physical_bytes":410518416,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":13589,"usage":{"cpu_time_ns":13308298458,"resident_bytes":222674944,"physical_bytes":410534824,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":13699,"usage":{"cpu_time_ns":13409774208,"resident_bytes":226902016,"physical_bytes":410534824,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":13810,"usage":{"cpu_time_ns":13519643458,"resident_bytes":232144896,"physical_bytes":410534824,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":13920,"usage":{"cpu_time_ns":13633731875,"resident_bytes":238993408,"physical_bytes":410534824,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":14030,"usage":{"cpu_time_ns":13743059458,"resident_bytes":241614848,"physical_bytes":410534824,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":14140,"usage":{"cpu_time_ns":13858286625,"resident_bytes":248463360,"physical_bytes":293094312,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":14250,"usage":{"cpu_time_ns":13959803458,"resident_bytes":253198336,"physical_bytes":292848552,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":14360,"usage":{"cpu_time_ns":14078302666,"resident_bytes":259538944,"physical_bytes":292864960,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":14470,"usage":{"cpu_time_ns":14179872833,"resident_bytes":263766016,"physical_bytes":292864960,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":14580,"usage":{"cpu_time_ns":14298278583,"resident_bytes":270106624,"physical_bytes":292864960,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":14690,"usage":{"cpu_time_ns":14408264416,"resident_bytes":278560768,"physical_bytes":296043456,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":14798,"usage":{"cpu_time_ns":14516257791,"resident_bytes":281722880,"physical_bytes":296043456,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":14909,"usage":{"cpu_time_ns":14626258250,"resident_bytes":290832384,"physical_bytes":296059864,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":15019,"usage":{"cpu_time_ns":14736240083,"resident_bytes":293994496,"physical_bytes":296059864,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":15129,"usage":{"cpu_time_ns":14840050625,"resident_bytes":300318720,"physical_bytes":296059864,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":15239,"usage":{"cpu_time_ns":14950110208,"resident_bytes":306642944,"physical_bytes":296059864,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":15349,"usage":{"cpu_time_ns":15060139375,"resident_bytes":312328192,"physical_bytes":296485848,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":15459,"usage":{"cpu_time_ns":15170168458,"resident_bytes":317587456,"physical_bytes":296485848,"peak_physical_bytes":410862480,"disk_read_bytes":368640,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_1025_4_128_bfloat16_0: exact +{"event":"test_resource_sample","elapsed_ms":15569,"usage":{"cpu_time_ns":15283569083,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":15679,"usage":{"cpu_time_ns":15375477375,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":15789,"usage":{"cpu_time_ns":15490568375,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":15899,"usage":{"cpu_time_ns":15599608208,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":16010,"usage":{"cpu_time_ns":15705440125,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":16120,"usage":{"cpu_time_ns":15820666166,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":16230,"usage":{"cpu_time_ns":15924542416,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":16340,"usage":{"cpu_time_ns":16039036666,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":16450,"usage":{"cpu_time_ns":16150731625,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":16560,"usage":{"cpu_time_ns":16254657375,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":16670,"usage":{"cpu_time_ns":16364796083,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":16780,"usage":{"cpu_time_ns":16474844166,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":16890,"usage":{"cpu_time_ns":16584879125,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17001,"usage":{"cpu_time_ns":16694921250,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17111,"usage":{"cpu_time_ns":16804969375,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17221,"usage":{"cpu_time_ns":16915015250,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17331,"usage":{"cpu_time_ns":17025061791,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17441,"usage":{"cpu_time_ns":17135103500,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17548,"usage":{"cpu_time_ns":17245129916,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17659,"usage":{"cpu_time_ns":17355164541,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17765,"usage":{"cpu_time_ns":17464839916,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17875,"usage":{"cpu_time_ns":17574848833,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":17985,"usage":{"cpu_time_ns":17675281000,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":18095,"usage":{"cpu_time_ns":17791999541,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":18206,"usage":{"cpu_time_ns":17902253250,"resident_bytes":317603840,"physical_bytes":296502232,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0}} +MTPLX canonical qsa_cache_graph_1025_4_128_bfloat16_1: exact +test engine::metal::qwen_mtplx::cache::mtplx_qsa_cache_graph_matches_reference ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out; finished in 18.21s + +{"event":"test_resource_summary","elapsed_ms":18316,"peak":{"cpu_time_ns":17902253250,"resident_bytes":317603840,"physical_bytes":414729176,"peak_physical_bytes":418186200,"disk_read_bytes":368640,"disk_write_bytes":0},"error":null} + +COMMAND FILTER: qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state +{"event":"test_supervisor_started","pid":94192,"limit_bytes":4294967296,"start_seconds":30.0,"idle_seconds":60.0,"sample_ms":100} +{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":105916,"resident_bytes":458752,"physical_bytes":180296,"peak_physical_bytes":180296,"disk_read_bytes":0,"disk_write_bytes":0}} + +running 1 test +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +test engine::metal::qwen_mtplx::session_checkpoint::qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state ... Qwen checkpoint: exact payload roundtrip, interrupted save, incompatible/corrupt restore passed +ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out; finished in 0.06s + +{"event":"test_resource_summary","elapsed_ms":110,"peak":{"cpu_time_ns":105916,"resident_bytes":458752,"physical_bytes":180296,"peak_physical_bytes":180296,"disk_read_bytes":0,"disk_write_bytes":0},"error":null} + diff --git a/docs/issue103-analysis/native-eval-controls.log b/docs/issue103-analysis/native-eval-controls.log new file mode 100644 index 0000000..1f5ad5d --- /dev/null +++ b/docs/issue103-analysis/native-eval-controls.log @@ -0,0 +1,33 @@ +{"event":"test_supervisor_started","pid":94012,"limit_bytes":4294967296,"start_seconds":30.0,"idle_seconds":60.0,"sample_ms":100} +{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":146916,"resident_bytes":622592,"physical_bytes":213088,"peak_physical_bytes":213088,"disk_read_bytes":0,"disk_write_bytes":0}} + +running 4 tests +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx + +thread 'engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_drains_failure_preserves_cause_and_retries_only_failed_nodes' (2307597) panicked at src/engine/metal/qwen_mtplx/eval.rs:242:37: +original primitive panic +note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_drains_failure_preserves_cause_and_retries_only_failed_nodes ... ok +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +{"event":"test_resource_sample","elapsed_ms":110,"usage":{"cpu_time_ns":88764166,"resident_bytes":73383936,"physical_bytes":137184000,"peak_physical_bytes":167707392,"disk_read_bytes":589824,"disk_write_bytes":0}} +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_releases_intermediates_and_preserves_descriptor_donation ... ok +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_retains_tracer_topology_without_input_donation ... ok +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_runs_cpu_gpu_siblings_and_async_continuations ... ok + +test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 513 filtered out; finished in 0.17s + +{"event":"test_resource_summary","elapsed_ms":220,"peak":{"cpu_time_ns":88764166,"resident_bytes":73383936,"physical_bytes":137184000,"peak_physical_bytes":167707392,"disk_read_bytes":589824,"disk_write_bytes":0},"error":null} diff --git a/docs/issue103-analysis/native-final.log b/docs/issue103-analysis/native-final.log new file mode 100644 index 0000000..b1e5747 --- /dev/null +++ b/docs/issue103-analysis/native-final.log @@ -0,0 +1,204 @@ +{"event":"test_supervisor_started","pid":93824,"limit_bytes":105226698752,"start_seconds":60.0,"idle_seconds":120.0,"sample_ms":100} +{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":107,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":214,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":315,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":424,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":533,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":641,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":745,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":847,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":958,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1068,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1169,"usage":{"cpu_time_ns":29166,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} + +running 6 tests +#103 loading unchanged Bare-Speed model +{"event":"test_resource_sample","elapsed_ms":1279,"usage":{"cpu_time_ns":33663083,"resident_bytes":17711104,"physical_bytes":4145752,"peak_physical_bytes":4145752,"disk_read_bytes":0,"disk_write_bytes":0}} +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +{"event":"test_resource_sample","elapsed_ms":1381,"usage":{"cpu_time_ns":133716166,"resident_bytes":277299200,"physical_bytes":235865096,"peak_physical_bytes":235832328,"disk_read_bytes":234725376,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1488,"usage":{"cpu_time_ns":291272000,"resident_bytes":1685209088,"physical_bytes":1644480504,"peak_physical_bytes":1644480504,"disk_read_bytes":1683353600,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1598,"usage":{"cpu_time_ns":416839791,"resident_bytes":3168059392,"physical_bytes":3128052760,"peak_physical_bytes":3128019992,"disk_read_bytes":3159019520,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1707,"usage":{"cpu_time_ns":555908916,"resident_bytes":4638261248,"physical_bytes":4598960160,"peak_physical_bytes":4598911008,"disk_read_bytes":4611289088,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1818,"usage":{"cpu_time_ns":680447958,"resident_bytes":6099501056,"physical_bytes":6060921920,"peak_physical_bytes":6060872768,"disk_read_bytes":6077902848,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1928,"usage":{"cpu_time_ns":802677541,"resident_bytes":7560085504,"physical_bytes":7522211912,"peak_physical_bytes":7522211912,"disk_read_bytes":7538245632,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2038,"usage":{"cpu_time_ns":933539333,"resident_bytes":9031892992,"physical_bytes":8994741352,"peak_physical_bytes":8994741352,"disk_read_bytes":8993546240,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2148,"usage":{"cpu_time_ns":1070241916,"resident_bytes":10499899392,"physical_bytes":10463518856,"peak_physical_bytes":10463518856,"disk_read_bytes":10491719680,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2258,"usage":{"cpu_time_ns":1208583833,"resident_bytes":11977539584,"physical_bytes":11941881000,"peak_physical_bytes":11941848232,"disk_read_bytes":11935870976,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2368,"usage":{"cpu_time_ns":1349718541,"resident_bytes":13461635072,"physical_bytes":13426698440,"peak_physical_bytes":13426665672,"disk_read_bytes":13471784960,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2478,"usage":{"cpu_time_ns":1483531666,"resident_bytes":14934671360,"physical_bytes":14900456680,"peak_physical_bytes":14900423912,"disk_read_bytes":14923816960,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2587,"usage":{"cpu_time_ns":1613819625,"resident_bytes":16382361600,"physical_bytes":16348885280,"peak_physical_bytes":16348852512,"disk_read_bytes":16368848896,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2697,"usage":{"cpu_time_ns":1744339958,"resident_bytes":17863016448,"physical_bytes":17830294800,"peak_physical_bytes":17830262032,"disk_read_bytes":17864884224,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2807,"usage":{"cpu_time_ns":1879441500,"resident_bytes":19351535616,"physical_bytes":19319535944,"peak_physical_bytes":19319503176,"disk_read_bytes":19357925376,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2909,"usage":{"cpu_time_ns":1994429250,"resident_bytes":20710408192,"physical_bytes":20679064840,"peak_physical_bytes":20679064840,"disk_read_bytes":20719484928,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3019,"usage":{"cpu_time_ns":2121548625,"resident_bytes":22194913280,"physical_bytes":22164291880,"peak_physical_bytes":22164259112,"disk_read_bytes":22181412864,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3129,"usage":{"cpu_time_ns":2257587500,"resident_bytes":23673552896,"physical_bytes":23643653448,"peak_physical_bytes":23643604296,"disk_read_bytes":23673425920,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3239,"usage":{"cpu_time_ns":2398571875,"resident_bytes":25149636608,"physical_bytes":25120459112,"peak_physical_bytes":25120409960,"disk_read_bytes":25124466688,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3347,"usage":{"cpu_time_ns":2530489958,"resident_bytes":26585907200,"physical_bytes":26557500808,"peak_physical_bytes":26557468040,"disk_read_bytes":26570489856,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3457,"usage":{"cpu_time_ns":2658953375,"resident_bytes":28059664384,"physical_bytes":28031963536,"peak_physical_bytes":28031947152,"disk_read_bytes":28032028672,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3567,"usage":{"cpu_time_ns":2800414791,"resident_bytes":29522657280,"physical_bytes":29495694792,"peak_physical_bytes":29495694792,"disk_read_bytes":29501595648,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3677,"usage":{"cpu_time_ns":2931685708,"resident_bytes":30989156352,"physical_bytes":30962915816,"peak_physical_bytes":30962866664,"disk_read_bytes":30974275584,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3787,"usage":{"cpu_time_ns":3070414708,"resident_bytes":32464125952,"physical_bytes":32438607368,"peak_physical_bytes":32438590984,"disk_read_bytes":32440795136,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":3897,"usage":{"cpu_time_ns":3193667916,"resident_bytes":33927659520,"physical_bytes":33902912040,"peak_physical_bytes":33902879272,"disk_read_bytes":33919111168,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4003,"usage":{"cpu_time_ns":3339872583,"resident_bytes":35339091968,"physical_bytes":35315017216,"peak_physical_bytes":35315000832,"disk_read_bytes":35291586560,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4114,"usage":{"cpu_time_ns":3529775791,"resident_bytes":36811866112,"physical_bytes":36788529720,"peak_physical_bytes":36788496952,"disk_read_bytes":36767703040,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4214,"usage":{"cpu_time_ns":3736453708,"resident_bytes":38154387456,"physical_bytes":38131690976,"peak_physical_bytes":38131690976,"disk_read_bytes":38134026240,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4323,"usage":{"cpu_time_ns":3896475333,"resident_bytes":39605485568,"physical_bytes":39583527448,"peak_physical_bytes":39583494680,"disk_read_bytes":39570821120,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4428,"usage":{"cpu_time_ns":4054501625,"resident_bytes":40999976960,"physical_bytes":40978740696,"peak_physical_bytes":40978724312,"disk_read_bytes":40968052736,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4528,"usage":{"cpu_time_ns":4234329958,"resident_bytes":42344955904,"physical_bytes":42324375984,"peak_physical_bytes":42324326832,"disk_read_bytes":42334523392,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4629,"usage":{"cpu_time_ns":4428187833,"resident_bytes":43685281792,"physical_bytes":43665358192,"peak_physical_bytes":43665358192,"disk_read_bytes":43699920896,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4738,"usage":{"cpu_time_ns":4620184041,"resident_bytes":45136707584,"physical_bytes":45117505936,"peak_physical_bytes":45117456784,"disk_read_bytes":45157412864,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4838,"usage":{"cpu_time_ns":4817709458,"resident_bytes":46482227200,"physical_bytes":46463665464,"peak_physical_bytes":46463632696,"disk_read_bytes":46461468672,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":4939,"usage":{"cpu_time_ns":5087111041,"resident_bytes":47124643840,"physical_bytes":47758149880,"peak_physical_bytes":47758117112,"disk_read_bytes":47805988864,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5039,"usage":{"cpu_time_ns":5330960250,"resident_bytes":47830368256,"physical_bytes":49067805856,"peak_physical_bytes":49067756704,"disk_read_bytes":49075716096,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5140,"usage":{"cpu_time_ns":5565245916,"resident_bytes":48424222720,"physical_bytes":50358259712,"peak_physical_bytes":50358177792,"disk_read_bytes":50312159232,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5241,"usage":{"cpu_time_ns":5809599958,"resident_bytes":49109893120,"physical_bytes":51695211504,"peak_physical_bytes":51695178736,"disk_read_bytes":51717169152,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5341,"usage":{"cpu_time_ns":6025637250,"resident_bytes":49509433344,"physical_bytes":52810307336,"peak_physical_bytes":52810307336,"disk_read_bytes":52828200960,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5441,"usage":{"cpu_time_ns":6277023708,"resident_bytes":50092883968,"physical_bytes":54057835136,"peak_physical_bytes":54057736832,"disk_read_bytes":54030950400,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5542,"usage":{"cpu_time_ns":6485585166,"resident_bytes":50358255616,"physical_bytes":55029800272,"peak_physical_bytes":55029800272,"disk_read_bytes":55103369216,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5647,"usage":{"cpu_time_ns":6724719041,"resident_bytes":50827952128,"physical_bytes":56224129152,"peak_physical_bytes":56224030848,"disk_read_bytes":56207822848,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5747,"usage":{"cpu_time_ns":6996546250,"resident_bytes":51449561088,"physical_bytes":57507456040,"peak_physical_bytes":57507259432,"disk_read_bytes":57500983296,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5858,"usage":{"cpu_time_ns":7243756750,"resident_bytes":51921190912,"physical_bytes":58790062008,"peak_physical_bytes":58789996472,"disk_read_bytes":58775035904,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":5958,"usage":{"cpu_time_ns":7446427916,"resident_bytes":52491010048,"physical_bytes":60089330528,"peak_physical_bytes":60089297760,"disk_read_bytes":60060053504,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6058,"usage":{"cpu_time_ns":7682755875,"resident_bytes":52835762176,"physical_bytes":61229625000,"peak_physical_bytes":61229510312,"disk_read_bytes":61182423040,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6159,"usage":{"cpu_time_ns":7907141916,"resident_bytes":53200027648,"physical_bytes":62299910568,"peak_physical_bytes":62299877800,"disk_read_bytes":62324518912,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6260,"usage":{"cpu_time_ns":8117985625,"resident_bytes":53427224576,"physical_bytes":63318340704,"peak_physical_bytes":63318291552,"disk_read_bytes":63299579904,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6366,"usage":{"cpu_time_ns":8344865375,"resident_bytes":53695791104,"physical_bytes":64402847632,"peak_physical_bytes":64402798480,"disk_read_bytes":64412700672,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6466,"usage":{"cpu_time_ns":8544942083,"resident_bytes":53949087744,"physical_bytes":65428470392,"peak_physical_bytes":65428421240,"disk_read_bytes":65453092864,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6567,"usage":{"cpu_time_ns":8765998375,"resident_bytes":54243868672,"physical_bytes":66507488608,"peak_physical_bytes":66507390304,"disk_read_bytes":66502717440,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6667,"usage":{"cpu_time_ns":8996212375,"resident_bytes":54710779904,"physical_bytes":67642228856,"peak_physical_bytes":67642196088,"disk_read_bytes":67640258560,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6768,"usage":{"cpu_time_ns":9237024000,"resident_bytes":55070507008,"physical_bytes":68751917968,"peak_physical_bytes":68751770512,"disk_read_bytes":68733317120,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6868,"usage":{"cpu_time_ns":9457918291,"resident_bytes":55224303616,"physical_bytes":69703845400,"peak_physical_bytes":69703845400,"disk_read_bytes":69683851264,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":6974,"usage":{"cpu_time_ns":9727646583,"resident_bytes":55649435648,"physical_bytes":70908709264,"peak_physical_bytes":70908676496,"disk_read_bytes":70916554752,"disk_write_bytes":0}} +test engine::metal::qwen_mtplx::issue103_analysis::issue103_mtp_final_receipts ... [qwen4_exp] ngram table: streamed from SSD (file-backed pages, hot cache 1023M, prefetch on) — resident only at >=160G RAM or MTPLX_NGRAM_RESIDENT=1 +{"event":"test_resource_sample","elapsed_ms":7082,"usage":{"cpu_time_ns":9956197708,"resident_bytes":55915675648,"physical_bytes":71831704712,"peak_physical_bytes":71831704712,"disk_read_bytes":71741497344,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7191,"usage":{"cpu_time_ns":10084534333,"resident_bytes":55804461056,"physical_bytes":72000394448,"peak_physical_bytes":72000394448,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7301,"usage":{"cpu_time_ns":10207332125,"resident_bytes":55804542976,"physical_bytes":72014091472,"peak_physical_bytes":72014091472,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7412,"usage":{"cpu_time_ns":10313632708,"resident_bytes":55804542976,"physical_bytes":72020497616,"peak_physical_bytes":72020497616,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7520,"usage":{"cpu_time_ns":10425756791,"resident_bytes":55746232320,"physical_bytes":72021021904,"peak_physical_bytes":72021021904,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7630,"usage":{"cpu_time_ns":10537092916,"resident_bytes":55746248704,"physical_bytes":72021136592,"peak_physical_bytes":72021136592,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7740,"usage":{"cpu_time_ns":10649325375,"resident_bytes":55746265088,"physical_bytes":72024397008,"peak_physical_bytes":72024397008,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7851,"usage":{"cpu_time_ns":10764781958,"resident_bytes":55746281472,"physical_bytes":72030803152,"peak_physical_bytes":72030803152,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":7961,"usage":{"cpu_time_ns":10876798000,"resident_bytes":55746281472,"physical_bytes":72031327440,"peak_physical_bytes":72031327440,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8071,"usage":{"cpu_time_ns":10988781500,"resident_bytes":55746281472,"physical_bytes":72031442128,"peak_physical_bytes":72031442128,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8181,"usage":{"cpu_time_ns":11098897291,"resident_bytes":55746281472,"physical_bytes":72031556816,"peak_physical_bytes":72031556816,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8291,"usage":{"cpu_time_ns":11213435208,"resident_bytes":55417700352,"physical_bytes":72034817232,"peak_physical_bytes":72034817232,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8401,"usage":{"cpu_time_ns":11319259291,"resident_bytes":54944546816,"physical_bytes":72041092304,"peak_physical_bytes":72041108688,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8511,"usage":{"cpu_time_ns":11422420500,"resident_bytes":54699704320,"physical_bytes":72041616592,"peak_physical_bytes":72041632976,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8621,"usage":{"cpu_time_ns":11538945875,"resident_bytes":54672785408,"physical_bytes":72041747664,"peak_physical_bytes":72041764048,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8730,"usage":{"cpu_time_ns":11644984250,"resident_bytes":54526328832,"physical_bytes":72041878736,"peak_physical_bytes":72041878736,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8841,"usage":{"cpu_time_ns":11750126416,"resident_bytes":54143795200,"physical_bytes":72045122768,"peak_physical_bytes":72045122768,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":8951,"usage":{"cpu_time_ns":11847817625,"resident_bytes":54062514176,"physical_bytes":72052184272,"peak_physical_bytes":72052200656,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9061,"usage":{"cpu_time_ns":11980132166,"resident_bytes":54046244864,"physical_bytes":72086361296,"peak_physical_bytes":72086377680,"disk_read_bytes":71745740800,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9162,"usage":{"cpu_time_ns":12132194791,"resident_bytes":54046769152,"physical_bytes":72378586680,"peak_physical_bytes":72378537528,"disk_read_bytes":72070569984,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9272,"usage":{"cpu_time_ns":12405722958,"resident_bytes":54622257152,"physical_bytes":73737034160,"peak_physical_bytes":73736935856,"disk_read_bytes":73403424768,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9382,"usage":{"cpu_time_ns":12518504416,"resident_bytes":54480044032,"physical_bytes":73775635776,"peak_physical_bytes":73775635776,"disk_read_bytes":73404215296,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9492,"usage":{"cpu_time_ns":12628616666,"resident_bytes":54474817536,"physical_bytes":73776880960,"peak_physical_bytes":73776897344,"disk_read_bytes":73404215296,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9602,"usage":{"cpu_time_ns":12751641666,"resident_bytes":54470852608,"physical_bytes":73777290560,"peak_physical_bytes":73777290560,"disk_read_bytes":73404215296,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9712,"usage":{"cpu_time_ns":12856346958,"resident_bytes":54468182016,"physical_bytes":73777438016,"peak_physical_bytes":73777438016,"disk_read_bytes":73404215296,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":9823,"usage":{"cpu_time_ns":12984643500,"resident_bytes":54468640768,"physical_bytes":73781042496,"peak_physical_bytes":73781042496,"disk_read_bytes":73404215296,"disk_write_bytes":0}} +{"case":"mtp_final","exit":"max_tokens","depth":1,"temperature":0.0,"prompt_ids":[9707,11,1075],"tokens":[11],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"first_primary_stop","depth":1,"temperature":0.0,"prompt_ids":[9707,11,1075],"tokens":[11],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"max_tokens","depth":1,"temperature":0.6,"prompt_ids":[9707,11,1075],"tokens":[314],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"first_primary_stop","depth":1,"temperature":0.6,"prompt_ids":[9707,11,1075],"tokens":[314],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"max_tokens","depth":2,"temperature":0.0,"prompt_ids":[9707,11,1075],"tokens":[11],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"first_primary_stop","depth":2,"temperature":0.0,"prompt_ids":[9707,11,1075],"tokens":[11],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"max_tokens","depth":2,"temperature":0.6,"prompt_ids":[9707,11,1075],"tokens":[314],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"first_primary_stop","depth":2,"temperature":0.6,"prompt_ids":[9707,11,1075],"tokens":[314],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"max_tokens","depth":3,"temperature":0.0,"prompt_ids":[9707,11,1075],"tokens":[11],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"first_primary_stop","depth":3,"temperature":0.0,"prompt_ids":[9707,11,1075],"tokens":[11],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"max_tokens","depth":3,"temperature":0.6,"prompt_ids":[9707,11,1075],"tokens":[314],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"case":"mtp_final","exit":"first_primary_stop","depth":3,"temperature":0.6,"prompt_ids":[9707,11,1075],"tokens":[314],"key_length":4,"restored_cached_tokens":4,"trunk_offsets":[3,3,3,3,3,3,3,3,3,3,3,3],"mtp_offset":2,"pending":null,"final_state_error":null} +{"event":"test_resource_sample","elapsed_ms":9933,"usage":{"cpu_time_ns":13093723958,"resident_bytes":54471442432,"physical_bytes":73832603712,"peak_physical_bytes":73832603712,"disk_read_bytes":73405005824,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10037,"usage":{"cpu_time_ns":13210911416,"resident_bytes":54474424320,"physical_bytes":73909314944,"peak_physical_bytes":73909314944,"disk_read_bytes":73406578688,"disk_write_bytes":0}} +{"case":"mtp_next_turn","missing_token":314,"suffix_ids":[1075],"different_logits":247643,"bad_sha256":"9c22c019eeb07fead0760dc06ef796e4fbf3f5b4946b02ad808f430ecc6cee5c","good_sha256":"9a556dd9d3063864b70c3338cb0cdf76cc578a01de1c7771b1da0abf3e776feb"} +{"event":"test_resource_sample","elapsed_ms":10145,"usage":{"cpu_time_ns":13329246375,"resident_bytes":54441443328,"physical_bytes":73715716392,"peak_physical_bytes":73909577088,"disk_read_bytes":73406578688,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10255,"usage":{"cpu_time_ns":13432198666,"resident_bytes":46125727744,"physical_bytes":59759195904,"peak_physical_bytes":73909577088,"disk_read_bytes":73406578688,"disk_write_bytes":0}} +ok +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +test engine::metal::qwen_mtplx::issue103_analysis::issue103_numeric_receipts ... {"case":"bf16_gate","gate":-6.84375,"eager":[0.00106048583984375,0.00106048583984375,0.00106048583984375,0.00106048583984375],"compiled":[0.001068115234375,0.001068115234375,0.001068115234375,0.001068115234375]} +{"case":"qsa_divisor","operand_bits":1093993715,"eager":[0.13258253037929535,0.13258254528045654,0.0883883461356163,0.0],"compiled":[0.13258251547813416,0.13258251547813416,0.08838833123445511,0.0]} +{"case":"yarn_amplitude","operand_bits":1066516124,"eager":[1.7079442739486694,1.707944393157959,1.13862943649292,0.0],"compiled":[1.7079435586929321,1.7079436779022217,1.1386289596557617,0.0]} +ok +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +{"event":"test_resource_sample","elapsed_ms":10365,"usage":{"cpu_time_ns":13542893125,"resident_bytes":38218661888,"physical_bytes":45081053912,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10476,"usage":{"cpu_time_ns":13635585125,"resident_bytes":25998786560,"physical_bytes":27171590880,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +test engine::metal::qwen_mtplx::issue103_analysis::issue103_sampling_receipts ... {"case":"sampling_law","temperature":1.0,"top_k":2,"top_p":0.5,"ordinary_probabilities":[0.5714286079212102,0.42857139207878986],"ordinary_counts":[266,246],"lazy_counts":[128,0],"expected_probabilities":[0.5714285714285714,0.42857142857142855]} +{"case":"invalid_logits","row":"nan","temperature":0.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"nan","temperature":0.6,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"nan","temperature":1.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits_lazy","row":"nan","result":{"Ok":[19]}} +{"case":"invalid_logits","row":"positive_inf","temperature":0.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"positive_inf","temperature":0.6,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"positive_inf","temperature":1.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits_lazy","row":"positive_inf","result":{"Ok":[0]}} +{"case":"invalid_logits","row":"all_negative_inf","temperature":0.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"all_negative_inf","temperature":0.6,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"all_negative_inf","temperature":1.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits_lazy","row":"all_negative_inf","result":{"Ok":[16]}} +{"case":"invalid_logits","row":"valid_mask","temperature":0.0,"ordinary":{"Ok":1}} +{"case":"invalid_logits","row":"valid_mask","temperature":0.6,"ordinary":{"Ok":1}} +{"case":"invalid_logits","row":"valid_mask","temperature":1.0,"ordinary":{"Ok":1}} +{"case":"invalid_logits_lazy","row":"valid_mask","result":{"Ok":[3]}} +ok +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +{"event":"test_resource_sample","elapsed_ms":10586,"usage":{"cpu_time_ns":13737424208,"resident_bytes":7683784704,"physical_bytes":8482839608,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10696,"usage":{"cpu_time_ns":13847494041,"resident_bytes":157466624,"physical_bytes":241943656,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +test engine::metal::qwen_mtplx::issue103_analysis::issue103_sorted_gather_receipts ... {"case":"sorted_gather","tokens":3276,"routed_rows":32760,"bad_rows":0,"first_bad":null,"last_bad":null,"dispatches":"[(\"affine_gather_qmm_rhs_nax_nt_bfloat16_t_gs_32_b_4_bm_64_bn_64_bk_64_wm_2_wn_2\", [4, 512, 1], [32, 2, 2], false)]"} +{"event":"test_resource_sample","elapsed_ms":10806,"usage":{"cpu_time_ns":13950310791,"resident_bytes":208158720,"physical_bytes":292668568,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":10916,"usage":{"cpu_time_ns":14060338041,"resident_bytes":208158720,"physical_bytes":292668568,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +{"case":"sorted_gather","tokens":3277,"routed_rows":32770,"bad_rows":32,"first_bad":0,"last_bad":31,"dispatches":"[(\"affine_gather_qmm_rhs_nax_nt_bfloat16_t_gs_32_b_4_bm_64_bn_64_bk_64_wm_2_wn_2\", [4, 513, 1], [32, 2, 2], false)]"} +{"event":"test_resource_sample","elapsed_ms":11026,"usage":{"cpu_time_ns":14169341750,"resident_bytes":260472832,"physical_bytes":344999088,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +{"case":"sorted_gather","tokens":3404,"routed_rows":34040,"bad_rows":1280,"first_bad":0,"last_bad":1279,"dispatches":"[(\"affine_gather_qmm_rhs_nax_nt_bfloat16_t_gs_32_b_4_bm_64_bn_64_bk_64_wm_2_wn_2\", [4, 532, 1], [32, 2, 2], false)]"} +{"event":"test_resource_sample","elapsed_ms":11136,"usage":{"cpu_time_ns":14278719916,"resident_bytes":323731456,"physical_bytes":408290528,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11246,"usage":{"cpu_time_ns":14388772416,"resident_bytes":323731456,"physical_bytes":408290528,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +{"case":"sorted_gather","tokens":4095,"routed_rows":40950,"bad_rows":8192,"first_bad":0,"last_bad":8191,"dispatches":"[(\"affine_gather_qmm_rhs_nax_nt_bfloat16_t_gs_32_b_4_bm_64_bn_64_bk_64_wm_2_wn_2\", [4, 640, 1], [32, 2, 2], false)]"} +{"event":"test_resource_sample","elapsed_ms":11353,"usage":{"cpu_time_ns":14496169333,"resident_bytes":323813376,"physical_bytes":408290528,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11464,"usage":{"cpu_time_ns":14606197250,"resident_bytes":323813376,"physical_bytes":408290528,"peak_physical_bytes":73909577088,"disk_read_bytes":73408286720,"disk_write_bytes":0}} +{"case":"sorted_gather","tokens":4096,"routed_rows":40960,"bad_rows":0,"first_bad":null,"last_bad":null,"dispatches":"[(\"affine_gather_qmm_rhs_nax_nt_bfloat16_t_gs_32_b_4_bm_64_bn_64_bk_64_wm_2_wn_2\", [4, 640, 1], [32, 2, 2], false)]"} +ok +{"event":"test_resource_sample","elapsed_ms":11570,"usage":{"cpu_time_ns":14715979375,"resident_bytes":176521216,"physical_bytes":143328360,"peak_physical_bytes":73909577088,"disk_read_bytes":73421099008,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11680,"usage":{"cpu_time_ns":14825996375,"resident_bytes":179535872,"physical_bytes":146343016,"peak_physical_bytes":73909577088,"disk_read_bytes":73421099008,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11790,"usage":{"cpu_time_ns":14936035125,"resident_bytes":191741952,"physical_bytes":141771880,"peak_physical_bytes":73909577088,"disk_read_bytes":73421099008,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":11900,"usage":{"cpu_time_ns":15046046750,"resident_bytes":198180864,"physical_bytes":148030568,"peak_physical_bytes":73909577088,"disk_read_bytes":73421099008,"disk_write_bytes":0}} +test engine::metal::qwen_mtplx::issue103_analysis::issue103_tokenizer_receipts ... {"case":"native_tokenizer","text":"भारत एक विशाल देश है जिसकी संस्कृति बहुत पुरानी है।","actual_ids":[149755,148504,76519,171549,165997,151224,197734,170046,184864,160357,172281,237627,158827,153570,199638,80752,151859,181117,170046,154845],"expected_ids":[149755,148504,76519,171549,165997,151224,197734,170046,184864,160357,172281,237627,158827,153570,199638,80752,151859,181117,170046,154845],"matches":true} +{"case":"native_tokenizer","text":"ประเทศไทยมีวัฒนธรรมที่เก่าแก่และสวยงามมาก","actual_ids":[156150,148450,186319,148285,178118,155545,149119,162566,149196],"expected_ids":[156150,148450,186319,148285,178118,155545,149119,162566,149196],"matches":true} +{"case":"native_tokenizer","text":"إِنَّ اللُّغَةَ الْعَرَبِيَّةَ جَمِيلَةٌ","actual_ids":[88792,164372,148784,159613,78958,191556,205629,153458,150765,151003,178402,27004,79333,152397,216808,233790],"expected_ids":[88792,164372,148784,159613,78958,191556,205629,153458,150765,151003,178402,27004,79333,152397,216808,233790],"matches":true} +{"case":"native_tokenizer","text":"The quick brown fox jumps over the lazy dog near the river bank.","actual_ids":[760,3841,13477,37550,33075,888,279,15217,5388,3043,279,14367,5883,13],"expected_ids":[760,3841,13477,37550,33075,888,279,15217,5388,3043,279,14367,5883,13],"matches":true} +{"case":"native_tokenizer","text":"def foo(x):\n return x + 1\n","actual_ids":[727,14785,2007,1590,198,262,460,830,478,220,16,198],"expected_ids":[727,14785,2007,1590,198,262,460,830,478,220,16,198],"matches":true} +{"case":"native_tokenizer","text":"中国的文化历史悠久。","actual_ids":[108733,96655,125188,1710],"expected_ids":[108733,96655,125188,1710],"matches":true} +{"case":"native_tokenizer","text":"a🙂b ©c ∑d —e …f €g ™h\n","actual_ids":[64,169171,65,7013,66,11654,239,67,1892,68,4444,69,12612,70,27717,95,71,198],"expected_ids":[64,169171,65,7013,66,11654,239,67,1892,68,4444,69,12612,70,27717,95,71,198],"matches":true} +{"case":"native_tokenizer","text":"x́y १२३ ١٢٣ ①Ⅷ²‍Z","actual_ids":[87,52033,88,220,12256,100,12256,101,12256,102,220,149,94,149,95,149,96,2765,239,254,68086,100,28495,373,235,57],"expected_ids":[87,52033,88,220,12256,100,12256,101,12256,102,220,149,94,149,95,149,96,220,46701,254,68086,100,28495,373,235,57],"matches":false} +{"case":"native_tokenizer","text":" ①","actual_ids":[2765,239,254],"expected_ids":[220,46701,254],"matches":false} +{"case":"native_tokenizer","text":" Ⅷ","actual_ids":[2765,88608],"expected_ids":[220,68086,100],"matches":false} +{"case":"native_tokenizer","text":" ²","actual_ids":[2071,110],"expected_ids":[220,28495],"matches":false} +{"case":"native_tokenizer","text":" 𝟘","actual_ids":[78449,253,246],"expected_ids":[220,54362,253,246],"matches":false} +{"case":"tokenizer_summary","mismatches":5} +ok +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +{"event":"test_resource_sample","elapsed_ms":12011,"usage":{"cpu_time_ns":15155875541,"resident_bytes":154468352,"physical_bytes":80725096,"peak_physical_bytes":73909577088,"disk_read_bytes":73421107200,"disk_write_bytes":0}} +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +test engine::metal::qwen_mtplx::issue103_analysis::issue103_vision_position_receipts ... {"case":"vision_positions","start":0,"rows":4,"table_end":4,"delta":-1,"different_rows":[],"actual_keys":[0.1982421875,0.396484375,0.59375,0.79296875,0.98828125,1.1875,1.3828125,1.5859375,-0.478515625,0.671875,0.85546875,0.9453125,1.1484375,1.171875,1.1875,1.2578125,-1.265625,0.515625,0.875,0.96484375,0.326171875,1.28125,1.15625,1.1640625,-1.2890625,0.55859375,0.90234375,0.9765625,0.373046875,1.265625,1.125,1.1171875],"expected_keys":[0.1982421875,0.396484375,0.59375,0.79296875,0.98828125,1.1875,1.3828125,1.5859375,-0.478515625,0.671875,0.85546875,0.9453125,1.1484375,1.171875,1.1875,1.2578125,-1.265625,0.515625,0.875,0.96484375,0.326171875,1.28125,1.15625,1.1640625,-1.2890625,0.55859375,0.90234375,0.9765625,0.373046875,1.265625,1.125,1.1171875]} +{"case":"vision_positions","start":4,"rows":2,"table_end":4,"delta":-1,"different_rows":[],"actual_keys":[-0.3359375,0.02783203125,0.55078125,0.7890625,-0.94921875,1.25,1.3984375,1.5859375,0.3125,0.294921875,0.8203125,0.94140625,-1.203125,1.3203125,1.2109375,1.2578125],"expected_keys":[-0.3359375,0.02783203125,0.55078125,0.7890625,-0.94921875,1.25,1.3984375,1.5859375,0.3125,0.294921875,0.8203125,0.94140625,-1.203125,1.3203125,1.2109375,1.2578125]} +{"case":"vision_positions","start":2,"rows":4,"table_end":4,"delta":-1,"different_rows":[0,1],"actual_keys":[-0.72265625,0.275390625,0.578125,0.79296875,0.69921875,1.21875,1.390625,1.5859375,-1.2265625,0.55078125,0.84375,0.94140625,0.216796875,1.234375,1.1953125,1.2578125,-0.9609375,0.515625,0.88671875,0.96484375,-0.890625,1.28125,1.140625,1.1640625,0.1962890625,0.4296875,0.90234375,0.9765625,-1.328125,1.3203125,1.125,1.125],"expected_keys":[-0.98046875,0.02783203125,0.5390625,0.7890625,-0.2314453125,1.25,1.40625,1.5859375,-1.2265625,0.423828125,0.8203125,0.94140625,0.216796875,1.28125,1.2109375,1.2578125,-0.9609375,0.515625,0.88671875,0.96484375,-0.890625,1.28125,1.140625,1.1640625,0.1962890625,0.4296875,0.90234375,0.9765625,-1.328125,1.3203125,1.125,1.125]} +{"case":"vision_sparse_qsa","prefix_rows":12,"topk_blocks":1,"ratio":4,"position_delta":0,"text_output":[0.10986328125,-0.1923828125,0.08544921875,0.498046875,-0.0791015625,0.16796875,0.31640625,-0.03173828125],"vision_output":[0.07568359375,-0.1455078125,0.05712890625,0.357421875,-0.034912109375,0.0927734375,0.25390625,-0.048828125]} +{"case":"active_attention_gate","gate":-6.84375,"actual":[0.0003185272216796875,-0.00074005126953125,0.00021266937255859375,0.00095367431640625,-0.00021266937255859375,0.0004253387451171875,0.000637054443359375,-0.00010633468627929688],"expected_verify":[0.0003204345703125,-0.0007476806640625,0.000213623046875,0.0009613037109375,-0.000213623046875,0.00042724609375,0.000640869140625,-0.0001068115234375]} +ok + +test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 511 filtered out; finished in 10.80s + +{"event":"test_resource_summary","elapsed_ms":12121,"peak":{"cpu_time_ns":15155875541,"resident_bytes":55915675648,"physical_bytes":73909314944,"peak_physical_bytes":73909577088,"disk_read_bytes":73421107200,"disk_write_bytes":0},"error":null} diff --git a/docs/issue103-analysis/native-gather.jsonl b/docs/issue103-analysis/native-gather.jsonl new file mode 100644 index 0000000..8315929 --- /dev/null +++ b/docs/issue103-analysis/native-gather.jsonl @@ -0,0 +1,4 @@ +{"case": "sorted_gather", "tokens": 3277, "routed_rows": 32770, "bad_rows": 32, "first_bad": 0, "last_bad": 31, "dispatches": "[(\"affine_gather_qmm_rhs_nax_nt_bfloat16_t_gs_32_b_4_bm_64_bn_64_bk_64_wm_2_wn_2\", [4, 513, 1], [32, 2, 2], false)]"} +{"case": "sorted_gather", "tokens": 3404, "routed_rows": 34040, "bad_rows": 1280, "first_bad": 0, "last_bad": 1279, "dispatches": "[(\"affine_gather_qmm_rhs_nax_nt_bfloat16_t_gs_32_b_4_bm_64_bn_64_bk_64_wm_2_wn_2\", [4, 532, 1], [32, 2, 2], false)]"} +{"case": "sorted_gather", "tokens": 4095, "routed_rows": 40950, "bad_rows": 8192, "first_bad": 0, "last_bad": 8191, "dispatches": "[(\"affine_gather_qmm_rhs_nax_nt_bfloat16_t_gs_32_b_4_bm_64_bn_64_bk_64_wm_2_wn_2\", [4, 640, 1], [32, 2, 2], false)]"} +{"case": "sorted_gather", "tokens": 4096, "routed_rows": 40960, "bad_rows": 0, "first_bad": null, "last_bad": null, "dispatches": "[(\"affine_gather_qmm_rhs_nax_nt_bfloat16_t_gs_32_b_4_bm_64_bn_64_bk_64_wm_2_wn_2\", [4, 640, 1], [32, 2, 2], false)]"} diff --git a/docs/issue103-analysis/native-model.jsonl b/docs/issue103-analysis/native-model.jsonl new file mode 100644 index 0000000..facbee1 --- /dev/null +++ b/docs/issue103-analysis/native-model.jsonl @@ -0,0 +1,13 @@ +{"case": "mtp_final", "exit": "max_tokens", "depth": 1, "temperature": 0.0, "prompt_ids": [9707, 11, 1075], "tokens": [11], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "first_primary_stop", "depth": 1, "temperature": 0.0, "prompt_ids": [9707, 11, 1075], "tokens": [11], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "max_tokens", "depth": 1, "temperature": 0.6, "prompt_ids": [9707, 11, 1075], "tokens": [314], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "first_primary_stop", "depth": 1, "temperature": 0.6, "prompt_ids": [9707, 11, 1075], "tokens": [314], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "max_tokens", "depth": 2, "temperature": 0.0, "prompt_ids": [9707, 11, 1075], "tokens": [11], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "first_primary_stop", "depth": 2, "temperature": 0.0, "prompt_ids": [9707, 11, 1075], "tokens": [11], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "max_tokens", "depth": 2, "temperature": 0.6, "prompt_ids": [9707, 11, 1075], "tokens": [314], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "first_primary_stop", "depth": 2, "temperature": 0.6, "prompt_ids": [9707, 11, 1075], "tokens": [314], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "max_tokens", "depth": 3, "temperature": 0.0, "prompt_ids": [9707, 11, 1075], "tokens": [11], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "first_primary_stop", "depth": 3, "temperature": 0.0, "prompt_ids": [9707, 11, 1075], "tokens": [11], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "max_tokens", "depth": 3, "temperature": 0.6, "prompt_ids": [9707, 11, 1075], "tokens": [314], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_final", "exit": "first_primary_stop", "depth": 3, "temperature": 0.6, "prompt_ids": [9707, 11, 1075], "tokens": [314], "key_length": 4, "restored_cached_tokens": 4, "trunk_offsets": [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "mtp_offset": 2, "pending": null, "final_state_error": null} +{"case": "mtp_next_turn", "missing_token": 314, "suffix_ids": [1075], "different_logits": 247643, "bad_sha256": "9c22c019eeb07fead0760dc06ef796e4fbf3f5b4946b02ad808f430ecc6cee5c", "good_sha256": "9a556dd9d3063864b70c3338cb0cdf76cc578a01de1c7771b1da0abf3e776feb"} diff --git a/docs/issue103-analysis/native-numeric.jsonl b/docs/issue103-analysis/native-numeric.jsonl new file mode 100644 index 0000000..d12c1d5 --- /dev/null +++ b/docs/issue103-analysis/native-numeric.jsonl @@ -0,0 +1,2 @@ +{"case": "qsa_divisor", "operand_bits": 1093993715, "eager": [0.13258253037929535, 0.13258254528045654, 0.0883883461356163, 0.0], "compiled": [0.13258251547813416, 0.13258251547813416, 0.08838833123445511, 0.0]} +{"case": "yarn_amplitude", "operand_bits": 1066516124, "eager": [1.7079442739486694, 1.707944393157959, 1.13862943649292, 0.0], "compiled": [1.7079435586929321, 1.7079436779022217, 1.1386289596557617, 0.0]} diff --git a/docs/issue103-analysis/native-sampling-final.log b/docs/issue103-analysis/native-sampling-final.log new file mode 100644 index 0000000..7cb5c30 --- /dev/null +++ b/docs/issue103-analysis/native-sampling-final.log @@ -0,0 +1,57 @@ +{"event":"test_supervisor_started","pid":94364,"limit_bytes":4294967296,"start_seconds":30.0,"idle_seconds":60.0,"sample_ms":100} +{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":100,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":206,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":316,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":426,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":528,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":638,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":744,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":854,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":963,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1066,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1176,"usage":{"cpu_time_ns":28750,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":0,"disk_write_bytes":0}} + +running 1 test +{"event":"test_resource_sample","elapsed_ms":1286,"usage":{"cpu_time_ns":26699458,"resident_bytes":21643264,"physical_bytes":6455944,"peak_physical_bytes":6455944,"disk_read_bytes":0,"disk_write_bytes":0}} +ds4: Metal device Apple M5 Max, 128.00 GiB RAM +ds4: Metal 4 tensor API enabled for Tensor kernels +ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on +ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx +{"event":"test_resource_sample","elapsed_ms":1396,"usage":{"cpu_time_ns":116719791,"resident_bytes":76595200,"physical_bytes":172147528,"peak_physical_bytes":172147528,"disk_read_bytes":0,"disk_write_bytes":0}} +test engine::metal::qwen_mtplx::issue103_analysis::issue103_sampling_receipts ... {"case":"sampling_law","temperature":1.0,"top_k":2,"top_p":0.5,"ordinary_probabilities":[0.5714286079212102,0.42857139207878986],"ordinary_counts":[266,246],"lazy_counts":[128,0],"expected_probabilities":[0.5714285714285714,0.42857142857142855]} +{"event":"test_resource_sample","elapsed_ms":1507,"usage":{"cpu_time_ns":201086416,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1617,"usage":{"cpu_time_ns":297881000,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1727,"usage":{"cpu_time_ns":394128958,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":1837,"usage":{"cpu_time_ns":490529041,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"case":"sampling_matrix","temperature":0.6,"top_k":20,"top_p":0.95,"ordinary_probabilities":[0.5081967493018351,0.3811475051809689,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.006147541417622011,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"lazy_counts":[274,208,2,5,3,3,1,4,2,2,3,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]} +{"event":"test_resource_sample","elapsed_ms":1945,"usage":{"cpu_time_ns":584859458,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2055,"usage":{"cpu_time_ns":681603625,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2165,"usage":{"cpu_time_ns":777497500,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2275,"usage":{"cpu_time_ns":873936416,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"case":"sampling_matrix","temperature":0.2,"top_k":8,"top_p":0.8,"ordinary_probabilities":[0.5486725996765459,0.41150438843847353,0.006637168647496752,0.006637168647496752,0.006637168647496752,0.006637168647496752,0.006637168647496752,0.006637168647496752,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"lazy_counts":[299,213,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]} +{"event":"test_resource_sample","elapsed_ms":2385,"usage":{"cpu_time_ns":969982000,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2496,"usage":{"cpu_time_ns":1064348916,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"event":"test_resource_sample","elapsed_ms":2606,"usage":{"cpu_time_ns":1158539375,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0}} +{"case":"sampling_matrix","temperature":1.0,"top_k":2,"top_p":1.0,"ordinary_probabilities":[0.5714285821011916,0.4285714178988084,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"lazy_counts":[299,213,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]} +{"case":"invalid_logits","row":"nan","temperature":0.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"nan","temperature":0.6,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"nan","temperature":1.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits_lazy","row":"nan","result":{"Ok":[19]}} +{"case":"invalid_logits","row":"positive_inf","temperature":0.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"positive_inf","temperature":0.6,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"positive_inf","temperature":1.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits_lazy","row":"positive_inf","result":{"Ok":[0]}} +{"case":"invalid_logits","row":"all_negative_inf","temperature":0.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"all_negative_inf","temperature":0.6,"ordinary":{"Ok":0}} +{"case":"invalid_logits","row":"all_negative_inf","temperature":1.0,"ordinary":{"Ok":0}} +{"case":"invalid_logits_lazy","row":"all_negative_inf","result":{"Ok":[16]}} +{"case":"invalid_logits","row":"valid_mask","temperature":0.0,"ordinary":{"Ok":1}} +{"case":"invalid_logits","row":"valid_mask","temperature":0.6,"ordinary":{"Ok":1}} +{"case":"invalid_logits","row":"valid_mask","temperature":1.0,"ordinary":{"Ok":1}} +{"case":"invalid_logits_lazy","row":"valid_mask","result":{"Ok":[3]}} +ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out; finished in 1.46s + +{"event":"test_resource_summary","elapsed_ms":2716,"peak":{"cpu_time_ns":1158539375,"resident_bytes":77283328,"physical_bytes":172245832,"peak_physical_bytes":172245832,"disk_read_bytes":0,"disk_write_bytes":0},"error":null} diff --git a/docs/issue103-analysis/native-sampling.jsonl b/docs/issue103-analysis/native-sampling.jsonl new file mode 100644 index 0000000..14d8332 --- /dev/null +++ b/docs/issue103-analysis/native-sampling.jsonl @@ -0,0 +1,19 @@ +{"case": "sampling_matrix", "temperature": 0.6, "top_k": 20, "top_p": 0.95, "ordinary_probabilities": [0.5081967493018351, 0.3811475051809689, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.006147541417622011, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "lazy_counts": [274, 208, 2, 5, 3, 3, 1, 4, 2, 2, 3, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]} +{"case": "sampling_matrix", "temperature": 0.2, "top_k": 8, "top_p": 0.8, "ordinary_probabilities": [0.5486725996765459, 0.41150438843847353, 0.006637168647496752, 0.006637168647496752, 0.006637168647496752, 0.006637168647496752, 0.006637168647496752, 0.006637168647496752, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "lazy_counts": [299, 213, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]} +{"case": "sampling_matrix", "temperature": 1.0, "top_k": 2, "top_p": 1.0, "ordinary_probabilities": [0.5714285821011916, 0.4285714178988084, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "lazy_counts": [299, 213, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]} +{"case": "invalid_logits", "row": "nan", "temperature": 0.0, "ordinary": {"Ok": 0}} +{"case": "invalid_logits", "row": "nan", "temperature": 0.6, "ordinary": {"Ok": 0}} +{"case": "invalid_logits", "row": "nan", "temperature": 1.0, "ordinary": {"Ok": 0}} +{"case": "invalid_logits_lazy", "row": "nan", "result": {"Ok": [19]}} +{"case": "invalid_logits", "row": "positive_inf", "temperature": 0.0, "ordinary": {"Ok": 0}} +{"case": "invalid_logits", "row": "positive_inf", "temperature": 0.6, "ordinary": {"Ok": 0}} +{"case": "invalid_logits", "row": "positive_inf", "temperature": 1.0, "ordinary": {"Ok": 0}} +{"case": "invalid_logits_lazy", "row": "positive_inf", "result": {"Ok": [0]}} +{"case": "invalid_logits", "row": "all_negative_inf", "temperature": 0.0, "ordinary": {"Ok": 0}} +{"case": "invalid_logits", "row": "all_negative_inf", "temperature": 0.6, "ordinary": {"Ok": 0}} +{"case": "invalid_logits", "row": "all_negative_inf", "temperature": 1.0, "ordinary": {"Ok": 0}} +{"case": "invalid_logits_lazy", "row": "all_negative_inf", "result": {"Ok": [16]}} +{"case": "invalid_logits", "row": "valid_mask", "temperature": 0.0, "ordinary": {"Ok": 1}} +{"case": "invalid_logits", "row": "valid_mask", "temperature": 0.6, "ordinary": {"Ok": 1}} +{"case": "invalid_logits", "row": "valid_mask", "temperature": 1.0, "ordinary": {"Ok": 1}} +{"case": "invalid_logits_lazy", "row": "valid_mask", "result": {"Ok": [3]}} diff --git a/docs/issue103-analysis/native-tokenizer-final.log b/docs/issue103-analysis/native-tokenizer-final.log new file mode 100644 index 0000000..4c76626 --- /dev/null +++ b/docs/issue103-analysis/native-tokenizer-final.log @@ -0,0 +1,19 @@ + +running 1 test +test engine::metal::qwen_mtplx::issue103_analysis::issue103_tokenizer_receipts ... {"case":"native_tokenizer","text":"भारत एक विशाल देश है जिसकी संस्कृति बहुत पुरानी है।","actual_ids":[149755,148504,76519,171549,165997,151224,197734,170046,184864,160357,172281,237627,158827,153570,199638,80752,151859,181117,170046,154845],"expected_ids":[149755,148504,76519,171549,165997,151224,197734,170046,184864,160357,172281,237627,158827,153570,199638,80752,151859,181117,170046,154845],"matches":true} +{"case":"native_tokenizer","text":"ประเทศไทยมีวัฒนธรรมที่เก่าแก่และสวยงามมาก","actual_ids":[156150,148450,186319,148285,178118,155545,149119,162566,149196],"expected_ids":[156150,148450,186319,148285,178118,155545,149119,162566,149196],"matches":true} +{"case":"native_tokenizer","text":"إِنَّ اللُّغَةَ الْعَرَبِيَّةَ جَمِيلَةٌ","actual_ids":[88792,164372,148784,159613,78958,191556,205629,153458,150765,151003,178402,27004,79333,152397,216808,233790],"expected_ids":[88792,164372,148784,159613,78958,191556,205629,153458,150765,151003,178402,27004,79333,152397,216808,233790],"matches":true} +{"case":"native_tokenizer","text":"The quick brown fox jumps over the lazy dog near the river bank.","actual_ids":[760,3841,13477,37550,33075,888,279,15217,5388,3043,279,14367,5883,13],"expected_ids":[760,3841,13477,37550,33075,888,279,15217,5388,3043,279,14367,5883,13],"matches":true} +{"case":"native_tokenizer","text":"def foo(x):\n return x + 1\n","actual_ids":[727,14785,2007,1590,198,262,460,830,478,220,16,198],"expected_ids":[727,14785,2007,1590,198,262,460,830,478,220,16,198],"matches":true} +{"case":"native_tokenizer","text":"中国的文化历史悠久。","actual_ids":[108733,96655,125188,1710],"expected_ids":[108733,96655,125188,1710],"matches":true} +{"case":"native_tokenizer","text":"a🙂b ©c ∑d —e …f €g ™h\n","actual_ids":[64,169171,65,7013,66,11654,239,67,1892,68,4444,69,12612,70,27717,95,71,198],"expected_ids":[64,169171,65,7013,66,11654,239,67,1892,68,4444,69,12612,70,27717,95,71,198],"matches":true} +{"case":"native_tokenizer","text":"x́y १२३ ١٢٣ ①Ⅷ²‍Z","actual_ids":[87,52033,88,220,12256,100,12256,101,12256,102,220,149,94,149,95,149,96,2765,239,254,68086,100,28495,373,235,57],"expected_ids":[87,52033,88,220,12256,100,12256,101,12256,102,220,149,94,149,95,149,96,220,46701,254,68086,100,28495,373,235,57],"matches":false} +{"case":"native_tokenizer","text":" ①","actual_ids":[2765,239,254],"expected_ids":[220,46701,254],"matches":false} +{"case":"native_tokenizer","text":" Ⅷ","actual_ids":[2765,88608],"expected_ids":[220,68086,100],"matches":false} +{"case":"native_tokenizer","text":" ²","actual_ids":[2071,110],"expected_ids":[220,28495],"matches":false} +{"case":"native_tokenizer","text":" 𝟘","actual_ids":[78449,253,246],"expected_ids":[220,54362,253,246],"matches":false} +{"case":"tokenizer_summary","mismatches":5} +ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out; finished in 0.47s + diff --git a/docs/issue103-analysis/native-tokenizer.jsonl b/docs/issue103-analysis/native-tokenizer.jsonl new file mode 100644 index 0000000..a17a241 --- /dev/null +++ b/docs/issue103-analysis/native-tokenizer.jsonl @@ -0,0 +1,12 @@ +{"case": "native_tokenizer", "text": "ประเทศไทยมีวัฒนธรรมที่เก่าแก่และสวยงามมาก", "actual_ids": [156150, 148450, 186319, 148285, 178118, 155545, 149119, 162566, 149196], "expected_ids": [156150, 148450, 186319, 148285, 178118, 155545, 149119, 162566, 149196], "matches": true} +{"case": "native_tokenizer", "text": "إِنَّ اللُّغَةَ الْعَرَبِيَّةَ جَمِيلَةٌ", "actual_ids": [88792, 164372, 148784, 159613, 78958, 191556, 205629, 153458, 150765, 151003, 178402, 27004, 79333, 152397, 216808, 233790], "expected_ids": [88792, 164372, 148784, 159613, 78958, 191556, 205629, 153458, 150765, 151003, 178402, 27004, 79333, 152397, 216808, 233790], "matches": true} +{"case": "native_tokenizer", "text": "The quick brown fox jumps over the lazy dog near the river bank.", "actual_ids": [760, 3841, 13477, 37550, 33075, 888, 279, 15217, 5388, 3043, 279, 14367, 5883, 13], "expected_ids": [760, 3841, 13477, 37550, 33075, 888, 279, 15217, 5388, 3043, 279, 14367, 5883, 13], "matches": true} +{"case": "native_tokenizer", "text": "def foo(x):\n return x + 1\n", "actual_ids": [727, 14785, 2007, 1590, 198, 262, 460, 830, 478, 220, 16, 198], "expected_ids": [727, 14785, 2007, 1590, 198, 262, 460, 830, 478, 220, 16, 198], "matches": true} +{"case": "native_tokenizer", "text": "中国的文化历史悠久。", "actual_ids": [108733, 96655, 125188, 1710], "expected_ids": [108733, 96655, 125188, 1710], "matches": true} +{"case": "native_tokenizer", "text": "a🙂b ©c ∑d —e …f €g ™h\n", "actual_ids": [64, 169171, 65, 7013, 66, 11654, 239, 67, 1892, 68, 4444, 69, 12612, 70, 27717, 95, 71, 198], "expected_ids": [64, 169171, 65, 7013, 66, 11654, 239, 67, 1892, 68, 4444, 69, 12612, 70, 27717, 95, 71, 198], "matches": true} +{"case": "native_tokenizer", "text": "x́y १२३ ١٢٣ ①Ⅷ²‍Z", "actual_ids": [87, 52033, 88, 220, 12256, 100, 12256, 101, 12256, 102, 220, 149, 94, 149, 95, 149, 96, 2765, 239, 254, 68086, 100, 28495, 373, 235, 57], "expected_ids": [87, 52033, 88, 220, 12256, 100, 12256, 101, 12256, 102, 220, 149, 94, 149, 95, 149, 96, 220, 46701, 254, 68086, 100, 28495, 373, 235, 57], "matches": false} +{"case": "native_tokenizer", "text": " ①", "actual_ids": [2765, 239, 254], "expected_ids": [220, 46701, 254], "matches": false} +{"case": "native_tokenizer", "text": " Ⅷ", "actual_ids": [2765, 88608], "expected_ids": [220, 68086, 100], "matches": false} +{"case": "native_tokenizer", "text": " ²", "actual_ids": [2071, 110], "expected_ids": [220, 28495], "matches": false} +{"case": "native_tokenizer", "text": " 𝟘", "actual_ids": [78449, 253, 246], "expected_ids": [220, 54362, 253, 246], "matches": false} +{"case": "tokenizer_summary", "mismatches": 5} diff --git a/docs/issue103-analysis/native-vision.jsonl b/docs/issue103-analysis/native-vision.jsonl new file mode 100644 index 0000000..be0ece7 --- /dev/null +++ b/docs/issue103-analysis/native-vision.jsonl @@ -0,0 +1,4 @@ +{"case": "vision_positions", "start": 4, "rows": 2, "table_end": 4, "delta": -1, "different_rows": [], "actual_keys": [-0.3359375, 0.02783203125, 0.55078125, 0.7890625, -0.94921875, 1.25, 1.3984375, 1.5859375, 0.3125, 0.294921875, 0.8203125, 0.94140625, -1.203125, 1.3203125, 1.2109375, 1.2578125], "expected_keys": [-0.3359375, 0.02783203125, 0.55078125, 0.7890625, -0.94921875, 1.25, 1.3984375, 1.5859375, 0.3125, 0.294921875, 0.8203125, 0.94140625, -1.203125, 1.3203125, 1.2109375, 1.2578125]} +{"case": "vision_positions", "start": 2, "rows": 4, "table_end": 4, "delta": -1, "different_rows": [0, 1], "actual_keys": [-0.72265625, 0.275390625, 0.578125, 0.79296875, 0.69921875, 1.21875, 1.390625, 1.5859375, -1.2265625, 0.55078125, 0.84375, 0.94140625, 0.216796875, 1.234375, 1.1953125, 1.2578125, -0.9609375, 0.515625, 0.88671875, 0.96484375, -0.890625, 1.28125, 1.140625, 1.1640625, 0.1962890625, 0.4296875, 0.90234375, 0.9765625, -1.328125, 1.3203125, 1.125, 1.125], "expected_keys": [-0.98046875, 0.02783203125, 0.5390625, 0.7890625, -0.2314453125, 1.25, 1.40625, 1.5859375, -1.2265625, 0.423828125, 0.8203125, 0.94140625, 0.216796875, 1.28125, 1.2109375, 1.2578125, -0.9609375, 0.515625, 0.88671875, 0.96484375, -0.890625, 1.28125, 1.140625, 1.1640625, 0.1962890625, 0.4296875, 0.90234375, 0.9765625, -1.328125, 1.3203125, 1.125, 1.125]} +{"case": "vision_sparse_qsa", "prefix_rows": 12, "topk_blocks": 1, "ratio": 4, "position_delta": 0, "text_output": [0.10986328125, -0.1923828125, 0.08544921875, 0.498046875, -0.0791015625, 0.16796875, 0.31640625, -0.03173828125], "vision_output": [0.07568359375, -0.1455078125, 0.05712890625, 0.357421875, -0.034912109375, 0.0927734375, 0.25390625, -0.048828125]} +{"case": "active_attention_gate", "gate": -6.84375, "actual": [0.0003185272216796875, -0.00074005126953125, 0.00021266937255859375, 0.00095367431640625, -0.00021266937255859375, 0.0004253387451171875, 0.000637054443359375, -0.00010633468627929688], "expected_verify": [0.0003204345703125, -0.0007476806640625, 0.000213623046875, 0.0009613037109375, -0.000213623046875, 0.00042724609375, 0.000640869140625, -0.0001068115234375]} diff --git a/docs/issue103-analysis/old-oracle.jsonl b/docs/issue103-analysis/old-oracle.jsonl new file mode 100644 index 0000000..882fe81 --- /dev/null +++ b/docs/issue103-analysis/old-oracle.jsonl @@ -0,0 +1,35 @@ +{"case": "environment", "mlx": "0.32.2", "device": {"device_name": "Apple M5 Max", "max_recommended_working_set_size": 115448725504, "memory_size": 137438953472, "architecture": "applegpu_g17s", "max_buffer_length": 86586540032, "resource_limit": 499000}, "hashes": {"config.json": "826302af9ff3293cfc9975c9a5a96546fa0aa9f0808505adfc30a62fef806965", "tokenizer.json": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3", "merges.txt": "a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d", "chat_template.jinja": "c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041"}} +{"case": "sampling_law", "probabilities": [0.5714285675041357, 0.4285714324958642]} +{"case": "sampling_law_lazy", "counts": [128, 0]} +{"case": "sampling_matrix", "temperature": 0.6, "top_k": 20, "top_p": 0.95, "ordinary_probabilities": [0.5081967376594639, 0.3811475366342391, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.006147540317016498, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "lazy_counts": [274, 208, 2, 5, 3, 3, 1, 4, 2, 2, 3, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]} +{"case": "sampling_matrix", "temperature": 0.2, "top_k": 8, "top_p": 0.8, "ordinary_probabilities": [0.5486725619186343, 0.4115044341651371, 0.006637167319371426, 0.006637167319371426, 0.006637167319371426, 0.006637167319371426, 0.006637167319371426, 0.006637167319371426, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "lazy_counts": [299, 213, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]} +{"case": "sampling_matrix", "temperature": 1.0, "top_k": 2, "top_p": 1.0, "ordinary_probabilities": [0.5714285675041357, 0.4285714324958642, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "lazy_counts": [299, 213, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]} +{"case": "invalid_logits", "row": "nan", "temperature": 0.0, "token": 0} +{"case": "invalid_logits", "row": "nan", "temperature": 0.6, "token": 0} +{"case": "invalid_logits", "row": "nan", "temperature": 1.0, "token": 0} +{"case": "invalid_logits_lazy", "row": "nan", "token": 19} +{"case": "invalid_logits", "row": "positive_inf", "temperature": 0.0, "token": 0} +{"case": "invalid_logits", "row": "positive_inf", "temperature": 0.6, "token": 0} +{"case": "invalid_logits", "row": "positive_inf", "temperature": 1.0, "token": 0} +{"case": "invalid_logits_lazy", "row": "positive_inf", "token": 0} +{"case": "invalid_logits", "row": "all_negative_inf", "temperature": 0.0, "token": 0} +{"case": "invalid_logits", "row": "all_negative_inf", "temperature": 0.6, "token": 0} +{"case": "invalid_logits", "row": "all_negative_inf", "temperature": 1.0, "token": 0} +{"case": "invalid_logits_lazy", "row": "all_negative_inf", "token": 16} +{"case": "invalid_logits", "row": "valid_mask", "temperature": 0.0, "token": 1} +{"case": "invalid_logits", "row": "valid_mask", "temperature": 0.6, "token": 1} +{"case": "invalid_logits", "row": "valid_mask", "temperature": 1.0, "token": 1} +{"case": "invalid_logits_lazy", "row": "valid_mask", "token": 3} +{"case": "tokenizer", "text": "भारत एक विशाल देश है जिसकी संस्कृति बहुत पुरानी है।", "ids": [149755, 148504, 76519, 171549, 148959, 211766, 151224, 148799, 151846, 81399, 148978, 148861, 152762, 62516, 42201, 65855, 193581, 28640, 62516, 204215, 37565, 148791, 90703, 70184, 76519, 80752, 151859, 149439, 42201, 81399, 148978, 154845]} +{"case": "tokenizer", "text": "ประเทศไทยมีวัฒนธรรมที่เก่าแก่และสวยงามมาก", "ids": [156150, 25443, 27399, 35982, 22547, 149374, 19571, 150327, 34694, 45596, 148385, 62551, 149530, 18064, 149119, 162566, 149196]} +{"case": "tokenizer", "text": "إِنَّ اللُّغَةَ الْعَرَبِيَّةَ جَمِيلَةٌ", "ids": [88792, 150708, 148515, 148784, 159613, 78958, 151189, 27004, 16660, 187581, 153458, 150765, 151003, 148515, 25046, 27004, 79333, 152397, 216808, 151189, 154876]} +{"case": "tokenizer", "text": "The quick brown fox jumps over the lazy dog near the river bank.", "ids": [760, 3841, 13477, 37550, 33075, 888, 279, 15217, 5388, 3043, 279, 14367, 5883, 13]} +{"case": "tokenizer", "text": "def foo(x):\n return x + 1\n", "ids": [727, 14785, 2007, 1590, 198, 262, 460, 830, 478, 220, 16, 198]} +{"case": "tokenizer", "text": "中国的文化历史悠久。", "ids": [108733, 96655, 125188, 1710]} +{"case": "tokenizer", "text": "a🙂b ©c ∑d —e …f €g ™h\n", "ids": [64, 169171, 65, 7013, 66, 11654, 239, 67, 1892, 68, 4444, 69, 12612, 70, 27717, 95, 71, 198]} +{"case": "tokenizer", "text": "x́y १२३ ١٢٣ ①Ⅷ²‍Z", "ids": [87, 52033, 88, 220, 12256, 100, 12256, 101, 12256, 102, 220, 149, 94, 149, 95, 149, 96, 220, 46701, 254, 68086, 100, 28495, 373, 235, 57]} +{"case": "tokenizer", "text": " ①", "ids": [220, 46701, 254]} +{"case": "tokenizer", "text": " Ⅷ", "ids": [220, 68086, 100]} +{"case": "tokenizer", "text": " ²", "ids": [220, 28495]} +{"case": "tokenizer", "text": " 𝟘", "ids": [220, 54362, 253, 246]} +{"case": "bf16_verify_gate", "available": false} diff --git a/docs/issue103-analysis/receipts-sha256.json b/docs/issue103-analysis/receipts-sha256.json new file mode 100644 index 0000000..1536ce9 --- /dev/null +++ b/docs/issue103-analysis/receipts-sha256.json @@ -0,0 +1,25 @@ +{ + "README.md": "7a276ada9beae8c3414cf7ac0db31a90e4b2c3e3ac8a8fefbb716892ca6fe472", + "bundle.log": "99fccf1baf416ea2b9d6f78f7b899afec1555cedd095f2b45ba2728f9b2d3735", + "clippy.log": "550a0c304401987d5fbac9706612fc85068a8b8f1cd3181c78544bd5c7263fb7", + "environment.json": "5c1821767e5b19d2dd14c20b9bf48081b77e729f1a7209a31fe306b42f100bf2", + "issues.json": "713e3d93bf0745d364c25d0371c6d3d835cf819a99f1430b8ade11996ad35387", + "latest-oracle.jsonl": "206389fec466bb3b4bd02d7941ff20033aad1900e0cc153f40822603c47186cb", + "mtp-binding.json": "858fccff52c0f34fb8d50ffad7d38aba692145cc186cf549c2341bef76ec90dc", + "native-controls.log": "e2782384870dc0f7972d1577ca4f9fcb4815edfcd60f09e49663480cd0b3a05c", + "native-eval-controls.log": "f28db6954744b586fbbb4635c997d5b0f71f6dc34addf678638cc134f058b8d4", + "native-final.log": "e7c30c3bec244e6070754bc72ea1f0cb957bff175f0ca9010a55bb7feaf29933", + "native-gather.jsonl": "10a31422dae0cc50f343cf9e9edfefaac8aea541ec9d5b0366d7fdc3c8a04403", + "native-model.jsonl": "23331f647e4ee57c8ff4c2b0d56d40cc59839750d7f07417378989e378c729a3", + "native-numeric.jsonl": "b1423ce71ac02ada97fb6ca14920d40340800f5c468380b8714172c0e4881cde", + "native-sampling-final.log": "ef4be1ff5e32da77116db2cb62f869a949b67e43376f4da408ac3338c4f803e4", + "native-sampling.jsonl": "2e28cc18c0696763705bbb8506c684cb4e17433de7d33d48f78194527e054e5e", + "native-tokenizer-final.log": "45fcf944c728118abfa5a0cb096dc4e3bd84b5ce8250f11d06f59e9ade845b4f", + "native-tokenizer.jsonl": "24b0d7949985b6ac0fe9cebba6dce2b12ed4db0e970ef1b256a770edfbf4c564", + "native-vision.jsonl": "ec100c776430eebb663862ef40023b619593355ea47f72a868e02abeccfe3f6e", + "old-oracle.jsonl": "04f1d4a03084aa7b0a0e50aeb4df173ed59f30c24eebcc201fe9fd9bb0979342", + "tests-serial.log": "b26dbcec36142d0d48973af863ff047f765efcd4fcac13ecfb78b9178ccb557d", + "tests.log": "8b0e4dec42fe0edb8c2596b0a3e63281b51165c15534ed27b6a9ae3320171efc", + "upstream-gates.log": "46cc6b5949fa5a20c21e8dc00dc17b895b4a5f88abe3ebc0cb13ae014333be89", + "upstream-tests.log": "7571d859c57232c4829163763046fa79517e133ecbbf6273414e031a33dafca6" +} diff --git a/docs/issue103-analysis/tests-serial.log b/docs/issue103-analysis/tests-serial.log new file mode 100644 index 0000000..668e2fd --- /dev/null +++ b/docs/issue103-analysis/tests-serial.log @@ -0,0 +1,559 @@ + Finished `test` profile [unoptimized + debuginfo] target(s) in 0.12s +warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6 +note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1` + Running unittests src/main.rs (target/debug/deps/ds4_server-5679c54d8848b708) + +running 517 tests +test a2ui::export::tests::static_svg_renderer_covers_the_complete_catalog ... ok +test a2ui::tests::all_basics_compose_into_one_surface ... ok +test a2ui::tests::basic_formatters_follow_catalog_examples ... ok +test a2ui::tests::catalog_accepts_pie_donut_and_heatmap_charts ... ok +test a2ui::tests::catalog_validation_and_actions_use_current_local_data ... ok +test a2ui::tests::catalog_validation_rejects_invalid_variants_and_function_arguments ... ok +test a2ui::tests::dismissal_boundaries_replay_history_and_a_fresh_active_epoch ... ok +test a2ui::tests::embedded_catalog_and_validator_cover_the_same_components ... ok +test a2ui::tests::latest_created_surface_is_the_active_surface ... ok +test a2ui::tests::lifecycle_updates_incrementally_and_json_pointer_upserts ... ok +test a2ui::tests::streaming_parser_only_accepts_complete_a2ui_jsonl_lines ... ok +test a2ui::tests::v1_action_and_function_round_trips_follow_current_schema ... ok +test agent::tests::ai_risk_response_requires_structured_risk_and_reason ... ok +test agent::tests::ai_shell_risk_assessment_receives_the_dev_brain_root ... ok +test agent::tests::anchored_edits_require_unique_head_and_tail ... ok +test agent::tests::denial_and_stop_cancel_commands_awaiting_approval ... ok +test agent::tests::dev_brain_paths_are_trusted_by_shell_risk_assessment ... ok +test agent::tests::dropping_session_tools_stops_live_jobs_and_removes_output ... kill: 95942: No such process +ok +test agent::tests::execution_failures_and_timeouts_have_distinct_codes ... ok +test agent::tests::generated_schemas_match_the_executable_tool_contracts ... ok +test agent::tests::glob_confinement_skips_symlinks_and_honors_cancellation ... ok +test agent::tests::glob_finds_all_regular_file_names_deterministically_without_reading_them ... ok +test agent::tests::local_file_and_bash_tools_execute_in_the_project ... ok +test agent::tests::model_adapters_preserve_primitives_and_report_recoverable_syntax_errors ... ok +test agent::tests::oversized_shell_output_is_bounded_and_retained_until_cleanup ... ok +test agent::tests::present_svg_executes_without_side_effects_and_maps_only_completed_cards ... ok +test agent::tests::present_svg_parsers_recover_exact_source_for_both_model_transports ... ok +test agent::tests::present_svg_validation_enforces_the_static_vector_boundary ... ok +test agent::tests::project_boundary_rejects_parent_paths ... ok +test agent::tests::prompts_and_parsers_expose_the_reference_tool_set ... ok +test agent::tests::ralph_loop_stops_on_cap_blocker_failure_and_cancellation ... ok +test agent::tests::ralph_report_schemas_are_child_only_and_parse_for_both_models ... ok +test agent::tests::ralph_report_semantics_and_handoff_bound_are_enforced ... ok +test agent::tests::ralph_round_contexts_are_unique_and_child_jobs_are_cleaned_up ... ok +test agent::tests::repeated_call_advisories_use_canonical_arguments_and_reset_per_user_turn ... ok +test agent::tests::risky_shell_commands_require_one_time_approval ... ok +test agent::tests::scripted_ralph_backend_gets_fresh_prompts_and_only_validated_handoffs ... ok +test agent::tests::shell_commands_skip_user_startup_files ... ok +test agent::tests::shell_environment_is_loaded_once_from_login_and_interactive_startup_files ... ok +test agent::tests::shell_environment_probe_stops_blocked_startup_files ... ok +test agent::tests::standard_agent_skills_are_discovered_prompted_and_read_only ... ok +test agent::tests::standard_file_tools_can_maintain_only_managed_dev_brain_pages ... ok +test agent::tests::stored_tool_cards_restore_ai_approval_reasons_by_call ... ok +test agent::tests::successful_structured_file_tools_report_project_paths ... ok +test agent::tests::tool_output_keeps_terminal_lines_but_parameter_summaries_do_not ... ok +test agent::tests::validation_is_precise_and_prevents_side_effects_until_corrected ... ok +test agent::web::tests::browser_helpers_preserve_ds4_wire_values ... ok +test agent::web::tests::headless_browser_extracts_a_local_page ... ignored, requires an installed Chrome browser +test app::generation::tests::a2ui_auto_switch_waits_for_a_renderable_surface ... ok +test app::generation::tests::assistant_markdown_keeps_llm_tables ... ok +test app::generation::tests::attached_image_marker_is_model_only ... ok +test app::generation::tests::complete_tool_call_in_reasoning_gets_one_correction ... ok +test app::generation::tests::extension_context_deduplicates_session_start_and_rearms_after_compaction ... ok +test app::generation::tests::forced_compaction_requires_new_visible_chat ... ok +test app::generation::tests::last_compaction_selects_its_tail_without_hiding_history ... ok +test app::generation::tests::legacy_generation_rows_collapse_into_their_user_turn ... ok +test app::generation::tests::model_history_uses_persisted_content_without_changing_the_transcript ... ok +test app::generation::tests::plain_chat_uses_the_same_unmodified_prompt_in_ui_and_headless ... ok +test app::generation::tests::queued_guidance_is_one_reference_style_user_turn ... ok +test app::generation::tests::title_context_starts_at_the_first_user_and_ignores_system_rows ... ok +test app::generation::tests::turn_summary_accumulates_model_continuations ... ok +test app::git::tests::configured_git_diff_options_change_the_rendered_patch ... ok +test app::git::tests::libgit2_status_merges_staged_and_worktree_changes ... ok +test app::git::tests::origin_fetch_pull_and_push_use_the_current_branch ... ok +test app::git::tests::unified_diff_parser_aligns_replacements_for_split_view ... ok +test app::git::tests::unstage_works_before_the_first_commit ... ok +test app::preferences::tests::multiline_system_prompt_round_trips_through_the_editor ... ok +test app::preferences::tests::runtime_config_updates_after_lock_poisoning ... ok +test app::preferences::tests::selectors_rehydrate_their_generation_and_model_profiles ... ok +test app::projects::tests::discarding_a_session_removes_every_checkpoint_stage ... ok +test app::projects::tests::draft_titles_follow_the_stored_session_count ... ok +test app::projects::tests::git_state_lists_and_switches_local_branches ... ok +test app::projects::tests::session_states_are_exclusive_and_ordered ... ok +test app::projects::tests::sweeping_removes_checkpoints_of_sessions_that_no_longer_exist ... ok +test app::tests::assistant_stream_keeps_reasoning_separate_from_the_answer ... ok +test app::tests::background_activity_drives_polling_indicators_and_quit_confirmation ... ok +test app::tests::chat_export_formats_visible_turns_reasoning_and_bash_tools ... ok +test app::tests::chat_export_preserves_completed_svg_and_keeps_failures_generic ... ok +test app::tests::chat_html_export_embeds_loaded_a2ui_images_as_data_urls ... ok +test app::tests::chat_html_export_is_safe_standalone_and_renders_embedded_surfaces ... ok +test app::tests::chat_only_follows_updates_while_near_the_tail ... ok +test app::tests::ds4_gib_and_streaming_cache_inputs_are_typed ... ok +test app::tests::one_shot_titles_are_reduced_to_a_single_sidebar_line ... ok +test app::tests::owned_runtime_paths_stay_in_application_support ... ok +test app::tests::preferences_shortcut_and_dspark_support_are_explicit ... ok +test app::tests::source_does_not_depend_on_a_sibling_ds4_checkout ... ok +test app::tests::stored_session_always_selects_its_model ... ok +test app::tests::tab_scrolls_a_field_back_into_the_preferences_viewport ... ok +test app::view::a2ui::tests::renderer_input_helpers_cover_dates_numbers_choices_and_weights ... ok +test app::view::chat::tests::chat_divider_groups_token_counts_by_thousands ... ok +test app::view::chat::tests::composer_uses_shift_enter_for_line_breaks ... ok +test app::view::git::tests::split_diff_width_covers_the_longest_side_and_tabs ... ok +test app::view::stats::tests::current_session_summary_uses_persisted_chat_facts ... ok +test app::view::tests::action_buttons_share_geometry ... ok +test app::view::tests::context_pie_wedge_follows_the_fill_grade ... ok +test app::view::tests::download_measurements_are_readable ... ok +test app::view::tests::raised_surfaces_stay_dark ... ok +test compaction::tests::a_new_summary_replaces_the_previous_rebuild_summary ... ok +test compaction::tests::hard_trigger_and_retry_reserve_match_the_reference ... ok +test compaction::tests::soft_trigger_matches_reference_thresholds ... ok +test compaction::tests::summary_budget_keeps_answer_room_and_rejects_exhausted_contexts ... ok +test compaction::tests::summary_control_markup_is_private ... ok +test compaction::tests::tail_prefers_the_first_user_boundary_after_target ... ok +test config::tests::defaults_write_an_empty_file_and_reload_unchanged ... ok +test config::tests::deprecated_flash_config_migrates_and_persists_only_0731 ... ok +test config::tests::explicit_0731_dspark_opt_out_survives_the_default ... ok +test config::tests::invalid_files_are_reported_instead_of_reset ... ok +test config::tests::legacy_flat_model_settings_are_dropped_but_the_prompt_is_kept ... ok +test config::tests::model_profiles_expose_only_their_upstream_thinking_modes ... ok +test config::tests::only_changed_values_are_written_and_read_back ... ok +test config::tests::qwen_selection_and_native_defaults_survive_restart ... ok +test database::tests::a2ui_dismissal_persists_a_fresh_surface_boundary ... ok +test database::tests::chat_and_compaction_history_survive_reopen_and_session_deletion ... ok +test database::tests::context_work_reactivates_an_archived_session ... ok +test database::tests::inline_svg_tool_call_survives_session_switch_compaction_and_reopen ... ok +test database::tests::migrations_and_crud_work ... ok +test database::tests::session_model_migration_preserves_data_and_accepts_catalog ... ok +test database::tests::sessions_are_ordered_by_recent_use_within_sidebar_groups ... ok +test dev_brain::tests::parser_and_index_rebuild_ignore_links_in_code ... ok +test dev_brain::tests::path_confinement_and_unrelated_notes_are_preserved ... ok +test dev_brain::tests::skill_index_migration_does_not_overwrite_an_unrelated_root_note ... ok +test dev_brain::tests::skill_validation_and_default_guide_restore_are_strict ... ok +test dev_brain::tests::source_hash_drift_excludes_page_from_authoritative_queries ... ok +test dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree ... ok +test dev_brain::tests::validation_keeps_broken_links_editable_and_reports_provenance_errors ... ok +test dev_brain::tests::validation_names_every_drifted_source_and_requires_semantic_refresh ... ok +test dev_brain::tests::verified_skills_are_indexed_and_injected_on_demand ... ok +test dsml::tests::parses_reference_and_legacy_dsml_syntaxes ... ok +test dsml::tests::recovers_complete_invokes_before_an_incomplete_one ... ok +test dsml::tests::recovers_complete_invokes_without_the_outer_closing_tag ... ok +test engine::ds4_sampling::tests::heap_fallback_preserves_rng_and_signed_zero_ties ... ok +test engine::gguf::tests::accepts_exact_mxfp4_block_layout ... ok +test engine::gguf::tests::keeps_unknown_tensor_kinds_rejected ... ok +test engine::gguf::tests::maps_metadata_and_tensor_payload_without_copying_weights ... ok +test engine::gguf::tests::mxfp4_scalar_dot_matches_independent_values ... ok +test engine::gguf::tests::rejects_mxfp4_dimension_overflow ... ok +test engine::gguf::tests::rejects_mxfp4_rows_that_are_not_block_aligned ... ok +test engine::gguf::tests::rejects_truncated_mxfp4_payload ... ok +test engine::kvstore::tests::budget_evicts_the_least_valuable_unprotected_entry ... ok +test engine::kvstore::tests::longest_compatible_prefix_wins_and_hits_survive_reopen ... ok +test engine::kvstore::tests::opening_reclaims_checkpoints_the_index_cannot_reach ... ok +test engine::metal::glm::tests::dsa_indexer_schedule_matches_the_reference ... ok +test engine::metal::glm::tests::glm53_m5_text_performance_gate ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal +test engine::metal::glm::tests::glm53_m5_vision_performance_and_oracle_gate ... ignored, requires GLM 5.3 Q2, its vision sidecar, a fixed image, and Apple Metal +test engine::metal::glm::tests::glm53_reference_logits_replay_separates_sampling_from_execution ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal +test engine::metal::glm::tests::glm53_reference_prompt_tokens_match_shared_runtime ... ignored, installed GLM and DS4SERVER_GLM_REFERENCE_LOG from the standalone reference +test engine::metal::glm::tests::glm53_two_row_verifier_restores_recurrent_state ... ignored, installed GLM 5.3 and Metal; run under test-supervisor +test engine::metal::glm::tests::glm_byte_budget_reserves_prefill_before_dynamic_experts ... ok +test engine::metal::glm::tests::glm_mtp_preserves_target_tokens_and_drafts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal +test engine::metal::glm::tests::glm_prefill_schedule_matches_ds4_boundaries ... ok +test engine::metal::glm::tests::glm_resident_decode_flushes_periodically_without_a_final_or_ssd_flush ... ok +test engine::metal::glm::tests::repeated_glm_prompt_rewinds_one_token_for_logits ... ok +test engine::metal::glm::tests::resident_and_streamed_glm_match_ds4_decode_oracles ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal +test engine::metal::glm::tests::streamed_glm_uses_ds4_indexed_prefill_for_long_prompts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal +test engine::metal::gpu::qwen_mtplx_allocation_tests::backend_allocation_policy_is_explicit_inside_and_outside_qwen_scope ... ignored, requires Apple Metal; allocation only, no GPU dispatch or model +test engine::metal::gpu::qwen_mtplx_allocation_tests::dispatch_retains_allocation_without_changing_data_donation ... ignored, requires Apple Metal, no model weights +test engine::metal::gpu::qwen_mtplx_allocation_tests::empty_storage_preserves_data_identity_views_and_zero_allocator_usage ... ignored, requires Apple Metal context; empty descriptors, no GPU allocation +test engine::metal::gpu::qwen_mtplx_allocation_tests::mtplx_tensor_cpu_access_releases_native_storage ... ignored, requires Apple Metal; CPU buffer access only, no GPU dispatch or model +test engine::metal::markov::tests::installed_markov_worker_dispatch ... ignored, requires installed 0731 DSpark support; CPU dispatch diagnostic, not parity acceptance +test engine::metal::markov::tests::markov_workers_reuse_threads_and_buffers_with_ordered_ties ... ok +test engine::metal::power_throttle_keeps_ds4_ewma_and_full_power_bypass ... ok +test engine::metal::profile::tests::profile_reports_locality_and_lru_hits ... ok +test engine::metal::qwen::tests::affine_split_k_matches_reference_shapes_and_scratch_bound ... ok +test engine::metal::qwen::tests::dense_attention_crossover_is_phase_specific ... ok +test engine::metal::qwen::tests::qsa_selection_and_capacity_cover_the_full_native_window ... ok +test engine::metal::qwen::tests::qwen_affine_qmv_and_qmm_match_mlx_reference ... ignored, requires the pinned Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_affine_qmv_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_core_boundary_and_checkpoint_are_stable ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_fused_gdn_input_rows_match_separate_projections ... ignored, requires the pinned Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_gdn_conv_norm_rows_matches_mtplx ... ignored, requires Apple Metal +test engine::metal::qwen::tests::qwen_gdn_delta_rows_matches_mtplx ... ignored, requires Apple Metal +test engine::metal::qwen::tests::qwen_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_lazy_target_probabilities_match_batched_sampling ... ignored, requires the pinned Qwen metadata and Apple Metal; no model inference +test engine::metal::qwen::tests::qwen_m5_performance_gate ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_metal_primitives_match_reference_vectors ... ignored, requires Apple Metal +test engine::metal::qwen::tests::qwen_mtp_matches_target_and_restores_its_cache ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_ple_gather_copies_only_the_requested_row ... ok +test engine::metal::qwen::tests::qwen_ple_hash_contract_matches_golden_boundaries ... ok +test engine::metal::qwen::tests::qwen_qsa_dense_mask_accepts_unsorted_blocks ... ignored, requires Apple Metal +test engine::metal::qwen::tests::qwen_qsa_full_context_preallocation_succeeds ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_qsa_logical_depth_oracles_cover_prefill_decode_and_state ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_qsa_real_layer_executes_the_first_sparse_token ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_qsa_selection_and_attention_remain_bounded_at_native_depths ... ignored, requires Apple Metal and allocates the full QSA score window +test engine::metal::qwen::tests::qwen_reference_layer_trace ... ignored, diagnostic trace against the pinned MLX-LM reference +test engine::metal::qwen::tests::qwen_reference_sdpa_trace ... ignored, diagnostic comparison with captured MTPLX SDPA inputs +test engine::metal::qwen::tests::qwen_reference_stateful_trace ... ignored, diagnostic trace against the pinned MLX-LM reference +test engine::metal::qwen::tests::qwen_reference_verify_trace ... ignored, diagnostic trace against the pinned MTPLX verifier +test engine::metal::qwen::tests::qwen_sorted_routed_prefill_matches_grouped_output ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_stage_counters_preserve_async_batches ... ignored, requires pinned Qwen embeddings and Apple Metal; stage profiler opt-in +test engine::metal::qwen::tests::qwen_target_submission_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_verify_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen_mtplx::allocator::mtplx_allocations_preserve_logical_bounds_with_reference_rounding ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::allocator::mtplx_allocator_cache_matches_live_reference_trace ... ok +test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_heap_limits_and_gpu_lifetime ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_replays_reference_allocations ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::ar::mtplx_ar_driver_matches_reference ... ignored, requires Apple Metal; unchanged MTPLX generate_ar orchestration oracle +test engine::metal::qwen_mtplx::array::mtplx_array_detach_clears_every_siblings_inputs_and_primitive ... ok +test engine::metal::qwen_mtplx::array::mtplx_array_layout_preserves_dtype_shape_stride_and_materialization_flags ... ok +test engine::metal::qwen_mtplx::array::mtplx_array_materialization_keeps_geometry_and_rejects_short_storage ... ignored, requires Apple Metal; typed materialized descriptors without model weights +test engine::metal::qwen_mtplx::array::mtplx_array_releases_sibling_cycles_and_deep_input_graphs ... ok +test engine::metal::qwen_mtplx::array::mtplx_dtype_promotion_matches_all_reference_pairs ... ok +test engine::metal::qwen_mtplx::attention::mtplx_qsa_dense_mask_graph_matches_reference ... ignored, requires Apple Metal; original MTPLX block mask including sentinel and tails +test engine::metal::qwen_mtplx::attention::mtplx_qsa_eager_array_graph_matches_reference ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::attention::mtplx_qsa_prefill_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX tiled prefill gather +test engine::metal::qwen_mtplx::attention::mtplx_qsa_rows_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX QSA rows-gather attention +test engine::metal::qwen_mtplx::attention::mtplx_sdpa_fallback_graph_matches_reference ... ignored, requires Apple M5 Max; actual Qwen SDPA fallback reference +test engine::metal::qwen_mtplx::binary::mtplx_graph_binary_preserves_reference_branches ... ignored, requires Apple Metal; binary graph broadcast/layout and donation branches +test engine::metal::qwen_mtplx::binding::mtplx_installed_module_binding_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS and SSD policy; only small norms and four table rows evaluated +test engine::metal::qwen_mtplx::cache::mtplx_kv_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX KVCache growth, trim and state receipts +test engine::metal::qwen_mtplx::cache::mtplx_qsa_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX positional cache, mirror and held state aliases +test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_scalar_bits_and_custom_kernel_identity ... ignored, requires Metal buffers and external supervisor; scalar ownership only, no GPU dispatch +test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_simplify_preserves_reference_parent_rules ... ok +test engine::metal::qwen_mtplx::compiled::mtplx_replay_replaces_descriptors_without_rebuilding_primitives ... ok +test engine::metal::qwen_mtplx::context_copy::prompt_copy_match_and_backoff_preserve_reference_policy ... ok +test engine::metal::qwen_mtplx::decoder::mtplx_decoder_connected_attention_and_moe_advance_caches ... ignored, requires Apple Metal and external test-supervisor; complete decoder/cache smoke +test engine::metal::qwen_mtplx::decoder::mtplx_decoder_residual_preserves_graph_order ... ok +test engine::metal::qwen_mtplx::dense::mtplx_dense_attention_batches_match_reference ... ignored, requires Apple M5 Max; actual BF16 batched matmul reference +test engine::metal::qwen_mtplx::dense::mtplx_dense_float_graph_matches_reference ... ignored, requires Apple M5 Max; floating Matmul factory and original dispatch +test engine::metal::qwen_mtplx::dense::mtplx_qsa_score_array_graph_matches_reference ... ignored, requires Apple M5 Max; canonical QSA F32 graph with the original TF32 setting +test engine::metal::qwen_mtplx::encoder::mtplx_empty_bindings_preserve_nil_dependencies_without_dummy_storage ... ignored, requires Apple Metal; empty original-kernel grids and nil bindings +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_barrier_epochs_and_commit_thresholds ... ok +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_commits_after_primitives_and_balances_completion ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_scope_routes_operators_and_allocator ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_shared_events_and_error_completion_order ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_untracked_dependencies_across_encoders_and_commands ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_drains_failure_preserves_cause_and_retries_only_failed_nodes ... ignored, requires Apple Metal; supervised evaluator failure and retry +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_releases_intermediates_and_preserves_descriptor_donation ... ignored, requires Apple Metal; graph ownership and last-consumer release +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_retains_tracer_topology_without_input_donation ... ignored, requires Apple Metal; evaluator tracer donation and retained topology +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_runs_cpu_gpu_siblings_and_async_continuations ... ignored, requires Apple Metal; tiny supervised CPU/GPU evaluator graph +test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_bfs_width_and_sibling_choice ... ok +test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_scheduled_detach_and_tracer_errors ... ok +test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_tracks_stream_fences_per_leaf_and_cross_device_union ... ok +test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ar_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_AR_REFERENCE required +test engine::metal::qwen_mtplx::execution::mtplx_full_installed_mtp_prefill_matches_reference ... ignored, full installed MTP model; external guard and DS4_QWEN38_MTP_PREFILL_REFERENCE required +test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ping_forward_matches_reference ... ignored, full installed model; requires external memory guard and DS4_QWEN38_FORWARD_REFERENCE +test engine::metal::qwen_mtplx::execution::qwen_power_throttle_drains_before_sleep_and_bypasses_at_full_power ... ignored, requires Apple Metal; run under test-supervisor +test engine::metal::qwen_mtplx::fused::mtplx_compile_fusion_preserves_parent_and_output_boundaries ... ok +test engine::metal::qwen_mtplx::fused::mtplx_compiled_constant_spelling ... ok +test engine::metal::qwen_mtplx::fused::mtplx_compiled_emitter_matches_all_pinned_silu_bodies ... ok +test engine::metal::qwen_mtplx::fused::mtplx_compiled_fusion_gpu_matches_reference ... ignored, requires Metal and external supervisor; generated fusion vs pinned SiLU plus multi-output/scalar dispatch +test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_forward_matches_model_reference ... ignored, requires Apple Metal; actual MTPLX GDN forward and cache receipts +test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_prefill_and_verify_match_model_reference ... ignored, requires Apple Metal; complete GDN Conv1d prefill and fused verify graphs +test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_staged_primitives_match_reference ... ignored, requires Apple Metal; 80 actual delta-update receipts and six conv-normalization kernels +test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_step_preserves_views_and_siblings ... ignored, requires Apple Metal; GDN custom primitive strided copies and sibling roots +test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_event_snapshots_preserve_values_streams_and_shared_errors ... ignored, requires Apple Metal events, no model weights +test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fence_coherence_preserves_empty_and_partial_word_spans ... ignored, requires Apple Metal3, model-free empty coherent dispatch +test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fences_transfer_cpu_gpu_gpu_cpu_in_both_modes ... ignored, requires Apple Metal3, tiny model-free coherent fence kernels +test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_forward_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX GatedResidual reference receipts +test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_pack_and_reads_match_reference ... ignored, requires Apple Metal; original hyper primitives and runtime pack +test engine::metal::qwen_mtplx::indexing::mtplx_graph_indexing_preserves_layouts_and_offsets ... ignored, requires Apple Metal; Select layout branches, GatherAxis views and empty arguments +test engine::metal::qwen_mtplx::indexing::mtplx_scatter_axis_graph_matches_reference ... ignored, requires Apple Metal; pinned Bool ScatterAxis layout and broadcast paths +test engine::metal::qwen_mtplx::issue103_analysis::issue103_mtp_final_receipts ... ignored, #103 diagnostic; full pinned model, SSD PLE and external memory supervisor +test engine::metal::qwen_mtplx::issue103_analysis::issue103_numeric_receipts ... ignored, #103 diagnostic; Metal required, eager versus compiled numeric operands +test engine::metal::qwen_mtplx::issue103_analysis::issue103_sampling_receipts ... ignored, #103 diagnostic; Metal required, pins currently broken sampling law and invalid rows +test engine::metal::qwen_mtplx::issue103_analysis::issue103_sorted_gather_receipts ... ignored, #103 diagnostic; M5 and pinned runtime shader required; initialized sentinels +test engine::metal::qwen_mtplx::issue103_analysis::issue103_tokenizer_receipts ... ignored, #103 diagnostic; installed tokenizer and latest oracle JSONL; no GPU +test engine::metal::qwen_mtplx::issue103_analysis::issue103_vision_position_receipts ... ignored, #103 diagnostic; small active attention module on Metal; mixed position boundary +test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_dequantize_matches_reference ... ignored, requires Apple Metal; original table dequantize dtype/group/bit/layout receipts +test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_quantize_matches_reference ... ignored, requires Apple Metal; original affine quantize dtype/group/bit/layout receipts +test engine::metal::qwen_mtplx::linear::mtplx_graph_gather_qmm_matches_reference ... ignored, requires Apple Metal; original GatherQMM array layouts and expert routing +test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_batched_identity ... ignored, requires Apple Metal and external supervisor; strided/broadcast batched affine dispatch +test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_matches_reference ... ignored, requires Apple M5 Max Metal; 78 pinned affine fixtures through lazy graph +test engine::metal::qwen_mtplx::load::mtplx_safetensors_load_laziness_boundaries_and_lifetime ... ignored, requires Apple Metal; bounded CPU Load and dependent GPU graph +test engine::metal::qwen_mtplx::mlp::mtplx_graph_shared_mlp_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX shared MLP reference receipts +test engine::metal::qwen_mtplx::mlp::mtplx_graph_split_and_compiled_silu_layouts ... ignored, requires Apple Metal; Split sibling ownership and compiled SiLU layouts +test engine::metal::qwen_mtplx::model::mtplx_model_compile_policy_and_run_boundaries ... ok +test engine::metal::qwen_mtplx::moe::mtplx_graph_sparse_moe_matches_reference ... ignored, requires Apple M5 Max; full MoE Array graph, no model weights; use test-supervisor +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_cpu_rows_and_raw_history_match_reference ... ok +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_embedding_matches_reference ... ignored, requires Apple Metal; full resident NGram graph and lazy decode history +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_gpu_rows_and_history_match_reference ... ignored, requires Apple Metal; original lazy NGram GPU row graph, no table +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_resident_matches_reference ... ignored, requires Apple Metal; original NGramTable resident gather/dequantize receipts +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_table_priority ... ignored, requires Apple Metal; NGram table routing and missing-sidecar behavior +test engine::metal::qwen_mtplx::ngram_sidecar::mtplx_ngram_sidecar_matches_reference ... ignored, requires Apple Metal; actual file-backed MTPLX sidecar/LRU/dequantization receipts +test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_allclose_branches ... ignored, requires Apple Metal; stage verification special values and original reduction branches +test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_matches_reference ... ignored, requires Apple Metal; original complete stage lifecycle and file-backed tables +test engine::metal::qwen_mtplx::normalization::mtplx_graph_norm_modules_match_reference ... ignored, requires Apple Metal; connected Qwen normalization graph golden fixtures +test engine::metal::qwen_mtplx::normalization::mtplx_graph_rms_norm_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph RMSNorm donation, copy, broadcast and looped kernels +test engine::metal::qwen_mtplx::normalization::mtplx_logsumexp_graph_matches_reference ... ignored, requires Apple Metal; original sampler LogSumExp branches +test engine::metal::qwen_mtplx::normalization::mtplx_softmax_graph_storage_matches_reference ... ignored, requires Apple Metal; original Softmax type and storage branches +test engine::metal::qwen_mtplx::ops::mtplx_concatenate_preserves_general_general_copy_semantics ... ignored, requires Apple Metal; Concatenate promotion, offsets, strided writers and validation +test engine::metal::qwen_mtplx::ops::mtplx_owned_operations_build_and_execute_qsa_layout_graph ... ignored, requires Apple Metal; self-contained lazy QSA-layout recipe, no callback map +test engine::metal::qwen_mtplx::ops::mtplx_slice_and_contiguous_preserve_cache_window_semantics ... ignored, requires Apple Metal; normalized Slice and Contiguous cache-window graph +test engine::metal::qwen_mtplx::owner::qwen_mtp_progress_publishes_without_turn_completion_or_double_counting ... ok +test engine::metal::qwen_mtplx::owner::qwen_product_generator_chat_matches_reference ... ignored, full installed model through product Generator; external supervisor and chat reference required +test engine::metal::qwen_mtplx::owner::qwen_vision_chat_checkpoint_preserves_image_identity ... ignored, requires installed Qwen, Metal, and DS4_QWEN_VISION_IMAGE; run under test-supervisor +test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_connected_matches_reference ... ignored, requires Apple M5 Max; complete original PLELayer with real lookup +test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_projection_and_cache_match_reference ... ignored, requires Apple M5 Max; actual MTPLX PLELayer after embedding lookup +test engine::metal::qwen_mtplx::policy::qwen_server_policy_matches_original_functions ... ignored, requires DS4_QWEN38_POLICY_REFERENCE from pinned MTPLX; CPU-only +test engine::metal::qwen_mtplx::prefill::mtplx_mtp_prefill_driver_matches_reference ... ignored, requires Apple Metal; original streaming MTP history prefill +test engine::metal::qwen_mtplx::prefill::mtplx_prefill_driver_matches_reference ... ignored, requires Apple Metal; original _prefill control flow and raw cache roots +test engine::metal::qwen_mtplx::qsa_attention::mtplx_qsa_attention_module_matches_reference ... ignored, requires Metal and unchanged whole-Attention receipts; no model download +test engine::metal::qwen_mtplx::qsa_compile::mtplx_compiled_indexer_array_replay_matches_reference ... ignored, requires Metal and original compiled indexer receipts; no model weights +test engine::metal::qwen_mtplx::qsa_core::mtplx_qsa_core_host_matches_reference ... ignored, requires Metal and original host-core receipts; no model weights +test engine::metal::qwen_mtplx::qsa_graph::qsa_cache_donation_requires_descriptor_data_and_allocation_contract ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::qsa_graph::qsa_completion_owns_buffers_until_gpu_finishes ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::qsa_graph::qsa_evaluator_retains_inputs_and_siblings_not_primary_data ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::qsa_graph::qsa_schedule_preserves_the_selected_sibling ... ok +test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_canonical_indexer_routes_match_reference ... ignored, requires Metal and original ongoing indexer routing receipts; no weights +test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_compiled_commit_matches_reference ... ignored, requires Metal and original indexer lifecycle receipts; no model weights +test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_prefix_matches_reference ... ignored, requires Apple Metal, original stateful QSAIndexer receipts, no model weights +test engine::metal::qwen_mtplx::qsa_prefill::mtplx_qsa_prefill_graph_matches_reference ... ignored, requires Metal and original two-stage prefill receipts, no weights +test engine::metal::qwen_mtplx::qsa_prepare::mtplx_qsa_prepare_graph_matches_reference ... ignored, requires Apple Metal; parameterized original QSA preparation and traced positions +test engine::metal::qwen_mtplx::qsa_select::mtplx_qsa_fused_graph_matches_reference ... ignored, requires Apple Metal and original QSA selector receipts; no model weights +test engine::metal::qwen_mtplx::random::mtplx_sampler_prefix_matches_reference ... ignored, requires Apple Metal; original target-prefix sampler and default RNG +test engine::metal::qwen_mtplx::random::mtplx_sampler_resident_matches_reference ... ignored, requires Apple Metal; original MTPLX resident sampling and keyed RNG +test engine::metal::qwen_mtplx::reduce::mtplx_sampler_argmax_matches_reference ... ignored, requires Apple Metal; original MTPLX greedy ArgReduce and storage +test engine::metal::qwen_mtplx::reduce::mtplx_sum_graph_matches_reference ... ignored, requires Apple Metal; original shared sum dispatch and QSA mean +test engine::metal::qwen_mtplx::request::qwen_request_progress_uses_ui_frontiers_and_decode_timer ... ok +test engine::metal::qwen_mtplx::request::qwen_request_settings_override_backend_policy ... ok +test engine::metal::qwen_mtplx::residency::mtplx_residency_matches_reference_lifecycle_cases ... ignored, requires Apple Metal 3 / macOS 15, no model weights +test engine::metal::qwen_mtplx::rope::mtplx_rope_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX RoPE, static YaRN and multi-axis receipts +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_batched_matches_reference ... ignored, requires Apple Metal; original MTPLX bound and serial batch distributions +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_distribution_and_rng_match_reference ... ok +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_entry_matches_reference ... ignored, requires Apple Metal; actual MTPLX generator sampling entry +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_fallback_matches_reference ... ignored, requires Apple Metal; original sparse fallback and host dense sampler +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_mixed_distributions_match_reference ... ok +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_penalties_match_reference ... ignored, requires Apple Metal; original GPU and CPU raw-logit penalties +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_selection_matches_reference ... ignored, requires Apple Metal; original MTPLX sampler selection, no model weights +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_serial_support_matches_reference ... ignored, requires Apple Metal; actual MTPLX serial candidate route, no weights +test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_dispatch_counts_tenth_and_completes_after_failed_work ... ok +test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_primitive_holds_data_and_temporaries_in_fifo_order ... ignored, requires Apple Metal shared storage, no GPU dispatch or model weights +test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_activity_preserves_reference_pressure_and_wait_rules ... ok +test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_cpu_events_propagate_errors_without_consuming_source ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_streams_preserve_fifo_errors_and_shutdown ... ok +test engine::metal::qwen_mtplx::sdpa::mtplx_sdpa_vector_matches_reference ... ignored, requires Apple M5 Max; actual pinned fused SDPA kernels +test engine::metal::qwen_mtplx::session_cache::mtplx_full_installed_chat_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_CHAT_REFERENCE required +test engine::metal::qwen_mtplx::session_cache::mtplx_session_boundary_policy ... ok +test engine::metal::qwen_mtplx::session_cache::mtplx_session_final_state_requires_capture_and_safe_cache ... ok +test engine::metal::qwen_mtplx::session_checkpoint::qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state ... ignored, model-free Metal checkpoint; external supervisor and DS4_QWEN38_CHECKPOINT_DIR required +test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_host_import_matches_reference ... ignored, model-free Metal imports; external supervisor and DS4_HOST_IMPORT_REFERENCE required +test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_matches_original_blobs_and_restore ... ignored, tiny GPU arrays; external supervisor and DS4_SESSION_CODEC_REFERENCE required +test engine::metal::qwen_mtplx::sort::mtplx_sort_graph_matches_reference ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::sparse_attention::mtplx_sparse_attention_graph_matches_reference ... ignored, requires Apple M5; actual MTPLX sparse Decode and TensorOps Prefill consumers +test engine::metal::qwen_mtplx::speculative::mtplx_full_installed_mtp_generation_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_MTP_GENERATION_REFERENCE required +test engine::metal::qwen_mtplx::stream::mtplx_stream_registry_preserves_thread_defaults_templates_and_clear ... ok +test engine::metal::qwen_mtplx::stream::mtplx_stream_routes_queues_events_and_finalizes_all_under_pressure ... ignored, requires Apple Metal, tiny model-free two-queue copy +test engine::metal::qwen_mtplx::submission::mtplx_all_native_command_commits_pass_the_submission_hook ... ok +test engine::metal::qwen_mtplx::submission::mtplx_submission_attaches_at_flush_eval_finish_and_drop ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::unary::mtplx_graph_unary_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph unary layout, donation and kernel boundaries +test engine::metal::qwen_mtplx::views::mtplx_dtype_conversion_and_views_preserve_values_layout_and_donation ... ignored, requires Apple Metal; original conversion/view kernels and donation in lazy eval +test engine::metal::qwen_mtplx::views::mtplx_dynamic_slice_graph_matches_reference ... ignored, requires Metal and original dynamic slice receipts; no model weights +test engine::metal::qwen_mtplx::views::mtplx_reshape_shares_or_copies_in_the_connected_evaluator ... ignored, requires Apple Metal; original Reshape copy kernels and shared backing +test engine::metal::qwen_mtplx::views::mtplx_views_preserve_aliases_offsets_strides_and_gpu_output ... ignored, requires Apple Metal; connected zero-copy views and offset-aware original copy kernel +test engine::metal::qwen_mtplx::vision::qwen_vision_tower_matches_mtplx_image ... ignored, requires Metal and pinned Qwen vision artifacts/reference values +test engine::metal::qwen_mtplx::weights::mtplx_checkpoint_projection_fusions_match_reference ... ignored, requires Apple Metal; bounded lazy checkpoint Load and fusion +test engine::metal::qwen_mtplx::weights::mtplx_installed_checkpoint_fusion_metadata_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS; headers only, never evaluate weights +test engine::metal::qwen_mtplx_tests::mtplx_affine_selection_preserves_reference_boundaries ... ok +test engine::metal::qwen_mtplx_tests::mtplx_canonical_hyper_and_gdn_kernels_match_reference ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_canonical_source_bodies_preserve_pinned_hashes ... ok +test engine::metal::qwen_mtplx_tests::mtplx_connected_delta_update_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_gather_sort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_decode_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_forward_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_staged_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_norm_modules_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_router_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_sparse_moe_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_switch_glu_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_gate_up_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_cache_lifecycle_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_backings_and_dynamic_copies_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_routing_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_window_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_capacity_matches_reference ... ok +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_indexer_modes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_scalar_sources_match_reference ... ok +test engine::metal::qwen_mtplx_tests::mtplx_qsa_connected_cache_selection_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_eager_topk_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_f32_scores_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_preparation_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_routes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_producer_dispatch_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_workspace_matches_reference ... ok +test engine::metal::qwen_mtplx_tests::mtplx_qsa_selector_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_runtime_affine_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_runtime_argsort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_runtime_gather_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_runtime_shaders_are_pinned ... ok +test engine::metal::tests::batch_raw_span_includes_the_chunk_and_previous_window ... ok +test engine::metal::tests::compression_schedule_tracks_the_deepseek_model_shape ... ok +test engine::metal::tests::decode_feature_policy_keeps_device_generations_and_rollbacks_separate ... ok +test engine::metal::tests::decode_raw_span_tracks_the_logical_sliding_window ... ok +test engine::metal::tests::directional_steering_matches_the_ds4_token_oracle ... ignored, requires the installed Flash GGUF, DS4 steering fixture, and a Metal device +test engine::metal::tests::dspark_confidence_quantizes_q8_input_like_ds4 ... ok +test engine::metal::tests::dspark_markov_activation_matches_q8_rounding ... ok +test engine::metal::tests::dspark_matches_original_summary_cycles ... ignored, requires installed 0731/DSpark and DS4SERVER_DSPARK_REFERENCE cycle recording +test engine::metal::tests::dspark_runs_a_target_owned_greedy_cycle ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures +test engine::metal::tests::dspark_scheduler_matches_the_ds4_default_window ... ok +test engine::metal::tests::dspark_verifier_selects_only_the_committed_row ... ok +test engine::metal::tests::flash_0731_long_context_crosses_indexed_prefill_boundary ... ignored, requires a 0731 Flash GGUF and an Apple M5 device +test engine::metal::tests::flash_0731_m5_decode_performance_gate ... ignored, requires a 0731 Flash GGUF and an Apple M5 device +test engine::metal::tests::flash_0731_resident_and_ssd_streaming_choose_the_same_tokens ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device +test engine::metal::tests::flash_0731_runs_exact_sampled_dspark ... ignored, requires the installed 0731 target and checkpoint-specific DSpark GGUF fixtures +test engine::metal::tests::flash_0731_ssd_streaming_maps_and_seeds_batched_prefill_layers ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device +test engine::metal::tests::generated_hotlists_match_each_model_shape ... ok +test engine::metal::tests::half_conversion_handles_normal_and_subnormal_values ... ok +test engine::metal::tests::pro_q4_model_spans_remain_isolated ... ok +test engine::metal::tests::pro_resident_admission_includes_weights_context_and_scratch ... ok +test engine::metal::tests::pro_ssd_streaming_full_layer_and_selected_batch_agree ... ignored, requires the installed DeepSeek V4 Pro GGUF and Apple Metal +test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device +test engine::metal::tests::ssd_streaming_supports_dspark ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures +test engine::metal::tests::streaming_memory_plan_matches_ds4_graph_formulas ... ok +test engine::metal::vision::tests::glm53_resize_matches_reference_token_grids ... ok +test engine::qwen::tests::frozen_inventory_and_memory_categories_are_exact ... ok +test engine::qwen::tests::metadata_and_safetensors_boundaries_fail_closed ... ok +test engine::qwen::tests::pinned_artifact_set_loads_and_renders_goldens ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source +test engine::qwen::tests::pinned_qwen_metadata_matches_product_without_executor_state ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source +test engine::qwen_vision::qwen_image_cache_keys_distinguish_pixels_and_preserve_prior_text ... ok +test engine::qwen_vision::qwen_image_preprocessing_matches_pillow_mtplx ... ok +test engine::qwen_vision::qwen_vision_positions_and_resize_contract ... ok +test engine::sampling_tests::checkpoint_headers_identify_the_exact_model ... ok +test engine::sampling_tests::checkpoint_rebuilds_explain_compatibility_and_history_misses ... ok +test engine::sampling_tests::checkpoint_tag_accepts_an_unchanged_prefix_before_reminders ... ok +test engine::sampling_tests::checkpoint_tag_covers_the_canonical_chat_state ... ok +test engine::sampling_tests::ds4_sampler_matches_original_reference_tokens_and_rng ... ok +test engine::sampling_tests::ds4_sampler_real_logits_cpu_benchmark ... ignored, CPU-only; requires DS4_SAMPLER_RECORDING with the recorded GLM logits +test engine::sampling_tests::exact_delta_sampling_accepts_or_corrects_the_draft ... ok +test engine::sampling_tests::generated_text_drains_incomplete_bytes_before_phase_changes_and_finish ... ok +test engine::sampling_tests::generated_text_preserves_thinking_utf8_and_split_stops ... ok +test engine::sampling_tests::metal_executes_real_flash_token ... ignored, requires the 80 GiB Flash checkpoint and Apple Metal +test engine::sampling_tests::probability_ratio_speculation_preserves_the_target_distribution ... ok +test engine::sampling_tests::qwen_rng_matches_numpy_pcg64 ... ok +test engine::sampling_tests::qwen_sampling_uses_mtplx_global_nucleus_mass_before_top_k ... ok +test engine::sampling_tests::resumed_prefill_crossover_matches_each_ds4_graph ... ok +test engine::sampling_tests::sampling_probabilities_match_ds4_filter_order ... ok +test engine::sampling_tests::split_utf8_token_bytes_are_joined_before_decoding ... ok +test engine::sampling_tests::top_k_and_stream_stops_are_applied_before_output ... ok +test engine::sampling_tests::workspace_instruction_identity_participates_in_append_only_kv_keys ... ok +test engine::sampling_tests::zero_temperature_is_greedy ... ok +test engine::tokenizer::tests::character_helpers_are_total_at_every_byte_offset ... ok +test engine::tokenizer::tests::ds4_chat_matches_original_session_tokens ... ignored, CPU-only; requires DS4SERVER_CHAT_REFERENCE and its installed GGUF +test engine::tokenizer::tests::qwen_checkpoint_continuation_matches_full_conversation ... ignored, requires the installed Qwen tokenizer, but no GPU or model weights +test engine::tokenizer::tests::qwen_plain_chat_matches_mtplx_server_tokens ... ignored, requires local tokenizer and tools/qwen-chat-reference.py output; no inference +test engine::tokenizer::tests::reasoning_prefixes_match_each_model_family_contract ... ok +test engine::validation::tests::configured_glm53_main_fixture_passes_the_exact_layout ... ok +test engine::validation::tests::configured_glm53_vision_fixture_passes_the_exact_layout ... ok +test engine::validation::tests::installed_ds4_fixture_opens_and_renders_a_prompt ... ok +test engine::validation::tests::installed_dspark_fixture_passes_the_target_layout ... ok +test engine::validation::tests::mxfp4_is_limited_to_routed_experts ... ok +test extensions::tests::direct_hook_runner_isolated_environment_and_handles_failures ... ok +test extensions::tests::duplicate_skill_names_and_update_rollback_are_deterministic ... ok +test extensions::tests::hook_failures_are_bounded_actionable_and_non_shell ... ok +test extensions::tests::hook_output_is_bounded_typed_and_event_matched ... ok +test extensions::tests::lifecycle_matchers_preserve_declared_event_order ... ok +test extensions::tests::manifest_and_paths_are_validated_without_following_escapes ... ok +test extensions::tests::pinned_ponytail_runs_all_lifecycle_modes ... ignored, downloads the pinned Ponytail reference integration +test extensions::tests::registry_persists_enable_disable_errors_and_uninstall ... ok +test extensions::tests::symlink_and_variable_escapes_are_rejected ... ok +test instructions::tests::changes_removals_resume_and_compaction_are_append_only ... ok +test instructions::tests::discovery_is_global_to_local_deduplicated_and_confined ... ok +test instructions::tests::empty_and_repaired_baselines_do_not_repeat_prompt_content ... ok +test instructions::tests::frames_escape_control_tags_and_enforce_utf8_byte_limits ... ok +test instructions::tests::newly_crossed_scope_is_added_once_and_unavailable_content_is_retained ... ok +test metrics::tests::cache_report_splits_buckets_and_folds_index_files_into_their_checkpoint ... ok +test metrics::tests::decode_timer_excludes_later_work_and_resets_between_requests ... ok +test metrics::tests::prompt_timing_keeps_unmeasured_components_distinct_from_zero ... ok +test metrics::tests::removed_preview_metric_code_stays_decodable ... ok +test metrics::tests::snapshots_track_runtime_and_server_counters ... ok +test model::qwen::qwen_vision_is_an_independent_managed_artifact_set ... ok +test model::qwen::tests::embedded_manifest_is_the_frozen_complete_set ... ok +test model::qwen::tests::verification_rejects_hash_mismatch_and_honors_cancellation ... ok +test model::transfer::tests::cancellation_keeps_the_partial_file_for_the_next_run ... ok +test model::transfer::tests::catalog_and_checksum_verification_are_explicit ... ok +test model::transfer::tests::managed_artifact_inventory_and_delete_include_partial_files ... ok +test model::transfer::tests::qwen_set_verifies_reloads_and_removes_as_one_managed_artifact ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source +test model::transfer::tests::restart_resumes_at_the_existing_partial_byte ... ok +test model::transfer::tests::stalled_download_times_out_and_keeps_the_partial_file ... ok +test model::transfer::tests::verification_reports_bytes_read ... ok +test model_eval::tests::acceleration_receipt_reports_effective_sampling_contract ... ok +test model_eval::tests::acceleration_switch_uses_each_models_native_path ... ok +test model_eval::tests::canary_cross_phase_samples_keep_their_start_phase ... ok +test model_eval::tests::cpu_accounting_matches_getrusage_units ... ok +test model_eval::tests::evaluations_exclude_overlapping_workers_and_release_on_exit ... ok +test model_eval::tests::external_canary_observes_phases_and_finishes_on_eof ... ignored, Metal probe only; DS4SERVER_CANARY_BINARY must point to the current release binary +test model_eval::tests::external_canary_protocol_accepts_only_known_phases ... ok +test model_eval::tests::parses_safe_defaults_and_overrides ... ok +test model_eval::tests::rejects_unbounded_or_invalid_limits ... ok +test model_eval::tests::repeated_prompts_form_one_ordered_conversation ... ok +test model_eval::tests::warmup_timing_separates_loading_without_inventing_missing_durations ... ok +test model_eval::tests::worker_progress_events_advance_the_watchdog ... ok +test native_edit::tests::edit_history_supports_undo_and_redo ... ok +test native_edit::tests::paste_replays_a_complete_command_shortcut ... ok +test native_media::tests::media_player_only_accepts_remote_http_urls ... ok +test native_menu::tests::edit_items_are_custom_menu_events ... ok +test native_menu::tests::user_guide_uses_unmodified_f1 ... ok +test native_menu::tests::view_items_keep_the_selected_pane_checked ... ok +test runtime::tests::dropping_active_generation_cancels_it ... ok +test runtime::tests::operation_failures_use_the_matching_event ... ok +test runtime::tests::operation_tracking_is_consistent ... ok +test runtime::tests::runtime_panic_recovery_accepts_the_next_operation ... ok +test server::request::tests::reasoning_requests_preserve_each_models_exact_modes ... ok +test server::tests::canonical_tool_replay_preserves_argument_types ... ok +test server::tests::connection_slots_enforce_the_limit_and_release_on_drop ... ok +test server::tests::content_arrays_match_ds4_text_projection ... ok +test server::tests::context_limit_errors_use_protocol_standard_fields ... ok +test server::tests::cors_headers_are_opt_in_and_cover_preflight ... ok +test server::tests::dsml_tool_stream_projects_incremental_json_arguments ... ok +test server::tests::generated_dsml_becomes_openai_tool_calls ... ok +test server::tests::generated_tool_parser_accepts_ds4_recovery_syntaxes ... ok +test server::tests::model_metadata_uses_the_requested_alias_and_default_token_limit ... ok +test server::tests::protocol_tool_results_require_live_or_replayed_call_ids ... ok +test server::tests::request_deadline_stops_slow_clients ... ok +test server::tests::responses_parser_rejects_dropped_state_and_preserves_hosted_calls ... ok +test server::tests::sampled_tool_replay_survives_server_restart ... ok +test server::tests::streaming_errors_match_ds4_server_framing ... ok +test server::tests::streaming_prefill_keeps_the_client_connection_alive ... ok +test server::tests::tool_prompt_matches_ds4_server_text ... ok +test server::tests::tool_stream_projects_ds4_recovery_syntaxes ... ok +test settings::tests::captured_ds4_cli_options_are_uniquely_classified ... ok +test settings::tests::effective_settings_build_one_engine_and_turn_configuration ... ok +test settings::tests::effective_settings_preserve_unset_model_defaults ... ok +test settings::tests::execution_settings_keep_ds4_automatic_values_and_glm_rules ... ok +test settings::tests::kv_cache_gates_follow_the_ds4_store_rules ... ok +test settings::tests::reasoning_selection_is_exact_and_model_specific ... ok +test settings::tests::runtime_settings_preserve_automatic_values_and_cross_field_rules ... ok +test settings::tests::speculative_settings_match_acceleration_defaults_and_dependencies ... ok + +test result: ok. 307 passed; 0 failed; 210 ignored; 0 measured; 0 filtered out; finished in 15.64s + + Running unittests tools/qwen38-artifacts.rs (target/debug/deps/qwen38_artifacts-1288d410762a1a77) + +running 4 tests +test tests::affine_sample_digest_uses_packed_values_scales_and_biases ... ok +test tests::artifact_download_resumes_an_authenticated_partial ... ok +test tests::artifact_names_cannot_escape_the_output_directory ... ok +test tests::safetensors_validation_rejects_duplicate_names_and_invalid_ranges ... ok + +test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s + + Running unittests tools/test-supervisor.rs (target/debug/deps/test_supervisor-df80f419c8443a12) + +running 4 tests +test supervisor::tests::direct_worker_arguments_are_not_rust_test_arguments ... ok +test supervisor::tests::split_probe_records_do_not_reset_progress ... ok +test supervisor::tests::split_records_cannot_be_interleaved_with_watchdog_samples ... ok +test supervisor::tests::watchdog_stops_memory_and_silence_but_not_total_runtime ... ready{"event":"gpu_canary_sample","ok":true} +{"event":"gpu_canary_sample","ok":true} +{"event":"gpu_canary_sample","ok":true} +{"event":"gpu_canary_sample","ok":true} +{"event":"gpu_canary_sample","ok":true} +{"event":"gpu_canary_sample","ok":true} +ready +{"event":"gpu_canary_sample","ok":true} +{"event":"gpu_canary_sample","ok":true} +{"event":"gpu_canary_sample","ok":true} +{"event":"gpu_canary_sample","ok":true} +{"event":"gpu_canary_sample","ok":true} +progressprogressprogressprogressprogressprogressprogressprogressok + +test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.38s + diff --git a/docs/issue103-analysis/tests.log b/docs/issue103-analysis/tests.log new file mode 100644 index 0000000..1994a5c --- /dev/null +++ b/docs/issue103-analysis/tests.log @@ -0,0 +1,545 @@ + Finished `test` profile [unoptimized + debuginfo] target(s) in 0.20s +warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6 +note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1` + Running unittests src/main.rs (target/debug/deps/ds4_server-5679c54d8848b708) + +running 517 tests +test agent::tests::anchored_edits_require_unique_head_and_tail ... ok +test agent::tests::ai_shell_risk_assessment_receives_the_dev_brain_root ... ok +test agent::tests::ai_risk_response_requires_structured_risk_and_reason ... ok +test a2ui::tests::streaming_parser_only_accepts_complete_a2ui_jsonl_lines ... ok +test a2ui::tests::embedded_catalog_and_validator_cover_the_same_components ... ok +test a2ui::tests::basic_formatters_follow_catalog_examples ... ok +test agent::tests::dev_brain_paths_are_trusted_by_shell_risk_assessment ... ok +test a2ui::tests::catalog_validation_rejects_invalid_variants_and_function_arguments ... ok +test a2ui::tests::catalog_accepts_pie_donut_and_heatmap_charts ... ok +test a2ui::tests::dismissal_boundaries_replay_history_and_a_fresh_active_epoch ... ok +test a2ui::tests::lifecycle_updates_incrementally_and_json_pointer_upserts ... ok +test a2ui::tests::latest_created_surface_is_the_active_surface ... ok +test agent::tests::present_svg_parsers_recover_exact_source_for_both_model_transports ... ok +test agent::tests::ralph_loop_stops_on_cap_blocker_failure_and_cancellation ... ok +test a2ui::tests::catalog_validation_and_actions_use_current_local_data ... ok +test a2ui::tests::all_basics_compose_into_one_surface ... ok +test a2ui::tests::v1_action_and_function_round_trips_follow_current_schema ... ok +test agent::tests::ralph_report_semantics_and_handoff_bound_are_enforced ... ok +test agent::tests::risky_shell_commands_require_one_time_approval ... ok +test agent::tests::model_adapters_preserve_primitives_and_report_recoverable_syntax_errors ... ok +test agent::tests::generated_schemas_match_the_executable_tool_contracts ... ok +test agent::tests::scripted_ralph_backend_gets_fresh_prompts_and_only_validated_handoffs ... ok +test agent::tests::ralph_report_schemas_are_child_only_and_parse_for_both_models ... ok +test agent::tests::project_boundary_rejects_parent_paths ... ok +test agent::tests::repeated_call_advisories_use_canonical_arguments_and_reset_per_user_turn ... ok +test a2ui::export::tests::static_svg_renderer_covers_the_complete_catalog ... ok +test agent::tests::stored_tool_cards_restore_ai_approval_reasons_by_call ... ok +test agent::tests::prompts_and_parsers_expose_the_reference_tool_set ... ok +test agent::tests::present_svg_executes_without_side_effects_and_maps_only_completed_cards ... ok +test agent::tests::tool_output_keeps_terminal_lines_but_parameter_summaries_do_not ... ok +test agent::web::tests::headless_browser_extracts_a_local_page ... ignored, requires an installed Chrome browser +test agent::web::tests::browser_helpers_preserve_ds4_wire_values ... ok +test agent::tests::glob_confinement_skips_symlinks_and_honors_cancellation ... ok +test agent::tests::present_svg_validation_enforces_the_static_vector_boundary ... ok +test agent::tests::validation_is_precise_and_prevents_side_effects_until_corrected ... ok +test agent::tests::successful_structured_file_tools_report_project_paths ... ok +test app::generation::tests::attached_image_marker_is_model_only ... ok +test agent::tests::standard_agent_skills_are_discovered_prompted_and_read_only ... ok +test agent::tests::glob_finds_all_regular_file_names_deterministically_without_reading_them ... ok +test agent::tests::shell_commands_skip_user_startup_files ... ok +test app::generation::tests::plain_chat_uses_the_same_unmodified_prompt_in_ui_and_headless ... ok +test app::generation::tests::queued_guidance_is_one_reference_style_user_turn ... ok +test app::generation::tests::title_context_starts_at_the_first_user_and_ignores_system_rows ... ok +test app::generation::tests::turn_summary_accumulates_model_continuations ... ok +test agent::tests::standard_file_tools_can_maintain_only_managed_dev_brain_pages ... ok +test agent::tests::shell_environment_is_loaded_once_from_login_and_interactive_startup_files ... ok +test agent::tests::ralph_round_contexts_are_unique_and_child_jobs_are_cleaned_up ... ok +test app::git::tests::unified_diff_parser_aligns_replacements_for_split_view ... ok +test agent::tests::denial_and_stop_cancel_commands_awaiting_approval ... ok +test app::git::tests::unstage_works_before_the_first_commit ... ok +test app::preferences::tests::runtime_config_updates_after_lock_poisoning ... ok +test app::git::tests::configured_git_diff_options_change_the_rendered_patch ... ok +test app::projects::tests::discarding_a_session_removes_every_checkpoint_stage ... ok +test app::projects::tests::draft_titles_follow_the_stored_session_count ... ok +test app::git::tests::libgit2_status_merges_staged_and_worktree_changes ... ok +test app::projects::tests::session_states_are_exclusive_and_ordered ... ok +test app::projects::tests::sweeping_removes_checkpoints_of_sessions_that_no_longer_exist ... ok +test app::projects::tests::git_state_lists_and_switches_local_branches ... ok +test app::tests::background_activity_drives_polling_indicators_and_quit_confirmation ... ok +test agent::tests::shell_environment_probe_stops_blocked_startup_files ... ok +test app::git::tests::origin_fetch_pull_and_push_use_the_current_branch ... ok +test app::tests::chat_html_export_embeds_loaded_a2ui_images_as_data_urls ... ok +test agent::tests::oversized_shell_output_is_bounded_and_retained_until_cleanup ... ok +test app::tests::chat_only_follows_updates_while_near_the_tail ... ok +test app::tests::ds4_gib_and_streaming_cache_inputs_are_typed ... ok +test app::tests::one_shot_titles_are_reduced_to_a_single_sidebar_line ... ok +test app::tests::owned_runtime_paths_stay_in_application_support ... ok +test app::tests::preferences_shortcut_and_dspark_support_are_explicit ... ok +test app::tests::source_does_not_depend_on_a_sibling_ds4_checkout ... ok +test app::tests::stored_session_always_selects_its_model ... ok +test app::tests::tab_scrolls_a_field_back_into_the_preferences_viewport ... ok +test app::view::a2ui::tests::renderer_input_helpers_cover_dates_numbers_choices_and_weights ... ok +test app::view::chat::tests::chat_divider_groups_token_counts_by_thousands ... ok +test app::view::chat::tests::composer_uses_shift_enter_for_line_breaks ... ok +test app::view::git::tests::split_diff_width_covers_the_longest_side_and_tabs ... ok +test app::tests::chat_export_preserves_completed_svg_and_keeps_failures_generic ... ok +test app::view::tests::action_buttons_share_geometry ... ok +test app::view::tests::context_pie_wedge_follows_the_fill_grade ... ok +test app::view::tests::download_measurements_are_readable ... ok +test app::view::tests::raised_surfaces_stay_dark ... ok +test compaction::tests::a_new_summary_replaces_the_previous_rebuild_summary ... ok +test compaction::tests::hard_trigger_and_retry_reserve_match_the_reference ... ok +test compaction::tests::soft_trigger_matches_reference_thresholds ... ok +test compaction::tests::summary_budget_keeps_answer_room_and_rejects_exhausted_contexts ... ok +test compaction::tests::summary_control_markup_is_private ... ok +test compaction::tests::tail_prefers_the_first_user_boundary_after_target ... ok +test config::tests::defaults_write_an_empty_file_and_reload_unchanged ... ok +test config::tests::deprecated_flash_config_migrates_and_persists_only_0731 ... ok +test config::tests::explicit_0731_dspark_opt_out_survives_the_default ... ok +test config::tests::invalid_files_are_reported_instead_of_reset ... ok +test config::tests::legacy_flat_model_settings_are_dropped_but_the_prompt_is_kept ... ok +test config::tests::model_profiles_expose_only_their_upstream_thinking_modes ... ok +test config::tests::only_changed_values_are_written_and_read_back ... ok +test app::preferences::tests::selectors_rehydrate_their_generation_and_model_profiles ... ok +test config::tests::qwen_selection_and_native_defaults_survive_restart ... ok +test app::generation::tests::model_history_uses_persisted_content_without_changing_the_transcript ... ok +test app::preferences::tests::multiline_system_prompt_round_trips_through_the_editor ... ok +test app::generation::tests::legacy_generation_rows_collapse_into_their_user_turn ... ok +test app::tests::assistant_stream_keeps_reasoning_separate_from_the_answer ... ok +test app::view::stats::tests::current_session_summary_uses_persisted_chat_facts ... ok +test app::generation::tests::last_compaction_selects_its_tail_without_hiding_history ... ok +test app::generation::tests::assistant_markdown_keeps_llm_tables ... ok +test app::tests::chat_export_formats_visible_turns_reasoning_and_bash_tools ... ok +test app::generation::tests::a2ui_auto_switch_waits_for_a_renderable_surface ... ok +test app::generation::tests::forced_compaction_requires_new_visible_chat ... ok +test app::generation::tests::extension_context_deduplicates_session_start_and_rearms_after_compaction ... ok +test app::generation::tests::complete_tool_call_in_reasoning_gets_one_correction ... ok +test app::tests::chat_html_export_is_safe_standalone_and_renders_embedded_surfaces ... ok +test dev_brain::tests::path_confinement_and_unrelated_notes_are_preserved ... ok +test dev_brain::tests::skill_index_migration_does_not_overwrite_an_unrelated_root_note ... ok +test dsml::tests::parses_reference_and_legacy_dsml_syntaxes ... ok +test dsml::tests::recovers_complete_invokes_before_an_incomplete_one ... ok +test dsml::tests::recovers_complete_invokes_without_the_outer_closing_tag ... ok +test engine::ds4_sampling::tests::heap_fallback_preserves_rng_and_signed_zero_ties ... ok +test engine::gguf::tests::accepts_exact_mxfp4_block_layout ... ok +test engine::gguf::tests::keeps_unknown_tensor_kinds_rejected ... ok +test engine::gguf::tests::maps_metadata_and_tensor_payload_without_copying_weights ... ok +test engine::gguf::tests::mxfp4_scalar_dot_matches_independent_values ... ok +test database::tests::context_work_reactivates_an_archived_session ... ok +test engine::gguf::tests::rejects_mxfp4_dimension_overflow ... ok +test engine::gguf::tests::rejects_mxfp4_rows_that_are_not_block_aligned ... ok +test engine::gguf::tests::rejects_truncated_mxfp4_payload ... ok +test database::tests::inline_svg_tool_call_survives_session_switch_compaction_and_reopen ... ok +test database::tests::a2ui_dismissal_persists_a_fresh_surface_boundary ... ok +test engine::metal::glm::tests::dsa_indexer_schedule_matches_the_reference ... ok +test engine::metal::glm::tests::glm53_m5_text_performance_gate ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal +test engine::metal::glm::tests::glm53_m5_vision_performance_and_oracle_gate ... ignored, requires GLM 5.3 Q2, its vision sidecar, a fixed image, and Apple Metal +test engine::metal::glm::tests::glm53_reference_logits_replay_separates_sampling_from_execution ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal +test engine::metal::glm::tests::glm53_reference_prompt_tokens_match_shared_runtime ... ignored, installed GLM and DS4SERVER_GLM_REFERENCE_LOG from the standalone reference +test engine::metal::glm::tests::glm53_two_row_verifier_restores_recurrent_state ... ignored, installed GLM 5.3 and Metal; run under test-supervisor +test database::tests::sessions_are_ordered_by_recent_use_within_sidebar_groups ... ok +test engine::metal::glm::tests::glm_byte_budget_reserves_prefill_before_dynamic_experts ... ok +test engine::metal::glm::tests::glm_mtp_preserves_target_tokens_and_drafts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal +test engine::metal::glm::tests::glm_prefill_schedule_matches_ds4_boundaries ... ok +test engine::metal::glm::tests::repeated_glm_prompt_rewinds_one_token_for_logits ... ok +test engine::metal::glm::tests::resident_and_streamed_glm_match_ds4_decode_oracles ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal +test engine::metal::glm::tests::streamed_glm_uses_ds4_indexed_prefill_for_long_prompts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal +test engine::metal::gpu::qwen_mtplx_allocation_tests::backend_allocation_policy_is_explicit_inside_and_outside_qwen_scope ... ignored, requires Apple Metal; allocation only, no GPU dispatch or model +test engine::metal::gpu::qwen_mtplx_allocation_tests::dispatch_retains_allocation_without_changing_data_donation ... ignored, requires Apple Metal, no model weights +test engine::metal::gpu::qwen_mtplx_allocation_tests::empty_storage_preserves_data_identity_views_and_zero_allocator_usage ... ignored, requires Apple Metal context; empty descriptors, no GPU allocation +test engine::metal::gpu::qwen_mtplx_allocation_tests::mtplx_tensor_cpu_access_releases_native_storage ... ignored, requires Apple Metal; CPU buffer access only, no GPU dispatch or model +test engine::metal::markov::tests::installed_markov_worker_dispatch ... ignored, requires installed 0731 DSpark support; CPU dispatch diagnostic, not parity acceptance +test engine::metal::glm::tests::glm_resident_decode_flushes_periodically_without_a_final_or_ssd_flush ... ok +test engine::metal::markov::tests::markov_workers_reuse_threads_and_buffers_with_ordered_ties ... ok +test engine::metal::power_throttle_keeps_ds4_ewma_and_full_power_bypass ... ok +test engine::metal::qwen::tests::affine_split_k_matches_reference_shapes_and_scratch_bound ... ok +test engine::metal::qwen::tests::dense_attention_crossover_is_phase_specific ... ok +test engine::metal::qwen::tests::qsa_selection_and_capacity_cover_the_full_native_window ... ok +test engine::metal::qwen::tests::qwen_affine_qmv_and_qmm_match_mlx_reference ... ignored, requires the pinned Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_affine_qmv_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_core_boundary_and_checkpoint_are_stable ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_fused_gdn_input_rows_match_separate_projections ... ignored, requires the pinned Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_gdn_conv_norm_rows_matches_mtplx ... ignored, requires Apple Metal +test engine::metal::qwen::tests::qwen_gdn_delta_rows_matches_mtplx ... ignored, requires Apple Metal +test engine::metal::qwen::tests::qwen_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_lazy_target_probabilities_match_batched_sampling ... ignored, requires the pinned Qwen metadata and Apple Metal; no model inference +test engine::metal::qwen::tests::qwen_m5_performance_gate ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_metal_primitives_match_reference_vectors ... ignored, requires Apple Metal +test engine::metal::qwen::tests::qwen_mtp_matches_target_and_restores_its_cache ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_ple_gather_copies_only_the_requested_row ... ok +test engine::kvstore::tests::budget_evicts_the_least_valuable_unprotected_entry ... ok +test engine::metal::qwen::tests::qwen_qsa_dense_mask_accepts_unsorted_blocks ... ignored, requires Apple Metal +test engine::metal::qwen::tests::qwen_qsa_full_context_preallocation_succeeds ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_qsa_logical_depth_oracles_cover_prefill_decode_and_state ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_qsa_real_layer_executes_the_first_sparse_token ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_qsa_selection_and_attention_remain_bounded_at_native_depths ... ignored, requires Apple Metal and allocates the full QSA score window +test engine::metal::qwen::tests::qwen_ple_hash_contract_matches_golden_boundaries ... ok +test engine::metal::qwen::tests::qwen_reference_layer_trace ... ignored, diagnostic trace against the pinned MLX-LM reference +test engine::metal::qwen::tests::qwen_reference_sdpa_trace ... ignored, diagnostic comparison with captured MTPLX SDPA inputs +test engine::metal::qwen::tests::qwen_reference_stateful_trace ... ignored, diagnostic trace against the pinned MLX-LM reference +test engine::metal::qwen::tests::qwen_reference_verify_trace ... ignored, diagnostic trace against the pinned MTPLX verifier +test engine::metal::qwen::tests::qwen_sorted_routed_prefill_matches_grouped_output ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_stage_counters_preserve_async_batches ... ignored, requires pinned Qwen embeddings and Apple Metal; stage profiler opt-in +test engine::kvstore::tests::opening_reclaims_checkpoints_the_index_cannot_reach ... ok +test engine::metal::qwen::tests::qwen_target_submission_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen::tests::qwen_verify_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal +test engine::metal::qwen_mtplx::allocator::mtplx_allocations_preserve_logical_bounds_with_reference_rounding ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_heap_limits_and_gpu_lifetime ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_replays_reference_allocations ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::ar::mtplx_ar_driver_matches_reference ... ignored, requires Apple Metal; unchanged MTPLX generate_ar orchestration oracle +test engine::kvstore::tests::longest_compatible_prefix_wins_and_hits_survive_reopen ... ok +test engine::metal::qwen_mtplx::array::mtplx_array_materialization_keeps_geometry_and_rejects_short_storage ... ignored, requires Apple Metal; typed materialized descriptors without model weights +test engine::metal::qwen_mtplx::array::mtplx_array_layout_preserves_dtype_shape_stride_and_materialization_flags ... ok +test engine::metal::qwen_mtplx::array::mtplx_array_detach_clears_every_siblings_inputs_and_primitive ... ok +test engine::metal::qwen_mtplx::attention::mtplx_qsa_dense_mask_graph_matches_reference ... ignored, requires Apple Metal; original MTPLX block mask including sentinel and tails +test engine::metal::qwen_mtplx::array::mtplx_dtype_promotion_matches_all_reference_pairs ... ok +test engine::metal::qwen_mtplx::attention::mtplx_qsa_eager_array_graph_matches_reference ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::attention::mtplx_qsa_prefill_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX tiled prefill gather +test engine::metal::qwen_mtplx::attention::mtplx_qsa_rows_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX QSA rows-gather attention +test engine::metal::qwen_mtplx::attention::mtplx_sdpa_fallback_graph_matches_reference ... ignored, requires Apple M5 Max; actual Qwen SDPA fallback reference +test engine::metal::qwen_mtplx::binary::mtplx_graph_binary_preserves_reference_branches ... ignored, requires Apple Metal; binary graph broadcast/layout and donation branches +test engine::metal::qwen_mtplx::binding::mtplx_installed_module_binding_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS and SSD policy; only small norms and four table rows evaluated +test engine::metal::qwen_mtplx::cache::mtplx_kv_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX KVCache growth, trim and state receipts +test engine::metal::qwen_mtplx::cache::mtplx_qsa_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX positional cache, mirror and held state aliases +test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_scalar_bits_and_custom_kernel_identity ... ignored, requires Metal buffers and external supervisor; scalar ownership only, no GPU dispatch +test database::tests::migrations_and_crud_work ... ok +test engine::metal::qwen_mtplx::allocator::mtplx_allocator_cache_matches_live_reference_trace ... ok +test engine::metal::qwen_mtplx::decoder::mtplx_decoder_connected_attention_and_moe_advance_caches ... ignored, requires Apple Metal and external test-supervisor; complete decoder/cache smoke +test engine::metal::qwen_mtplx::context_copy::prompt_copy_match_and_backoff_preserve_reference_policy ... ok +test database::tests::session_model_migration_preserves_data_and_accepts_catalog ... ok +test engine::metal::qwen_mtplx::dense::mtplx_dense_attention_batches_match_reference ... ignored, requires Apple M5 Max; actual BF16 batched matmul reference +test engine::metal::qwen_mtplx::dense::mtplx_dense_float_graph_matches_reference ... ignored, requires Apple M5 Max; floating Matmul factory and original dispatch +test engine::metal::qwen_mtplx::dense::mtplx_qsa_score_array_graph_matches_reference ... ignored, requires Apple M5 Max; canonical QSA F32 graph with the original TF32 setting +test engine::metal::qwen_mtplx::encoder::mtplx_empty_bindings_preserve_nil_dependencies_without_dummy_storage ... ignored, requires Apple Metal; empty original-kernel grids and nil bindings +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_commits_after_primitives_and_balances_completion ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_scope_routes_operators_and_allocator ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_shared_events_and_error_completion_order ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_untracked_dependencies_across_encoders_and_commands ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_drains_failure_preserves_cause_and_retries_only_failed_nodes ... ignored, requires Apple Metal; supervised evaluator failure and retry +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_releases_intermediates_and_preserves_descriptor_donation ... ignored, requires Apple Metal; graph ownership and last-consumer release +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_retains_tracer_topology_without_input_donation ... ignored, requires Apple Metal; evaluator tracer donation and retained topology +test engine::metal::qwen_mtplx::decoder::mtplx_decoder_residual_preserves_graph_order ... ok +test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_runs_cpu_gpu_siblings_and_async_continuations ... ignored, requires Apple Metal; tiny supervised CPU/GPU evaluator graph +test engine::metal::qwen_mtplx::compiled::mtplx_replay_replaces_descriptors_without_rebuilding_primitives ... ok +test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_tracks_stream_fences_per_leaf_and_cross_device_union ... ok +test engine::metal::qwen_mtplx::encoder::mtplx_encoder_barrier_epochs_and_commit_thresholds ... ok +test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_bfs_width_and_sibling_choice ... ok +test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ar_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_AR_REFERENCE required +test engine::metal::qwen_mtplx::execution::mtplx_full_installed_mtp_prefill_matches_reference ... ignored, full installed MTP model; external guard and DS4_QWEN38_MTP_PREFILL_REFERENCE required +test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ping_forward_matches_reference ... ignored, full installed model; requires external memory guard and DS4_QWEN38_FORWARD_REFERENCE +test engine::metal::qwen_mtplx::execution::qwen_power_throttle_drains_before_sleep_and_bypasses_at_full_power ... ignored, requires Apple Metal; run under test-supervisor +test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_scheduled_detach_and_tracer_errors ... ok +test engine::metal::qwen_mtplx::fused::mtplx_compiled_fusion_gpu_matches_reference ... ignored, requires Metal and external supervisor; generated fusion vs pinned SiLU plus multi-output/scalar dispatch +test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_forward_matches_model_reference ... ignored, requires Apple Metal; actual MTPLX GDN forward and cache receipts +test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_prefill_and_verify_match_model_reference ... ignored, requires Apple Metal; complete GDN Conv1d prefill and fused verify graphs +test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_staged_primitives_match_reference ... ignored, requires Apple Metal; 80 actual delta-update receipts and six conv-normalization kernels +test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_step_preserves_views_and_siblings ... ignored, requires Apple Metal; GDN custom primitive strided copies and sibling roots +test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_event_snapshots_preserve_values_streams_and_shared_errors ... ignored, requires Apple Metal events, no model weights +test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fence_coherence_preserves_empty_and_partial_word_spans ... ignored, requires Apple Metal3, model-free empty coherent dispatch +test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fences_transfer_cpu_gpu_gpu_cpu_in_both_modes ... ignored, requires Apple Metal3, tiny model-free coherent fence kernels +test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_forward_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX GatedResidual reference receipts +test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_pack_and_reads_match_reference ... ignored, requires Apple Metal; original hyper primitives and runtime pack +test engine::metal::qwen_mtplx::indexing::mtplx_graph_indexing_preserves_layouts_and_offsets ... ignored, requires Apple Metal; Select layout branches, GatherAxis views and empty arguments +test engine::metal::qwen_mtplx::indexing::mtplx_scatter_axis_graph_matches_reference ... ignored, requires Apple Metal; pinned Bool ScatterAxis layout and broadcast paths +test engine::metal::qwen_mtplx::issue103_analysis::issue103_mtp_final_receipts ... ignored, #103 diagnostic; full pinned model, SSD PLE and external memory supervisor +test engine::metal::qwen_mtplx::issue103_analysis::issue103_numeric_receipts ... ignored, #103 diagnostic; Metal required, eager versus compiled numeric operands +test engine::metal::qwen_mtplx::issue103_analysis::issue103_sampling_receipts ... ignored, #103 diagnostic; Metal required, pins currently broken sampling law and invalid rows +test engine::metal::qwen_mtplx::issue103_analysis::issue103_sorted_gather_receipts ... ignored, #103 diagnostic; M5 and pinned runtime shader required; initialized sentinels +test engine::metal::qwen_mtplx::issue103_analysis::issue103_tokenizer_receipts ... ignored, #103 diagnostic; installed tokenizer and latest oracle JSONL; no GPU +test engine::metal::qwen_mtplx::issue103_analysis::issue103_vision_position_receipts ... ignored, #103 diagnostic; small active attention module on Metal; mixed position boundary +test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_dequantize_matches_reference ... ignored, requires Apple Metal; original table dequantize dtype/group/bit/layout receipts +test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_quantize_matches_reference ... ignored, requires Apple Metal; original affine quantize dtype/group/bit/layout receipts +test engine::metal::qwen_mtplx::linear::mtplx_graph_gather_qmm_matches_reference ... ignored, requires Apple Metal; original GatherQMM array layouts and expert routing +test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_batched_identity ... ignored, requires Apple Metal and external supervisor; strided/broadcast batched affine dispatch +test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_matches_reference ... ignored, requires Apple M5 Max Metal; 78 pinned affine fixtures through lazy graph +test engine::metal::qwen_mtplx::load::mtplx_safetensors_load_laziness_boundaries_and_lifetime ... ignored, requires Apple Metal; bounded CPU Load and dependent GPU graph +test engine::metal::qwen_mtplx::mlp::mtplx_graph_shared_mlp_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX shared MLP reference receipts +test engine::metal::qwen_mtplx::mlp::mtplx_graph_split_and_compiled_silu_layouts ... ignored, requires Apple Metal; Split sibling ownership and compiled SiLU layouts +test engine::metal::qwen_mtplx::model::mtplx_model_compile_policy_and_run_boundaries ... ok +test engine::metal::qwen_mtplx::moe::mtplx_graph_sparse_moe_matches_reference ... ignored, requires Apple M5 Max; full MoE Array graph, no model weights; use test-supervisor +test engine::metal::qwen_mtplx::fused::mtplx_compiled_constant_spelling ... ok +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_embedding_matches_reference ... ignored, requires Apple Metal; full resident NGram graph and lazy decode history +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_gpu_rows_and_history_match_reference ... ignored, requires Apple Metal; original lazy NGram GPU row graph, no table +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_resident_matches_reference ... ignored, requires Apple Metal; original NGramTable resident gather/dequantize receipts +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_table_priority ... ignored, requires Apple Metal; NGram table routing and missing-sidecar behavior +test engine::metal::qwen_mtplx::ngram_sidecar::mtplx_ngram_sidecar_matches_reference ... ignored, requires Apple Metal; actual file-backed MTPLX sidecar/LRU/dequantization receipts +test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_allclose_branches ... ignored, requires Apple Metal; stage verification special values and original reduction branches +test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_matches_reference ... ignored, requires Apple Metal; original complete stage lifecycle and file-backed tables +test engine::metal::qwen_mtplx::normalization::mtplx_graph_norm_modules_match_reference ... ignored, requires Apple Metal; connected Qwen normalization graph golden fixtures +test engine::metal::qwen_mtplx::normalization::mtplx_graph_rms_norm_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph RMSNorm donation, copy, broadcast and looped kernels +test engine::metal::qwen_mtplx::normalization::mtplx_logsumexp_graph_matches_reference ... ignored, requires Apple Metal; original sampler LogSumExp branches +test engine::metal::qwen_mtplx::normalization::mtplx_softmax_graph_storage_matches_reference ... ignored, requires Apple Metal; original Softmax type and storage branches +test engine::metal::qwen_mtplx::ops::mtplx_concatenate_preserves_general_general_copy_semantics ... ignored, requires Apple Metal; Concatenate promotion, offsets, strided writers and validation +test engine::metal::qwen_mtplx::ops::mtplx_owned_operations_build_and_execute_qsa_layout_graph ... ignored, requires Apple Metal; self-contained lazy QSA-layout recipe, no callback map +test engine::metal::qwen_mtplx::ops::mtplx_slice_and_contiguous_preserve_cache_window_semantics ... ignored, requires Apple Metal; normalized Slice and Contiguous cache-window graph +test engine::metal::qwen_mtplx::owner::qwen_mtp_progress_publishes_without_turn_completion_or_double_counting ... ok +test engine::metal::qwen_mtplx::owner::qwen_product_generator_chat_matches_reference ... ignored, full installed model through product Generator; external supervisor and chat reference required +test engine::metal::qwen_mtplx::owner::qwen_vision_chat_checkpoint_preserves_image_identity ... ignored, requires installed Qwen, Metal, and DS4_QWEN_VISION_IMAGE; run under test-supervisor +test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_connected_matches_reference ... ignored, requires Apple M5 Max; complete original PLELayer with real lookup +test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_projection_and_cache_match_reference ... ignored, requires Apple M5 Max; actual MTPLX PLELayer after embedding lookup +test engine::metal::qwen_mtplx::policy::qwen_server_policy_matches_original_functions ... ignored, requires DS4_QWEN38_POLICY_REFERENCE from pinned MTPLX; CPU-only +test engine::metal::qwen_mtplx::prefill::mtplx_mtp_prefill_driver_matches_reference ... ignored, requires Apple Metal; original streaming MTP history prefill +test engine::metal::qwen_mtplx::prefill::mtplx_prefill_driver_matches_reference ... ignored, requires Apple Metal; original _prefill control flow and raw cache roots +test engine::metal::qwen_mtplx::qsa_attention::mtplx_qsa_attention_module_matches_reference ... ignored, requires Metal and unchanged whole-Attention receipts; no model download +test engine::metal::qwen_mtplx::qsa_compile::mtplx_compiled_indexer_array_replay_matches_reference ... ignored, requires Metal and original compiled indexer receipts; no model weights +test engine::metal::qwen_mtplx::qsa_core::mtplx_qsa_core_host_matches_reference ... ignored, requires Metal and original host-core receipts; no model weights +test engine::metal::qwen_mtplx::qsa_graph::qsa_cache_donation_requires_descriptor_data_and_allocation_contract ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::qsa_graph::qsa_completion_owns_buffers_until_gpu_finishes ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::qsa_graph::qsa_evaluator_retains_inputs_and_siblings_not_primary_data ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::qsa_graph::qsa_schedule_preserves_the_selected_sibling ... ok +test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_canonical_indexer_routes_match_reference ... ignored, requires Metal and original ongoing indexer routing receipts; no weights +test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_compiled_commit_matches_reference ... ignored, requires Metal and original indexer lifecycle receipts; no model weights +test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_prefix_matches_reference ... ignored, requires Apple Metal, original stateful QSAIndexer receipts, no model weights +test engine::metal::qwen_mtplx::qsa_prefill::mtplx_qsa_prefill_graph_matches_reference ... ignored, requires Metal and original two-stage prefill receipts, no weights +test engine::metal::qwen_mtplx::qsa_prepare::mtplx_qsa_prepare_graph_matches_reference ... ignored, requires Apple Metal; parameterized original QSA preparation and traced positions +test engine::metal::qwen_mtplx::qsa_select::mtplx_qsa_fused_graph_matches_reference ... ignored, requires Apple Metal and original QSA selector receipts; no model weights +test engine::metal::qwen_mtplx::random::mtplx_sampler_prefix_matches_reference ... ignored, requires Apple Metal; original target-prefix sampler and default RNG +test engine::metal::qwen_mtplx::random::mtplx_sampler_resident_matches_reference ... ignored, requires Apple Metal; original MTPLX resident sampling and keyed RNG +test engine::metal::qwen_mtplx::reduce::mtplx_sampler_argmax_matches_reference ... ignored, requires Apple Metal; original MTPLX greedy ArgReduce and storage +test engine::metal::qwen_mtplx::reduce::mtplx_sum_graph_matches_reference ... ignored, requires Apple Metal; original shared sum dispatch and QSA mean +test engine::metal::qwen_mtplx::fused::mtplx_compile_fusion_preserves_parent_and_output_boundaries ... ok +test engine::metal::qwen_mtplx::request::qwen_request_progress_uses_ui_frontiers_and_decode_timer ... ok +test engine::metal::qwen_mtplx::request::qwen_request_settings_override_backend_policy ... ok +test engine::metal::qwen_mtplx::fused::mtplx_compiled_emitter_matches_all_pinned_silu_bodies ... ok +test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_simplify_preserves_reference_parent_rules ... ok +test engine::metal::qwen_mtplx::residency::mtplx_residency_matches_reference_lifecycle_cases ... ignored, requires Apple Metal 3 / macOS 15, no model weights +test engine::metal::qwen_mtplx::rope::mtplx_rope_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX RoPE, static YaRN and multi-axis receipts +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_batched_matches_reference ... ignored, requires Apple Metal; original MTPLX bound and serial batch distributions +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_entry_matches_reference ... ignored, requires Apple Metal; actual MTPLX generator sampling entry +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_fallback_matches_reference ... ignored, requires Apple Metal; original sparse fallback and host dense sampler +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_penalties_match_reference ... ignored, requires Apple Metal; original GPU and CPU raw-logit penalties +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_selection_matches_reference ... ignored, requires Apple Metal; original MTPLX sampler selection, no model weights +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_serial_support_matches_reference ... ignored, requires Apple Metal; actual MTPLX serial candidate route, no weights +test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_primitive_holds_data_and_temporaries_in_fifo_order ... ignored, requires Apple Metal shared storage, no GPU dispatch or model weights +test engine::metal::profile::tests::profile_reports_locality_and_lru_hits ... ok +test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_cpu_events_propagate_errors_without_consuming_source ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_dispatch_counts_tenth_and_completes_after_failed_work ... ok +test engine::metal::qwen_mtplx::sdpa::mtplx_sdpa_vector_matches_reference ... ignored, requires Apple M5 Max; actual pinned fused SDPA kernels +test engine::metal::qwen_mtplx::session_cache::mtplx_full_installed_chat_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_CHAT_REFERENCE required +test database::tests::chat_and_compaction_history_survive_reopen_and_session_deletion ... ok +test engine::metal::qwen_mtplx::session_cache::mtplx_session_final_state_requires_capture_and_safe_cache ... ok +test engine::metal::qwen_mtplx::session_checkpoint::qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state ... ignored, model-free Metal checkpoint; external supervisor and DS4_QWEN38_CHECKPOINT_DIR required +test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_host_import_matches_reference ... ignored, model-free Metal imports; external supervisor and DS4_HOST_IMPORT_REFERENCE required +test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_matches_original_blobs_and_restore ... ignored, tiny GPU arrays; external supervisor and DS4_SESSION_CODEC_REFERENCE required +test engine::metal::qwen_mtplx::sort::mtplx_sort_graph_matches_reference ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::sparse_attention::mtplx_sparse_attention_graph_matches_reference ... ignored, requires Apple M5; actual MTPLX sparse Decode and TensorOps Prefill consumers +test engine::metal::qwen_mtplx::speculative::mtplx_full_installed_mtp_generation_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_MTP_GENERATION_REFERENCE required +test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_streams_preserve_fifo_errors_and_shutdown ... ok +test engine::metal::qwen_mtplx::stream::mtplx_stream_routes_queues_events_and_finalizes_all_under_pressure ... ignored, requires Apple Metal, tiny model-free two-queue copy +test engine::metal::qwen_mtplx::session_cache::mtplx_session_boundary_policy ... ok +test engine::metal::qwen_mtplx::submission::mtplx_submission_attaches_at_flush_eval_finish_and_drop ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::unary::mtplx_graph_unary_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph unary layout, donation and kernel boundaries +test engine::metal::qwen_mtplx::views::mtplx_dtype_conversion_and_views_preserve_values_layout_and_donation ... ignored, requires Apple Metal; original conversion/view kernels and donation in lazy eval +test engine::metal::qwen_mtplx::views::mtplx_dynamic_slice_graph_matches_reference ... ignored, requires Metal and original dynamic slice receipts; no model weights +test engine::metal::qwen_mtplx::views::mtplx_reshape_shares_or_copies_in_the_connected_evaluator ... ignored, requires Apple Metal; original Reshape copy kernels and shared backing +test engine::metal::qwen_mtplx::views::mtplx_views_preserve_aliases_offsets_strides_and_gpu_output ... ignored, requires Apple Metal; connected zero-copy views and offset-aware original copy kernel +test engine::metal::qwen_mtplx::vision::qwen_vision_tower_matches_mtplx_image ... ignored, requires Metal and pinned Qwen vision artifacts/reference values +test engine::metal::qwen_mtplx::weights::mtplx_checkpoint_projection_fusions_match_reference ... ignored, requires Apple Metal; bounded lazy checkpoint Load and fusion +test engine::metal::qwen_mtplx::weights::mtplx_installed_checkpoint_fusion_metadata_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS; headers only, never evaluate weights +test engine::metal::qwen_mtplx_tests::mtplx_affine_selection_preserves_reference_boundaries ... ok +test engine::metal::qwen_mtplx_tests::mtplx_canonical_hyper_and_gdn_kernels_match_reference ... ignored, requires Apple Metal, no model weights +test engine::metal::qwen_mtplx::stream::mtplx_stream_registry_preserves_thread_defaults_templates_and_clear ... ok +test engine::metal::qwen_mtplx_tests::mtplx_connected_delta_update_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_gather_sort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_decode_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_forward_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_staged_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_norm_modules_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_router_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_sparse_moe_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_connected_switch_glu_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_gate_up_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_cache_lifecycle_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_backings_and_dynamic_copies_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_routing_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_window_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_capacity_matches_reference ... ok +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_indexer_modes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_scalar_sources_match_reference ... ok +test engine::metal::qwen_mtplx_tests::mtplx_qsa_connected_cache_selection_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_eager_topk_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_f32_scores_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_preparation_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_routes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_producer_dispatch_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_canonical_source_bodies_preserve_pinned_hashes ... ok +test engine::metal::qwen_mtplx_tests::mtplx_qsa_selector_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_runtime_affine_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_runtime_argsort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test engine::metal::qwen_mtplx_tests::mtplx_runtime_gather_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights +test dev_brain::tests::parser_and_index_rebuild_ignore_links_in_code ... ok +test engine::metal::tests::batch_raw_span_includes_the_chunk_and_previous_window ... ok +test engine::metal::tests::compression_schedule_tracks_the_deepseek_model_shape ... ok +test engine::metal::tests::decode_feature_policy_keeps_device_generations_and_rollbacks_separate ... ok +test engine::metal::tests::decode_raw_span_tracks_the_logical_sliding_window ... ok +test engine::metal::tests::directional_steering_matches_the_ds4_token_oracle ... ignored, requires the installed Flash GGUF, DS4 steering fixture, and a Metal device +test engine::metal::qwen_mtplx::submission::mtplx_all_native_command_commits_pass_the_submission_hook ... ok +test engine::metal::tests::dspark_confidence_quantizes_q8_input_like_ds4 ... ok +test engine::metal::tests::dspark_markov_activation_matches_q8_rounding ... ok +test engine::metal::tests::dspark_matches_original_summary_cycles ... ignored, requires installed 0731/DSpark and DS4SERVER_DSPARK_REFERENCE cycle recording +test engine::metal::tests::dspark_runs_a_target_owned_greedy_cycle ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures +test engine::metal::tests::dspark_scheduler_matches_the_ds4_default_window ... ok +test engine::metal::tests::flash_0731_long_context_crosses_indexed_prefill_boundary ... ignored, requires a 0731 Flash GGUF and an Apple M5 device +test engine::metal::tests::dspark_verifier_selects_only_the_committed_row ... ok +test engine::metal::tests::flash_0731_m5_decode_performance_gate ... ignored, requires a 0731 Flash GGUF and an Apple M5 device +test engine::metal::tests::flash_0731_resident_and_ssd_streaming_choose_the_same_tokens ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device +test engine::metal::tests::flash_0731_runs_exact_sampled_dspark ... ignored, requires the installed 0731 target and checkpoint-specific DSpark GGUF fixtures +test engine::metal::tests::flash_0731_ssd_streaming_maps_and_seeds_batched_prefill_layers ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device +test engine::metal::tests::half_conversion_handles_normal_and_subnormal_values ... ok +test engine::metal::tests::pro_q4_model_spans_remain_isolated ... ok +test engine::metal::tests::generated_hotlists_match_each_model_shape ... ok +test engine::metal::tests::pro_ssd_streaming_full_layer_and_selected_batch_agree ... ignored, requires the installed DeepSeek V4 Pro GGUF and Apple Metal +test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device +test engine::metal::tests::pro_resident_admission_includes_weights_context_and_scratch ... ok +test engine::metal::tests::ssd_streaming_supports_dspark ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures +test engine::metal::tests::streaming_memory_plan_matches_ds4_graph_formulas ... ok +test engine::metal::vision::tests::glm53_resize_matches_reference_token_grids ... ok +test engine::qwen::tests::metadata_and_safetensors_boundaries_fail_closed ... ok +test engine::qwen::tests::pinned_artifact_set_loads_and_renders_goldens ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source +test engine::qwen::tests::pinned_qwen_metadata_matches_product_without_executor_state ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source +test engine::qwen_vision::qwen_image_cache_keys_distinguish_pixels_and_preserve_prior_text ... ok +test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_activity_preserves_reference_pressure_and_wait_rules ... ok +test engine::qwen_vision::qwen_vision_positions_and_resize_contract ... ok +test engine::sampling_tests::checkpoint_headers_identify_the_exact_model ... ok +test engine::sampling_tests::checkpoint_rebuilds_explain_compatibility_and_history_misses ... ok +test engine::sampling_tests::checkpoint_tag_accepts_an_unchanged_prefix_before_reminders ... ok +test engine::sampling_tests::checkpoint_tag_covers_the_canonical_chat_state ... ok +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_distribution_and_rng_match_reference ... ok +test engine::sampling_tests::ds4_sampler_real_logits_cpu_benchmark ... ignored, CPU-only; requires DS4_SAMPLER_RECORDING with the recorded GLM logits +test engine::sampling_tests::exact_delta_sampling_accepts_or_corrects_the_draft ... ok +test dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree ... FAILED +test engine::sampling_tests::generated_text_drains_incomplete_bytes_before_phase_changes_and_finish ... ok +test engine::sampling_tests::metal_executes_real_flash_token ... ignored, requires the 80 GiB Flash checkpoint and Apple Metal +test engine::sampling_tests::generated_text_preserves_thinking_utf8_and_split_stops ... ok +test engine::sampling_tests::qwen_rng_matches_numpy_pcg64 ... ok +test engine::sampling_tests::qwen_sampling_uses_mtplx_global_nucleus_mass_before_top_k ... ok +test engine::qwen::tests::frozen_inventory_and_memory_categories_are_exact ... ok +test engine::sampling_tests::resumed_prefill_crossover_matches_each_ds4_graph ... ok +test engine::sampling_tests::sampling_probabilities_match_ds4_filter_order ... ok +test engine::sampling_tests::split_utf8_token_bytes_are_joined_before_decoding ... ok +test engine::sampling_tests::top_k_and_stream_stops_are_applied_before_output ... ok +test engine::sampling_tests::workspace_instruction_identity_participates_in_append_only_kv_keys ... ok +test engine::sampling_tests::zero_temperature_is_greedy ... ok +test engine::tokenizer::tests::ds4_chat_matches_original_session_tokens ... ignored, CPU-only; requires DS4SERVER_CHAT_REFERENCE and its installed GGUF +test engine::tokenizer::tests::character_helpers_are_total_at_every_byte_offset ... ok +test engine::tokenizer::tests::qwen_checkpoint_continuation_matches_full_conversation ... ignored, requires the installed Qwen tokenizer, but no GPU or model weights +test engine::tokenizer::tests::qwen_plain_chat_matches_mtplx_server_tokens ... ignored, requires local tokenizer and tools/qwen-chat-reference.py output; no inference +test engine::tokenizer::tests::reasoning_prefixes_match_each_model_family_contract ... ok +test engine::validation::tests::configured_glm53_main_fixture_passes_the_exact_layout ... ok +test engine::validation::tests::configured_glm53_vision_fixture_passes_the_exact_layout ... ok +test engine::validation::tests::installed_dspark_fixture_passes_the_target_layout ... ok +test engine::validation::tests::mxfp4_is_limited_to_routed_experts ... ok +test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_workspace_matches_reference ... ok +test engine::metal::qwen_mtplx::ngram::mtplx_ngram_cpu_rows_and_raw_history_match_reference ... ok +test extensions::tests::duplicate_skill_names_and_update_rollback_are_deterministic ... ok +test extensions::tests::hook_output_is_bounded_typed_and_event_matched ... ok +test dev_brain::tests::skill_validation_and_default_guide_restore_are_strict ... ok +test dev_brain::tests::validation_names_every_drifted_source_and_requires_semantic_refresh ... ok +test extensions::tests::pinned_ponytail_runs_all_lifecycle_modes ... ignored, downloads the pinned Ponytail reference integration +test extensions::tests::manifest_and_paths_are_validated_without_following_escapes ... ok +test extensions::tests::symlink_and_variable_escapes_are_rejected ... ok +test extensions::tests::registry_persists_enable_disable_errors_and_uninstall ... ok +test instructions::tests::changes_removals_resume_and_compaction_are_append_only ... ok +test instructions::tests::empty_and_repaired_baselines_do_not_repeat_prompt_content ... ok +test instructions::tests::discovery_is_global_to_local_deduplicated_and_confined ... ok +test instructions::tests::newly_crossed_scope_is_added_once_and_unavailable_content_is_retained ... ok +test metrics::tests::cache_report_splits_buckets_and_folds_index_files_into_their_checkpoint ... ok +test metrics::tests::decode_timer_excludes_later_work_and_resets_between_requests ... ok +test metrics::tests::prompt_timing_keeps_unmeasured_components_distinct_from_zero ... ok +test metrics::tests::removed_preview_metric_code_stays_decodable ... ok +test metrics::tests::snapshots_track_runtime_and_server_counters ... ok +test model::qwen::qwen_vision_is_an_independent_managed_artifact_set ... ok +test model::qwen::tests::embedded_manifest_is_the_frozen_complete_set ... ok +test model::qwen::tests::verification_rejects_hash_mismatch_and_honors_cancellation ... ok +test model::transfer::tests::cancellation_keeps_the_partial_file_for_the_next_run ... ok +test model::transfer::tests::catalog_and_checksum_verification_are_explicit ... ok +test model::transfer::tests::managed_artifact_inventory_and_delete_include_partial_files ... ok +test model::transfer::tests::qwen_set_verifies_reloads_and_removes_as_one_managed_artifact ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source +test engine::sampling_tests::probability_ratio_speculation_preserves_the_target_distribution ... ok +test instructions::tests::frames_escape_control_tags_and_enforce_utf8_byte_limits ... ok +test model::transfer::tests::verification_reports_bytes_read ... ok +test model_eval::tests::acceleration_receipt_reports_effective_sampling_contract ... ok +test model_eval::tests::acceleration_switch_uses_each_models_native_path ... ok +test model_eval::tests::canary_cross_phase_samples_keep_their_start_phase ... ok +test model::transfer::tests::restart_resumes_at_the_existing_partial_byte ... ok +test model_eval::tests::evaluations_exclude_overlapping_workers_and_release_on_exit ... ok +test model_eval::tests::external_canary_observes_phases_and_finishes_on_eof ... ignored, Metal probe only; DS4SERVER_CANARY_BINARY must point to the current release binary +test model_eval::tests::external_canary_protocol_accepts_only_known_phases ... ok +test model_eval::tests::parses_safe_defaults_and_overrides ... ok +test model_eval::tests::rejects_unbounded_or_invalid_limits ... ok +test model_eval::tests::repeated_prompts_form_one_ordered_conversation ... ok +test model_eval::tests::warmup_timing_separates_loading_without_inventing_missing_durations ... ok +test model_eval::tests::worker_progress_events_advance_the_watchdog ... ok +test native_edit::tests::edit_history_supports_undo_and_redo ... ok +test native_edit::tests::paste_replays_a_complete_command_shortcut ... ok +test native_media::tests::media_player_only_accepts_remote_http_urls ... ok +test native_menu::tests::edit_items_are_custom_menu_events ... ok +test native_menu::tests::user_guide_uses_unmodified_f1 ... ok +test native_menu::tests::view_items_keep_the_selected_pane_checked ... ok +test runtime::tests::dropping_active_generation_cancels_it ... ok +test runtime::tests::operation_failures_use_the_matching_event ... ok +test runtime::tests::operation_tracking_is_consistent ... ok +test runtime::tests::runtime_panic_recovery_accepts_the_next_operation ... ok +test server::request::tests::reasoning_requests_preserve_each_models_exact_modes ... ok +test server::tests::canonical_tool_replay_preserves_argument_types ... ok +test server::tests::connection_slots_enforce_the_limit_and_release_on_drop ... ok +test server::tests::content_arrays_match_ds4_text_projection ... ok +test server::tests::context_limit_errors_use_protocol_standard_fields ... ok +test server::tests::cors_headers_are_opt_in_and_cover_preflight ... ok +test server::tests::dsml_tool_stream_projects_incremental_json_arguments ... ok +test server::tests::generated_dsml_becomes_openai_tool_calls ... ok +test server::tests::generated_tool_parser_accepts_ds4_recovery_syntaxes ... ok +test server::tests::model_metadata_uses_the_requested_alias_and_default_token_limit ... ok +test server::tests::protocol_tool_results_require_live_or_replayed_call_ids ... ok +test dev_brain::tests::source_hash_drift_excludes_page_from_authoritative_queries ... ok +test server::tests::responses_parser_rejects_dropped_state_and_preserves_hosted_calls ... ok +test engine::metal::qwen_mtplx::array::mtplx_array_releases_sibling_cycles_and_deep_input_graphs ... ok +test server::tests::streaming_errors_match_ds4_server_framing ... ok +test server::tests::sampled_tool_replay_survives_server_restart ... ok +test server::tests::tool_prompt_matches_ds4_server_text ... ok +test server::tests::tool_stream_projects_ds4_recovery_syntaxes ... ok +test settings::tests::captured_ds4_cli_options_are_uniquely_classified ... ok +test settings::tests::effective_settings_build_one_engine_and_turn_configuration ... ok +test settings::tests::effective_settings_preserve_unset_model_defaults ... ok +test settings::tests::execution_settings_keep_ds4_automatic_values_and_glm_rules ... ok +test settings::tests::kv_cache_gates_follow_the_ds4_store_rules ... ok +test settings::tests::reasoning_selection_is_exact_and_model_specific ... ok +test settings::tests::runtime_settings_preserve_automatic_values_and_cross_field_rules ... ok +test settings::tests::speculative_settings_match_acceleration_defaults_and_dependencies ... ok +test dev_brain::tests::validation_keeps_broken_links_editable_and_reports_provenance_errors ... ok +test dev_brain::tests::verified_skills_are_indexed_and_injected_on_demand ... ok +test server::tests::streaming_prefill_keeps_the_client_connection_alive ... ok +test model_eval::tests::cpu_accounting_matches_getrusage_units ... FAILED +test engine::validation::tests::installed_ds4_fixture_opens_and_renders_a_prompt ... ok +test model::transfer::tests::stalled_download_times_out_and_keeps_the_partial_file ... ok +test server::tests::request_deadline_stops_slow_clients ... ok +test engine::sampling_tests::ds4_sampler_matches_original_reference_tokens_and_rng ... ok +test extensions::tests::direct_hook_runner_isolated_environment_and_handles_failures ... ok +test agent::tests::execution_failures_and_timeouts_have_distinct_codes ... ok +test engine::qwen_vision::qwen_image_preprocessing_matches_pillow_mtplx ... ok +kill: 95702: No such process +test agent::tests::dropping_session_tools_stops_live_jobs_and_removes_output ... ok +test agent::tests::local_file_and_bash_tools_execute_in_the_project ... ok +test extensions::tests::lifecycle_matchers_preserve_declared_event_order ... ok +test engine::metal::qwen_mtplx_tests::mtplx_runtime_shaders_are_pinned ... ok +test extensions::tests::hook_failures_are_bounded_actionable_and_non_shell ... ok +test engine::metal::qwen_mtplx::sampling::mtplx_sampler_mixed_distributions_match_reference ... ok + +failures: + +---- dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree stdout ---- + +thread 'dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree' (2320365) panicked at src/dev_brain.rs:2054:51: +called `Result::unwrap()` on an `Err` value: "Invalid frontmatter in concepts/answer.md: invalid type: integer `8872200`, expected a string" + +---- model_eval::tests::cpu_accounting_matches_getrusage_units stdout ---- + +thread 'model_eval::tests::cpu_accounting_matches_getrusage_units' (2320543) panicked at src/model_eval.rs:1859:9: +proc_pid_rusage=336441833ns getrusage=323741000ns + + +failures: + dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree + model_eval::tests::cpu_accounting_matches_getrusage_units + +test result: FAILED. 305 passed; 2 failed; 210 ignored; 0 measured; 0 filtered out; finished in 5.81s + +error: test failed, to rerun pass `--bin ds4-server` diff --git a/docs/issue103-analysis/upstream-gates.log b/docs/issue103-analysis/upstream-gates.log new file mode 100644 index 0000000..85a204e --- /dev/null +++ b/docs/issue103-analysis/upstream-gates.log @@ -0,0 +1,2 @@ +............ [100%] +12 passed in 1.86s diff --git a/docs/issue103-analysis/upstream-tests.log b/docs/issue103-analysis/upstream-tests.log new file mode 100644 index 0000000..f9beb86 --- /dev/null +++ b/docs/issue103-analysis/upstream-tests.log @@ -0,0 +1,16 @@ +........................................................................ [ 56%] +......................................................s [100%] +=============================== warnings summary =============================== +tests/test_non_finite_logits_and_lazy_sampler_law.py::test_sample_from_logits_raises_on_every_non_finite_shape[0.6-row0] +tests/test_non_finite_logits_and_lazy_sampler_law.py::test_sample_from_logits_raises_on_every_non_finite_shape[1.0-row0] + /tmp/ds4-issue103/mtplx/mtplx/fast_sampling.py:482: RuntimeWarning: All-NaN slice encountered + spill = np.nanmin(cand_val_rows, axis=1) >= cutoff + +tests/test_non_finite_logits_and_lazy_sampler_law.py::test_sample_from_logits_raises_on_every_non_finite_shape[0.6-row1] +tests/test_non_finite_logits_and_lazy_sampler_law.py::test_sample_from_logits_raises_on_every_non_finite_shape[0.6-row3] +tests/test_non_finite_logits_and_lazy_sampler_law.py::test_sample_from_logits_raises_on_every_non_finite_shape[1.0-row1] +tests/test_non_finite_logits_and_lazy_sampler_law.py::test_sample_from_logits_raises_on_every_non_finite_shape[1.0-row3] + /tmp/ds4-issue103/mtplx/mtplx/sampling.py:138: RuntimeWarning: invalid value encountered in subtract + scaled = scaled - np.max(scaled) + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html diff --git a/src/engine/metal/qwen_mtplx.rs b/src/engine/metal/qwen_mtplx.rs index b2b22c4..b512997 100644 --- a/src/engine/metal/qwen_mtplx.rs +++ b/src/engine/metal/qwen_mtplx.rs @@ -27,6 +27,8 @@ mod gdn; mod graph_sync; mod hyper; mod indexing; +#[cfg(test)] +mod issue103_analysis; mod linear; mod load; mod mlp; diff --git a/src/engine/metal/qwen_mtplx/issue103_analysis.rs b/src/engine/metal/qwen_mtplx/issue103_analysis.rs new file mode 100644 index 0000000..09c9b3d --- /dev/null +++ b/src/engine/metal/qwen_mtplx/issue103_analysis.rs @@ -0,0 +1,759 @@ +//! Isolated diagnostics for #103. Assertions pin observed defects, not fixes. +//! Run serially with --ignored --nocapture; no production option is changed. +use super::array::{Array, Dtype}; +use super::binary::{Binary, binary}; +use super::compiled::Replay; +use super::sampling::{self, SamplerConfig}; +use super::stream::{Device, Streams}; +use super::{ops, random, unary}; +use crate::engine::Rng; + +fn f32_array(shape: &[i32], values: &[f32]) -> Array { + let bytes = values + .iter() + .flat_map(|v| v.to_le_bytes()) + .collect::>(); + Array::new(shape, Dtype::F32, super::scalar_buffer(&bytes).unwrap()).unwrap() +} + +#[test] +#[ignore = "#103 diagnostic; Metal required, pins currently broken sampling law and invalid rows"] +fn issue103_sampling_receipts() { + super::configure_sources().unwrap(); + let _context = super::super::gpu::Context::open_qwen(0).unwrap(); + let streams = Streams::new(Some(super::allocator::Allocator::new().unwrap())); + let gpu = streams.default_stream(Device::Gpu).unwrap(); + let values = (0..64) + .map(|i| match i { + 0 => 0.4_f32.ln(), + 1 => 0.3_f32.ln(), + _ => (0.3_f32 / 62.).ln(), + }) + .collect::>(); + let row = f32_array(&[64], &values); + let config = SamplerConfig { + temperature: 1., + top_p: 0.5, + top_k: 2, + ..Default::default() + }; + let distribution = + sampling::distribution_from_logits(&row, &config, &[], &[], &streams, gpu).unwrap(); + let mut ordinary = [0; 64]; + let mut rng = Rng::new_qwen(0); + for _ in 0..512 { + ordinary[distribution.sample(&mut rng).unwrap() as usize] += 1; + } + let mut lazy = [0; 64]; + for seed in 0..128 { + let token = sampling::lazy_sample(&row, &config, &random::key(seed).unwrap(), gpu).unwrap(); + let id = ops::read_i64(&token, &streams, gpu).unwrap()[0]; + lazy[id as usize] += 1; + } + println!( + "{}", + serde_json::json!({"case":"sampling_law", "temperature":1.,"top_k":2,"top_p":0.5,"ordinary_probabilities":[distribution.probability(0).unwrap(),distribution.probability(1).unwrap()],"ordinary_counts":&ordinary[..2],"lazy_counts":&lazy[..2],"expected_probabilities":[4./7.,3./7.]}) + ); + assert!(ordinary[0] > 0 && ordinary[1] > 0); + assert_eq!( + &lazy[..2], + &[128, 0], + "diagnosis no longer reproduces; update child regression" + ); + // Official sampler, with logits scaled so its post-temperature mass is known. + for (temperature, top_k, top_p) in [(0.6, 20, 0.95), (0.2, 8, 0.8), (1.0, 2, 1.0)] { + let row = f32_array( + &[64], + &values + .iter() + .map(|v| v * temperature as f32) + .collect::>(), + ); + let config = SamplerConfig { + temperature, + top_k, + top_p, + ..Default::default() + }; + let distribution = + sampling::distribution_from_logits(&row, &config, &[], &[], &streams, gpu).unwrap(); + let probabilities = (0..64) + .map(|i| distribution.probability(i).unwrap()) + .collect::>(); + let mut counts = [0; 64]; + for seed in 0..512 { + let token = + sampling::lazy_sample(&row, &config, &random::key(seed).unwrap(), gpu).unwrap(); + counts[ops::read_i64(&token, &streams, gpu).unwrap()[0] as usize] += 1; + } + println!( + "{}", + serde_json::json!({"case":"sampling_matrix","temperature":temperature,"top_k":top_k,"top_p":top_p,"ordinary_probabilities":probabilities,"lazy_counts":counts.to_vec()}) + ); + } + for (name, values) in [ + ("nan", vec![f32::NAN; 64]), + ("positive_inf", [vec![f32::INFINITY], vec![0.; 63]].concat()), + ("all_negative_inf", vec![f32::NEG_INFINITY; 64]), + ( + "valid_mask", + [ + vec![f32::NEG_INFINITY, 2., f32::NEG_INFINITY, 1.], + vec![f32::NEG_INFINITY; 60], + ] + .concat(), + ), + ] { + let row = f32_array(&[64], &values); + for temperature in [0., 0.6, 1.] { + let config = SamplerConfig { + temperature, + ..Default::default() + }; + let result = sampling::sample_from_logits( + &row, + &config, + &mut Rng::new_qwen(0), + &[], + &[], + &streams, + gpu, + ); + println!( + "{}", + serde_json::json!({"case":"invalid_logits","row":name,"temperature":temperature,"ordinary":result.as_ref().map(|r|r.0).map_err(Clone::clone)}) + ); + if name == "valid_mask" { + assert!([1, 3].contains(&result.unwrap().0)); + } else { + assert!(result.is_ok(), "diagnosis no longer reproduces"); + } + } + let config = SamplerConfig { + temperature: 1., + ..Default::default() + }; + let result = sampling::lazy_sample(&row, &config, &random::key(7).unwrap(), gpu) + .and_then(|a| ops::read_i64(&a, &streams, gpu)); + println!( + "{}", + serde_json::json!({"case":"invalid_logits_lazy","row":name,"result":result}) + ); + } +} + +#[test] +#[ignore = "#103 diagnostic; Metal required, eager versus compiled numeric operands"] +fn issue103_numeric_receipts() { + super::configure_sources().unwrap(); + let _context = super::super::gpu::Context::open_qwen(0).unwrap(); + let streams = Streams::new(Some(super::allocator::Allocator::new().unwrap())); + let gpu = streams.default_stream(Device::Gpu).unwrap(); + let gate = ops::astype(&f32_array(&[4], &[-6.84375; 4]), Dtype::BF16, false, gpu).unwrap(); + let value = ops::astype(&f32_array(&[4], &[1.; 4]), Dtype::BF16, false, gpu).unwrap(); + ops::evaluate(&streams, &[gate.clone(), value.clone()], gpu, false).unwrap(); + let body = |a: &[Array]| { + binary( + &unary::unary(&a[0], unary::Unary::Sigmoid, gpu)?, + &a[1], + Binary::Multiply, + gpu, + ) + .map(|o| vec![o]) + }; + let eager = body(&[gate.clone(), value.clone()]).unwrap(); + let compiled = Replay::trace_compiled(&[gate.clone(), value.clone()], body) + .unwrap() + .run(&[gate, value]) + .unwrap(); + let read = |a: &Array| { + ops::read_f32_as_f64( + &ops::astype(a, Dtype::F32, false, gpu).unwrap(), + &streams, + gpu, + ) + .unwrap() + }; + let e = read(&eager[0]); + let c = read(&compiled[0]); + println!( + "{}", + serde_json::json!({"case":"bf16_gate","gate":-6.84375,"eager":e,"compiled":c}) + ); + assert_ne!(e, c, "diagnosis no longer reproduces"); + for (name, operand, mode) in [ + ("qsa_divisor", 128_f32.sqrt(), Binary::Divide), + ( + "yarn_amplitude", + (0.1 * 4_f64.ln() + 1.) as f32, + Binary::Multiply, + ), + ] { + let x = f32_array(&[4], &[1.5000001, 1.5000002, 1., 0.]); + let scalar = f32_array(&[], &[operand]); + let body = |a: &[Array]| binary(&a[0], &scalar, mode, gpu).map(|o| vec![o]); + let eager = body(std::slice::from_ref(&x)).unwrap(); + let compiled = Replay::trace_compiled(std::slice::from_ref(&x), body) + .unwrap() + .run(&[x]) + .unwrap(); + let e = read(&eager[0]); + let c = read(&compiled[0]); + println!( + "{}", + serde_json::json!({"case":name,"operand_bits":operand.to_bits(),"eager":e,"compiled":c}) + ); + assert_ne!(e, c, "diagnosis no longer reproduces"); + } +} + +#[test] +#[ignore = "#103 diagnostic; M5 and pinned runtime shader required; initialized sentinels"] +fn issue103_sorted_gather_receipts() { + super::configure_sources().unwrap(); + let _context = super::super::gpu::Context::open_qwen(0).unwrap(); + assert!(super::sparse_attention::nax_available()); + let scope = super::encoder::Scope::new().unwrap(); + let buffer = + |count: usize, bits: u16| super::scalar_buffer(&bits.to_le_bytes().repeat(count)).unwrap(); + let w = super::scalar_buffer(&vec![0; 64 * 256 * 256 / 8 * 4]).unwrap(); + let scales = buffer(64 * 256 * 8, 0); + let biases = buffer(64 * 256 * 8, 0x3f80); // all dequantized weights exactly 1 + for tokens in [3276_u32, 3277, 3404, 4095, 4096] { + let rows = tokens * 10; + let x = buffer(rows as usize * 256, 0x3f80); + let lhs = super::scalar_buffer(&(0..rows).flat_map(u32::to_le_bytes).collect::>()) + .unwrap(); + let rhs = super::scalar_buffer( + &(0..rows) + .flat_map(|r| (r * 64 / rows).to_le_bytes()) + .collect::>(), + ) + .unwrap(); + let y = buffer(rows as usize * 256, 0x7fc1); // initialized NaN, independent of allocator reuse + let (_, dispatches) = super::super::qwen_mtplx_tests::capture_dispatches(|| { + super::gather_bf16( + [&w, &scales, &biases, &x, &lhs, &rhs, &y], + [rows, 1, 256, 256, 64, rows], + [4, 32], + true, + true, + ) + .unwrap(); + }); + scope.synchronize().unwrap(); + let mut data = vec![0; rows as usize * 256 * 2]; + y.read(0, &mut data).unwrap(); + let bad = data + .chunks_exact(512) + .enumerate() + .filter_map(|(r, bytes)| { + bytes + .chunks_exact(2) + .any(|b| u16::from_le_bytes(b.try_into().unwrap()) != 0x4380) + .then_some(r) + }) + .collect::>(); + println!( + "{}", + serde_json::json!({"case":"sorted_gather","tokens":tokens,"routed_rows":rows,"bad_rows":bad.len(),"first_bad":bad.first(),"last_bad":bad.last(),"dispatches":format!("{dispatches:?}")}) + ); + if tokens == 3276 || tokens == 4096 { + assert!(bad.is_empty()); + } else { + assert!(!bad.is_empty(), "diagnosis no longer reproduces"); + } + } +} + +#[test] +#[ignore = "#103 diagnostic; installed tokenizer and latest oracle JSONL; no GPU"] +fn issue103_tokenizer_receipts() { + let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap()); + let tokenizer = + crate::engine::tokenizer::Tokenizer::load_qwen(&root.join("tokenizer.json")).unwrap(); + let input = std::fs::read_to_string(std::env::var_os("DS4_ISSUE103_ORACLE").unwrap()).unwrap(); + let mut mismatches = 0; + for case in input + .lines() + .filter_map(|l| serde_json::from_str::(l).ok()) + .filter(|c| c["case"] == "tokenizer") + { + let text = case["text"].as_str().unwrap(); + let ids = tokenizer.tokenize(text); + let matches = serde_json::json!(ids) == case["ids"]; + mismatches += usize::from(!matches); + println!( + "{}", + serde_json::json!({"case":"native_tokenizer","text":text,"actual_ids":ids,"expected_ids":case["ids"],"matches":matches}) + ); + } + println!( + "{}", + serde_json::json!({"case":"tokenizer_summary","mismatches":mismatches}) + ); + assert_eq!(mismatches, 5, "diagnosis changed; inspect exact token IDs"); +} + +#[test] +#[ignore = "#103 diagnostic; full pinned model, SSD PLE and external memory supervisor"] +fn issue103_mtp_final_receipts() { + use super::decoder::AttentionCache; + use super::execution::Execution; + use super::speculative::Options; + let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap()); + assert!(!super::ngram::resident_policy(), "use SSD PLE"); + eprintln!("#103 loading unchanged Bare-Speed model"); + let mut execution = Execution::load(&root, true, false, 100).unwrap(); + let ids = [9707, 11, 1075]; // fixed valid IDs; no template is involved in this model-level repro + let prepared = execution + .prefill_mtp_history( + &ids, + &super::prefill::Options { + chunk_size: Some(2048), + final_logits_only: true, + external_cache_only: false, + external_emit_logits: true, + return_hidden: true, + }, + None, + false, + |_| Ok(()), + |_| Ok(()), + ) + .unwrap(); + let baseline = execution + .capture_session( + &ids, + &prepared.target.logits, + prepared.target.hidden.as_ref(), + true, + false, + ) + .unwrap(); + let offset = |e: &Execution| { + e.cache + .iter() + .filter_map(|c| match &c.as_ref()?.attention { + AttentionCache::Qsa(c) => Some(c.kv.offset), + _ => None, + }) + .collect::>() + }; + let mut final_token = 0; + for depth in [1, 2, 3] { + for temperature in [0., 0.6] { + for stop_exit in [false, true] { + let (logits, hidden, _) = execution.restore_session(&baseline, &ids).unwrap(); + let sampler = SamplerConfig { + temperature, + ..Default::default() + }; + let result = execution + .decode_mtp( + &logits, + hidden.as_ref().unwrap(), + &sampler, + &sampler, + &Options { + prompt_ids: &ids, + depth, + max_tokens: if stop_exit { 8 } else { 1 }, + seed: 0, + stops: if stop_exit { + if temperature == 0. { &[11] } else { &[314] } + } else { + &[] + }, + committed_history: true, + family_capture: true, + lazy_history: false, + defer_repair_eval: false, + capture_final_state: true, + }, + |_| Ok(()), + |_| Ok(()), + ) + .unwrap(); + let mut key = ids.to_vec(); + key.push(result.tokens[0] as i32); + let snapshot = execution + .capture_session(&key, &result.logits, Some(&result.hidden), true, false) + .unwrap(); + let (.., reused) = execution.restore_session(&snapshot, &key).unwrap(); + let offsets = offset(&execution); + println!( + "{}", + serde_json::json!({"case":"mtp_final","exit":if stop_exit {"first_primary_stop"} else {"max_tokens"},"depth":depth,"temperature":temperature,"prompt_ids":ids,"tokens":result.tokens,"key_length":key.len(),"restored_cached_tokens":reused,"trunk_offsets":offsets,"mtp_offset":execution.mtp_offset().unwrap(),"pending":result.pending,"final_state_error":result.final_state_error}) + ); + assert_eq!(offsets, vec![3; 12]); + assert_eq!(reused, 4); + assert!(result.pending.is_none() && result.final_state_error.is_none()); + final_token = result.tokens[0] as i32; + } + } + } + execution.restore_session(&baseline, &ids).unwrap(); + let bad = execution + .forward(&[1075], false, false) + .unwrap() + .logits + .unwrap(); + let bad = execution.values_f32(&bad).unwrap(); + execution.restore_session(&baseline, &ids).unwrap(); + let good = execution + .forward(&[final_token, 1075], false, false) + .unwrap() + .logits + .unwrap(); + let good = execution + .values_f32(&super::ar::last_row(&good, execution.stream).unwrap()) + .unwrap(); + let different = bad + .iter() + .zip(&good) + .filter(|(a, b)| a.to_bits() != b.to_bits()) + .count(); + println!( + "{}", + serde_json::json!({"case":"mtp_next_turn","missing_token":final_token,"suffix_ids":[1075],"different_logits":different,"bad_sha256":super::super::qwen_mtplx_tests::hex_digest(&bad.iter().flat_map(|v|v.to_le_bytes()).collect::>()),"good_sha256":super::super::qwen_mtplx_tests::hex_digest(&good.iter().flat_map(|v|v.to_le_bytes()).collect::>())}) + ); + assert!(different > 0); +} + +#[test] +#[ignore = "#103 diagnostic; small active attention module on Metal; mixed position boundary"] +fn issue103_vision_position_receipts() { + use super::cache::QsaCache; + use super::mlp::Linear; + use super::qsa_attention::{Attention, Input, Options}; + super::configure_sources().unwrap(); + let _context = super::super::gpu::Context::open_qwen(0).unwrap(); + let streams = Streams::new(Some(super::allocator::Allocator::new().unwrap())); + let gpu = streams.default_stream(Device::Gpu).unwrap(); + let bf = |shape: &[i32], v: &[f32]| { + ops::astype(&f32_array(shape, v), Dtype::BF16, false, gpu).unwrap() + }; + let matrix = |n: usize| { + bf( + &[n as i32, 8], + &(0..n * 8) + .map(|i| f32::from(i / 8 % 8 == i % 8)) + .collect::>(), + ) + }; + let q = matrix(16); + let k = matrix(8); + let v = matrix(8); + let out = matrix(8); + let norm = bf(&[8], &[1.; 8]); + let freq = f32_array(&[4], &[1., 0.1, 0.01, 0.001]); + let axes = Array::new( + &[4], + Dtype::I32, + super::scalar_buffer( + &[0_i32, 1, 2, 0] + .into_iter() + .flat_map(i32::to_le_bytes) + .collect::>(), + ) + .unwrap(), + ) + .unwrap(); + let table = Array::new( + &[3, 4], + Dtype::I32, + super::scalar_buffer( + &[0_i32, 1, 2, 2, 0, 1, 3, 3, 0, 1, 4, 4] + .into_iter() + .flat_map(i32::to_le_bytes) + .collect::>(), + ) + .unwrap(), + ) + .unwrap(); + let options = Options { + indexer: super::QsaIndexerOptions { + fused: false, + compiled: false, + prefill: false, + phase_prefill: true, + prefill_min_rows: 128, + prefill_compile_rows: 2048, + prefill_min_context: 16384, + prefill_workspace: 0, + tile_rows: 64, + flash: false, + gather_decode: false, + gather: false, + gather_max_rows: 4, + gather_min_context: 0, + tf32: false, + }, + flash_min_context: 0, + prefill_gather: false, + gather_tile_rows: 64, + }; + for (pos, rows) in [(0, 4), (4, 2), (2, 4)] { + let mut attention = Attention { + input: Input::Separate( + [Linear::Dense(&q), Linear::Dense(&k), Linear::Dense(&v)], + [None; 3], + ), + output: Linear::Dense(&out), + output_bias: None, + heads: 1, + kv_heads: 1, + dim: 8, + eps: 1e-6, + scale: 1. / 8_f64.sqrt(), + q_norm: &norm, + k_norm: &norm, + inv_freq: &freq, + rope_scaling: 1., + mrope_axes: Some(&axes), + indexer: None, + }; + let mut cache = QsaCache::new(4); + if pos > 0 { + let prefix = bf(&[1, 1, pos, 8], &vec![0.; pos as usize * 8]); + cache.kv.update_and_fetch(&prefix, &prefix, gpu).unwrap(); + } + let x = bf( + &[1, rows, 8], + &(0..rows * 8) + .map(|i| (i + 1) as f32 / 32.) + .collect::>(), + ); + let output = attention + .call( + &x, + &mut cache, + &options, + Some((Some(&table), -1)), + &streams, + gpu, + ) + .unwrap(); + ops::evaluate(&streams, &[output], gpu, false).unwrap(); + let projected = ops::reshape( + &ops::rms_norm(&x, Some(&norm), 1e-6, gpu).unwrap(), + &[1, rows, 1, 8], + gpu, + ) + .unwrap(); + let positions = (0..3) + .flat_map(|axis| { + (pos..pos + rows).map(move |p| { + if p < 4 { + [vec![0, 1, 2, 2], vec![0, 1, 3, 3], vec![0, 1, 4, 4]][axis][p as usize] + } else { + p - 1 + } + }) + }) + .collect::>(); + let positions = Array::new( + &[3, rows], + Dtype::I32, + super::scalar_buffer( + &positions + .iter() + .flat_map(|p| p.to_le_bytes()) + .collect::>(), + ) + .unwrap(), + ) + .unwrap(); + let (cos, sin) = super::rope::mrope_cos_sin(&positions, &freq, &axes, gpu).unwrap(); + let expected = super::rope::apply_partial(&projected, &cos, &sin, gpu).unwrap(); + let actual = ops::slice( + &cache.kv.keys.as_ref().unwrap().borrow(), + &[0, 0, pos, 0], + &[1, 1, pos + rows, 8], + &[1; 4], + gpu, + ) + .unwrap(); + let read = |a: &Array| { + ops::read_f32_as_f64( + &ops::astype(a, Dtype::F32, false, gpu).unwrap(), + &streams, + gpu, + ) + .unwrap() + }; + let a = read(&actual); + let e = read(&expected); + let bad = a + .chunks_exact(8) + .zip(e.chunks_exact(8)) + .enumerate() + .filter_map(|(i, (a, e))| (a != e).then_some(i)) + .collect::>(); + println!( + "{}", + serde_json::json!({"case":"vision_positions","start":pos,"rows":rows,"table_end":4,"delta":-1,"different_rows":bad,"actual_keys":a,"expected_keys":e}) + ); + if pos == 2 { + assert_eq!(bad, vec![0, 1]); + } else { + assert!(bad.is_empty()); + } + } + let prefix = bf( + &[1, 12, 8], + &(0..96) + .map(|i| ((i * 17 % 31) as f32 - 15.) / 16.) + .collect::>(), + ); + let current = bf(&[1, 1, 8], &[0.3, -0.7, 0.2, 0.9, -0.2, 0.4, 0.6, -0.1]); + let mut outputs = Vec::new(); + for vision in [false, true] { + let mut attention = Attention { + input: Input::Separate( + [Linear::Dense(&q), Linear::Dense(&k), Linear::Dense(&v)], + [None; 3], + ), + output: Linear::Dense(&out), + output_bias: None, + heads: 1, + kv_heads: 1, + dim: 8, + eps: 1e-6, + scale: 1. / 8_f64.sqrt(), + q_norm: &norm, + k_norm: &norm, + inv_freq: &freq, + rope_scaling: 1., + mrope_axes: Some(&axes), + indexer: Some(super::qsa_indexer::Indexer { + projection: Some(Linear::Dense(&q)), + heads: 1, + kv_heads: 1, + head_dim: 8, + block_topk: 1, + ratio: 4, + q_norm: &norm, + k_norm: &norm, + inv_freq: &freq, + eps: 1e-6, + scaling: 1., + scratch: 32 * 1024 * 1024, + compiled: None, + }), + }; + let mut cache = QsaCache::new(4); + let initial = attention + .call(&prefix, &mut cache, &options, None, &streams, gpu) + .unwrap(); + ops::evaluate(&streams, &[initial], gpu, false).unwrap(); + let output = attention + .call( + ¤t, + &mut cache, + &options, + vision.then_some((None, 0)), + &streams, + gpu, + ) + .unwrap(); + outputs.push( + ops::read_f32_as_f64( + &ops::astype(&output, Dtype::F32, false, gpu).unwrap(), + &streams, + gpu, + ) + .unwrap(), + ); + } + println!( + "{}", + serde_json::json!({"case":"vision_sparse_qsa","prefix_rows":12,"topk_blocks":1,"ratio":4,"position_delta":0,"text_output":outputs[0],"vision_output":outputs[1]}) + ); + assert_ne!(outputs[0], outputs[1], "diagnosis no longer reproduces"); + let qweight = bf( + &[16, 8], + &(0..128) + .map(|i| f32::from(i / 8 < 8 && i / 8 == i % 8)) + .collect::>(), + ); + let mut gate_outputs = Vec::new(); + for gate in [0., -6.84375] { + let bias = bf(&[16], &[vec![0.; 8], vec![gate; 8]].concat()); + let mut attention = Attention { + input: Input::Separate( + [ + Linear::Dense(&qweight), + Linear::Dense(&k), + Linear::Dense(&v), + ], + [Some(&bias), None, None], + ), + output: Linear::Dense(&out), + output_bias: None, + heads: 1, + kv_heads: 1, + dim: 8, + eps: 1e-6, + scale: 1. / 8_f64.sqrt(), + q_norm: &norm, + k_norm: &norm, + inv_freq: &freq, + rope_scaling: 1., + mrope_axes: Some(&axes), + indexer: None, + }; + let output = attention + .call( + ¤t, + &mut QsaCache::new(4), + &options, + None, + &streams, + gpu, + ) + .unwrap(); + ops::evaluate(&streams, std::slice::from_ref(&output), gpu, false).unwrap(); + gate_outputs.push(output); + } + let ungated = binary(&gate_outputs[0], &bf(&[], &[2.]), Binary::Multiply, gpu).unwrap(); + let gate = bf(&[1, 1, 8], &[-6.84375; 8]); + ops::evaluate(&streams, &[ungated.clone(), gate.clone()], gpu, false).unwrap(); + let args = [ungated, gate]; + let replay = Replay::trace_compiled(&args, |a| { + binary( + &a[0], + &unary::unary( + &ops::astype( + &ops::astype(&a[1], Dtype::F32, false, gpu)?, + Dtype::BF16, + false, + gpu, + )?, + unary::Unary::Sigmoid, + gpu, + )?, + Binary::Multiply, + gpu, + ) + .map(|a| vec![a]) + }) + .unwrap(); + let expected = replay.run(&args).unwrap(); + let read = |a: &Array| { + ops::read_f32_as_f64( + &ops::astype(a, Dtype::F32, false, gpu).unwrap(), + &streams, + gpu, + ) + .unwrap() + }; + let actual = read(&gate_outputs[1]); + let expected = read(&expected[0]); + println!( + "{}", + serde_json::json!({"case":"active_attention_gate","gate":-6.84375,"actual":actual,"expected_verify":expected}) + ); + assert_ne!(actual, expected, "diagnosis no longer reproduces"); +} diff --git a/tools/issue103-reference.py b/tools/issue103-reference.py new file mode 100644 index 0000000..cb0e763 --- /dev/null +++ b/tools/issue103-reference.py @@ -0,0 +1,92 @@ +"""Model-free #103 oracle receipts. Set PYTHONPATH to the immutable MTPLX checkout. + +Run in the existing MLX 0.32.2 reference environment, passing the unchanged +Bare-Speed artifact folder. Only tokenizer assets are read; no model is loaded. +""" +import hashlib +import importlib.metadata +import json +import math +import sys +from pathlib import Path + +import mlx.core as mx +import numpy as np +from mtplx.generation import _mx_lazy_sample, _sample_from_logits +from mtplx.sampling import SamplerConfig, distribution_from_logits +from mtplx.runtime import _load_tokenizer_resilient + + +def emit(**receipt): + print(json.dumps(receipt, ensure_ascii=False), flush=True) + + +root = Path(sys.argv[1]) +emit(case="environment", mlx=importlib.metadata.version("mlx"), + device=mx.device_info(), + hashes={n: hashlib.sha256((root/n).read_bytes()).hexdigest() + for n in ["config.json", "tokenizer.json", "merges.txt", "chat_template.jinja"]}) +config = SamplerConfig(temperature=1., top_p=0.5, top_k=2) +values = np.array([math.log(.4), math.log(.3)] + [math.log(.3/62)]*62, dtype=np.float32) +p = distribution_from_logits(values.astype(np.float64), config) +emit(case="sampling_law", probabilities=p[:2].tolist()) +counts = [0]*64 +for seed in range(128): + token = _mx_lazy_sample(mx.array(values), config, mx.random.key(seed)) + mx.eval(token) + counts[int(token.item())] += 1 +emit(case="sampling_law_lazy", counts=counts[:2]) +for temperature, top_k, top_p in [(.6,20,.95),(.2,8,.8),(1.,2,1.)]: + row = values * np.float32(temperature) + config = SamplerConfig(temperature=temperature, top_k=top_k, top_p=top_p) + probabilities = distribution_from_logits(row.astype(np.float64), config) + counts = [0]*64 + for seed in range(512): + token = _mx_lazy_sample(mx.array(row), config, mx.random.key(seed)) + mx.eval(token) + counts[int(token.item())] += 1 + emit(case="sampling_matrix", temperature=temperature, top_k=top_k, top_p=top_p, + ordinary_probabilities=probabilities.tolist(), lazy_counts=counts) +for name, values in [ + ("nan", [float("nan")]*64), + ("positive_inf", [float("inf")]+[0.]*63), + ("all_negative_inf", [float("-inf")]*64), + ("valid_mask", [float("-inf"),2.,float("-inf"),1.]+[float("-inf")]*60), +]: + for temperature in [0.,.6,1.]: + config = SamplerConfig(temperature=temperature, top_p=.95, top_k=20) + try: + token, _ = _sample_from_logits(mx.array(values), config, np.random.default_rng(0)) + emit(case="invalid_logits", row=name, temperature=temperature, token=int(token)) + except (ValueError, RuntimeError) as error: + emit(case="invalid_logits", row=name, temperature=temperature, error=type(error).__name__) + token = _mx_lazy_sample(mx.array(values), SamplerConfig(temperature=1., top_k=20, top_p=.95), mx.random.key(7)) + mx.eval(token) + emit(case="invalid_logits_lazy", row=name, token=int(token.item())) + +tokenizer = _load_tokenizer_resilient(root, json.loads((root/"config.json").read_text())) +for text in [ + "भारत एक विशाल देश है जिसकी संस्कृति बहुत पुरानी है।", + "ประเทศไทยมีวัฒนธรรมที่เก่าแก่และสวยงามมาก", + "إِنَّ اللُّغَةَ الْعَرَبِيَّةَ جَمِيلَةٌ", + "The quick brown fox jumps over the lazy dog near the river bank.", + "def foo(x):\n return x + 1\n", + "中国的文化历史悠久。", + "a🙂b ©c ∑d —e …f €g ™h\n", + "x\u0301y १२३ ١٢٣ ①Ⅷ²\u200dZ", + " ①", " Ⅷ", " ²", " 𝟘", +]: + emit(case="tokenizer", text=text, ids=tokenizer.encode(text, add_special_tokens=False)) + +try: + from mtplx.attention_context import attention_phase + from mtplx.attention_math import attention_gate +except ImportError: + emit(case="bf16_verify_gate", available=False) + sys.exit(0) +gate = mx.array([-6.84375]*4, dtype=mx.bfloat16) +value = mx.ones((4,), dtype=mx.bfloat16) +with attention_phase("decode_verify"): + output = attention_gate(value, gate) + mx.eval(output) + emit(case="bf16_verify_gate", values=output.astype(mx.float32).tolist())