hugo/DS4Server
1
0 Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Match Qwen compiled verification gate math

Browse Source
This commit is contained in:
Georg Bauer committed 2026-10-04 14:50:29 +02:00
1 parent 892f8a70b9
commit 57b8ac28a7
33 files changed
+1250 -40
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/issue109-verify-gate/README.md
+86
View File
@@ -0,0 +1,86 @@
# Issue 109: compiled verification gates
Qwen target forwards inside the MTP loop now carry the pinned MTPLX
`decode_verify` phase through attention, eager HC injection, captured-prefix
repair and final-primary commit. Draft/history and ordinary prefill/AR retain
their existing phase. Compiled layer replay keys include the phase to prevent
reuse of a graph with the other gate law.
The BF16 attention gate compiles exactly
`output * sigmoid(gate.astype(f32).astype(bf16))`; the casts select the fused
lowering and do not move the sigmoid to F32. F32 uses its separate compiled
expression; FP16 remains standalone. The existing native Rust graph compiler
and Metal execution implement these expressions. No global sigmoid, compiler
math flags, DS4/GLM implementation, or shared family helper changed.
Two actual eager `GatedResidual._verify_inject` calls also demonstrated a
phase/site failure (the adjacent `active-hc-sites.jsonl` records inputs and both
outputs). That sibling therefore satisfies this issue's explicit scope rule:
its verify-only divide/sigmoid/multiply now compiles as pinned MTPLX does.
Unrelated PLE/shared-expert gates remain unchanged. This is oracle parity, not a
new DS4Server feature or a full fixed-M4 verifier.
Persisted MTP snapshots without the new gate policy are rejected before
restoration; old AR snapshots remain compatible. Tests use both a constructed
old manifest and an actual pre-fix MTP file, and verify failed restoration leaves
the current live state intact. New checkpoints reproduce resident warm state
exactly after reset/reopen.
## Verification and boundaries
The pinned oracle is MTPLX `9882703f3105363ddc37eca9f97aa09a1d387112`
with MLX 0.32.2. The installed Bare-Speed artifact and hardware are recorded in
`../issue104-ar/environment.json`.
- Exhaustive finite BF16 (65,280) and FP16 (63,488), plus 65,536 dense F32
gate values, compare every output byte in all three phases. BF16 also covers
all three HC-injection phases. The owned original `active_attention_gate`
and `bf16_gate` regressions require the corrected expression and neighbors.
- Real cold/warm MTP requests compare each target forward's input IDs and all
logits/hidden bytes, generated tokens, cache offsets, final logits/hidden and
trunk/MTP state arrays. Native resident and disk-reopened full state hashes
must agree exactly.
- The complete final-primary test covers greedy/sampled, depths 1–3,
stop/max-token termination and failure/cancellation controls.
- Existing whole-attention fixtures exercise both TF32 modes and prepared/lazy
input routes; hyper fixtures control the unchanged non-verify route.
The real-model comparison explicitly sets the oracle's
`MTPLX_QWEN4_VERIFY_SDPA_HEAD_CHUNK=0` to isolate this gate change with the same
attention consumer. Independent default-oracle comparison exposed a missing
head-chunked consumer before the gate, tracked separately in
[#113](https://git.rfc1437.de/hugo/DS4Server/issues/113). This controlled run does
not establish default-route model parity. `default-route-mismatch.jsonl`
retains the evidence.
Cache receipts retain raw hashes as well as value hashes with zero signs
canonicalized. A first-position prefill key has one signed-zero bit difference;
this is reported explicitly rather than hidden by a tolerance, and is tracked
separately in [#114](https://git.rfc1437.de/hugo/DS4Server/issues/114). Target outputs,
final logits/hidden and resident-versus-reopened state use raw byte exactness.
All four mandatory gates and serial Metal/oracle checks are listed in
`checks.json`. These are correctness receipts, not performance measurements.
## Reproduction
```sh
local-eval-results/mtplx-reference-env-0.32.2/bin/python \
tools/issue109-gate-reference.py /path/to/pinned/mtplx /tmp/issue109-gate
cargo test --all-features --bin ds4-server --no-run
cargo build --bin test-supervisor
export DS4_ISSUE109_GATE=/tmp/issue109-gate
# Resolve TEST_EXECUTABLE from cargo; run Metal processes serially.
target/debug/test-supervisor 4096 60 120 --command TEST_EXECUTABLE \
issue109_gate_phase_and_dtype_laws --ignored --nocapture --test-threads=1
export MTPLX_CONTEXT_COPY=0
export DS4_QWEN38_ARTIFACTS=/path/to/model
export DS4_ISSUE109_DIR=/tmp/issue109-product
export DS4_ISSUE109_OLD=/path/to/pre-fix-mtp-checkpoint
target/debug/test-supervisor 114688 60 120 --command TEST_EXECUTABLE \
issue109_verify_rows_and_warm_checkpoint --ignored --nocapture --test-threads=1
MTPLX_QWEN4_VERIFY_SDPA_HEAD_CHUNK=0 \
local-eval-results/mtplx-reference-env-0.32.2/bin/python \
tools/issue109-model-reference.py /path/to/pinned/mtplx \
"$DS4_QWEN38_ARTIFACTS" "$DS4_ISSUE109_DIR" /tmp/issue109-model-oracle
```
docs/issue109-verify-gate/active-hc-sites.jsonl
+2
View File
@@ -0,0 +1,2 @@
{"case":"active_sibling_gate","owner":"GatedResidual","method":"_verify_inject","phase":"decode_verify","gate_shape":[1,4,4],"different":1,"logits":[-14.5,-25.625,-27.125,-15.8125,-11.8125,-19.875,-27.375,-16.25,-12.9375,-25.75,-30.625,-17.0,-11.6875,-22.625,-26.5,-16.5],"actual":[0.052001953125,0.0033111572265625,0.0022735595703125,0.037841796875,0.099609375,0.0137939453125,0.00213623046875,0.033935546875,0.07568359375,0.003204345703125,0.000946044921875,0.0281982421875,0.10205078125,0.0069580078125,0.002655029296875,0.031982421875],"stock":[0.052001953125,0.0033111572265625,0.0022735595703125,0.037841796875,0.099609375,0.0137939453125,0.0021209716796875,0.033935546875,0.07568359375,0.003204345703125,0.000946044921875,0.0281982421875,0.10205078125,0.0069580078125,0.002655029296875,0.031982421875]}
{"case":"active_sibling_gate","owner":"GatedResidual","method":"_verify_inject","phase":"decode_verify","gate_shape":[1,4,4],"different":1,"logits":[14.4375,-5.0625,-12.4375,-1.359375,23.75,-26.25,-19.875,-8.6875,27.625,-29.125,-18.375,-8.75,27.0,-27.375,-22.625,-7.71875],"actual":[1.9453125,0.4375,0.08544921875,0.83203125,1.9921875,0.0028228759765625,0.0137939453125,0.205078125,2.0,0.001373291015625,0.02001953125,0.201171875,2.0,0.00213623046875,0.0069580078125,0.25390625],"stock":[1.9453125,0.4375,0.08544921875,0.83203125,1.9921875,0.0028228759765625,0.0137939453125,0.205078125,2.0,0.001373291015625,0.02001953125,0.201171875,2.0,0.0021209716796875,0.0069580078125,0.25390625]}
docs/issue109-verify-gate/attention.jsonl
+8
View File
@@ -0,0 +1,8 @@
{"case":"vision_positions","start":0,"rows":4,"table_end":4,"delta":-1,"different_rows":[],"actual_keys":[0.1982421875,0.396484375,0.59375,0.79296875,0.98828125,1.1875,1.3828125,1.5859375,-0.478515625,0.671875,0.85546875,0.9453125,1.1484375,1.171875,1.1875,1.2578125,-1.265625,0.515625,0.875,0.96484375,0.326171875,1.28125,1.15625,1.1640625,-1.2890625,0.55859375,0.90234375,0.9765625,0.373046875,1.265625,1.125,1.1171875],"expected_keys":[0.1982421875,0.396484375,0.59375,0.79296875,0.98828125,1.1875,1.3828125,1.5859375,-0.478515625,0.671875,0.85546875,0.9453125,1.1484375,1.171875,1.1875,1.2578125,-1.265625,0.515625,0.875,0.96484375,0.326171875,1.28125,1.15625,1.1640625,-1.2890625,0.55859375,0.90234375,0.9765625,0.373046875,1.265625,1.125,1.1171875]}
{"case":"vision_chunk_next_state","start":0,"rows":4,"offset":5,"exact":true}
{"case":"vision_positions","start":4,"rows":2,"table_end":4,"delta":-1,"different_rows":[],"actual_keys":[-0.3359375,0.02783203125,0.55078125,0.7890625,-0.94921875,1.25,1.3984375,1.5859375,0.3125,0.294921875,0.8203125,0.94140625,-1.203125,1.3203125,1.2109375,1.2578125],"expected_keys":[-0.3359375,0.02783203125,0.55078125,0.7890625,-0.94921875,1.25,1.3984375,1.5859375,0.3125,0.294921875,0.8203125,0.94140625,-1.203125,1.3203125,1.2109375,1.2578125]}
{"case":"vision_chunk_next_state","start":4,"rows":2,"offset":7,"exact":true}
{"case":"vision_positions","start":2,"rows":4,"table_end":4,"delta":-1,"different_rows":[],"actual_keys":[-0.98046875,0.02783203125,0.5390625,0.7890625,-0.2314453125,1.25,1.40625,1.5859375,-1.2265625,0.423828125,0.8203125,0.94140625,0.216796875,1.28125,1.2109375,1.2578125,-0.9609375,0.515625,0.88671875,0.96484375,-0.890625,1.28125,1.140625,1.1640625,0.1962890625,0.4296875,0.90234375,0.9765625,-1.328125,1.3203125,1.125,1.125],"expected_keys":[-0.98046875,0.02783203125,0.5390625,0.7890625,-0.2314453125,1.25,1.40625,1.5859375,-1.2265625,0.423828125,0.8203125,0.94140625,0.216796875,1.28125,1.2109375,1.2578125,-0.9609375,0.515625,0.88671875,0.96484375,-0.890625,1.28125,1.140625,1.1640625,0.1962890625,0.4296875,0.90234375,0.9765625,-1.328125,1.3203125,1.125,1.125]}
{"case":"vision_chunk_next_state","start":2,"rows":4,"offset":7,"exact":true}
{"case":"vision_sparse_qsa","prefix_rows":12,"topk_blocks":1,"ratio":4,"position_delta":0,"text_output":[0.10986328125,-0.1923828125,0.08544921875,0.498046875,-0.0791015625,0.16796875,0.31640625,-0.03173828125],"vision_output":[0.10986328125,-0.1923828125,0.08544921875,0.498046875,-0.0791015625,0.16796875,0.31640625,-0.03173828125]}
{"case":"active_attention_gate","gate":-6.84375,"actual":[0.0003204345703125,-0.0007476806640625,0.000213623046875,0.0009613037109375,-0.000213623046875,0.00042724609375,0.000640869140625,-0.0001068115234375],"expected_verify":[0.0003204345703125,-0.0007476806640625,0.000213623046875,0.0009613037109375,-0.000213623046875,0.00042724609375,0.000640869140625,-0.0001068115234375]}
docs/issue109-verify-gate/checks.json
+219
View File
@@ -0,0 +1,219 @@
[
{
"check": "fmt",
"passed": true
},
{
"check": "clippy",
"passed": true
},
{
"check": "bundle",
"passed": true
},
{
"check": "tests",
"passed": true,
"results": [
"test result: ok. 308 passed; 0 failed; 221 ignored; 0 measured; 0 filtered out; finished in 5.57s",
"test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s",
"test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.29s"
]
},
{
"check": "gate-native",
"passed": true,
"results": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 528 filtered out; finished in 0.28s"
],
"resources": {
"event": "test_resource_summary",
"elapsed_ms": 1677,
"peak": {
"cpu_time_ns": 198776708,
"resident_bytes": 82509824,
"physical_bytes": 160219976,
"peak_physical_bytes": 160744264,
"disk_read_bytes": 15327232,
"disk_write_bytes": 0
},
"error": null
}
},
{
"check": "product",
"passed": true,
"results": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 528 filtered out; finished in 47.53s"
],
"resources": {
"event": "test_resource_summary",
"elapsed_ms": 48500,
"peak": {
"cpu_time_ns": 50595900083,
"resident_bytes": 26181681152,
"physical_bytes": 77006202736,
"peak_physical_bytes": 77006202736,
"disk_read_bytes": 73689870336,
"disk_write_bytes": 286756864
},
"error": null
}
},
{
"check": "model-oracle",
"passed": true,
"control": "MTPLX_QWEN4_VERIFY_SDPA_HEAD_CHUNK=0; MTPLX_CONTEXT_COPY=0",
"known_separate_findings": [
113,
114
]
},
{
"check": "codec",
"passed": true,
"results": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 528 filtered out; finished in 0.13s"
],
"resources": {
"event": "test_resource_summary",
"elapsed_ms": 1455,
"peak": {
"cpu_time_ns": 41537833,
"resident_bytes": 17907712,
"physical_bytes": 4817472,
"peak_physical_bytes": 4817472,
"disk_read_bytes": 0,
"disk_write_bytes": 0
},
"error": null
}
},
{
"check": "attention",
"passed": true,
"results": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 528 filtered out; finished in 0.15s"
],
"resources": {
"event": "test_resource_summary",
"elapsed_ms": 210,
"peak": {
"cpu_time_ns": 85113875,
"resident_bytes": 79396864,
"physical_bytes": 40239968,
"peak_physical_bytes": 40239968,
"disk_read_bytes": 65536,
"disk_write_bytes": 0
},
"error": null
}
},
{
"check": "numeric",
"passed": true,
"results": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 528 filtered out; finished in 0.29s"
],
"resources": {
"event": "test_resource_summary",
"elapsed_ms": 315,
"peak": {
"cpu_time_ns": 88259416,
"resident_bytes": 80084992,
"physical_bytes": 159744816,
"peak_physical_bytes": 159859504,
"disk_read_bytes": 4096,
"disk_write_bytes": 0
},
"error": null
}
},
{
"check": "finalprimary",
"passed": true,
"results": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 528 filtered out; finished in 98.22s"
],
"resources": {
"event": "test_resource_summary",
"elapsed_ms": 98338,
"peak": {
"cpu_time_ns": 98248792166,
"resident_bytes": 63166431232,
"physical_bytes": 74583434072,
"peak_physical_bytes": 74584138584,
"disk_read_bytes": 73602744320,
"disk_write_bytes": 1397932032
},
"error": null
}
},
{
"check": "module-1",
"passed": true,
"results": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 528 filtered out; finished in 14.20s"
],
"resources": {
"event": "test_resource_summary",
"elapsed_ms": 14223,
"peak": {
"cpu_time_ns": 13641618583,
"resident_bytes": 548356096,
"physical_bytes": 1595852072,
"peak_physical_bytes": 1604257088,
"disk_read_bytes": 51740672,
"disk_write_bytes": 0
},
"error": null
}
},
{
"check": "module-0",
"passed": true,
"results": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 528 filtered out; finished in 14.05s"
],
"resources": {
"event": "test_resource_summary",
"elapsed_ms": 14115,
"peak": {
"cpu_time_ns": 13634514125,
"resident_bytes": 556367872,
"physical_bytes": 1602864448,
"peak_physical_bytes": 1604191552,
"disk_read_bytes": 278528,
"disk_write_bytes": 0
},
"error": null
}
},
{
"check": "hyper",
"passed": true,
"results": [
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 528 filtered out; finished in 28.54s"
],
"resources": {
"event": "test_resource_summary",
"elapsed_ms": 28607,
"peak": {
"cpu_time_ns": 28219361375,
"resident_bytes": 207831040,
"physical_bytes": 416056280,
"peak_physical_bytes": 416089048,
"disk_read_bytes": 1949696,
"disk_write_bytes": 0
},
"error": null
}
},
{
"check": "upstream",
"passed": true,
"results": [
"9 passed in 0.12s"
]
}
]
docs/issue109-verify-gate/cold-native-arrays.json
+1
View File
@@ -0,0 +1 @@
{"trunk":[[{"shape":[1,3,10240],"sha256":"1dadce7f74c7b48ad423170b8884bf5024c3d6dd91b2f9d800afca7d32679d7b","value_sha256":"1dadce7f74c7b48ad423170b8884bf5024c3d6dd91b2f9d800afca7d32679d7b"},{"shape":[1,48,128,128],"sha256":"d7436faf9d50e278f7fa095e9c217467020a6f534a48195775c212caa0ece52d","value_sha256":"d7436faf9d50e278f7fa095e9c217467020a6f534a48195775c212caa0ece52d"}],[{"shape":[1,3,10240],"sha256":"ca1b3cf93598b62575f01aea05ff0d0f087c00fdf1b63654626ef5ec3487715a","value_sha256":"ca1b3cf93598b62575f01aea05ff0d0f087c00fdf1b63654626ef5ec3487715a"},{"shape":[1,48,128,128],"sha256":"f1364bb8165b9189279b7769e4081f213c562f4e0541acc5962cf2ae9f42a831","value_sha256":"f1364bb8165b9189279b7769e4081f213c562f4e0541acc5962cf2ae9f42a831"},{"shape":[1,9,10240],"sha256":"c9110c820fc39f8c19c770594c15e0997693015090a3c7af4b6c8ec9a31ebc4e","value_sha256":"c9110c820fc39f8c19c770594c15e0997693015090a3c7af4b6c8ec9a31ebc4e"},{"shape":[1,2],"sha256":"9febaa06152dac026dba5f2c1151c11ff6a1b2f00e92278cdfd2905c87cf2c2f","value_sha256":"9febaa06152dac026dba5f2c1151c11ff6a1b2f00e92278cdfd2905c87cf2c2f"}],[{"shape":[1,3,10240],"sha256":"320fdcf9c7ab899133952e093d1f3de613ba5581b3aaad45c4ad7b1b4e43f1f3","value_sha256":"320fdcf9c7ab899133952e093d1f3de613ba5581b3aaad45c4ad7b1b4e43f1f3"},{"shape":[1,48,128,128],"sha256":"8cae4ba20fd0e54f8da130e5f8681ece7c54672b7a6404800cded8e64a977e08","value_sha256":"8cae4ba20fd0e54f8da130e5f8681ece7c54672b7a6404800cded8e64a977e08"}],[{"shape":[1,2,58,256],"sha256":"404510256f371f0bc2e7b8af2d1f7eb2ae0322d42eb9a0ee5827a9a5c8d489c0","value_sha256":"404510256f371f0bc2e7b8af2d1f7eb2ae0322d42eb9a0ee5827a9a5c8d489c0"},{"shape":[1,2,58,256],"sha256":"f14afc7fe1c5e4c62c84121e804e14aa484db378861904d4e234ffeef10252ef","value_sha256":"f14afc7fe1c5e4c62c84121e804e14aa484db378861904d4e234ffeef10252ef"},{"shape":[1,58,128],"sha256":"ca2d0b2f09e87fab8d0a6aede7073ec53bc37b584851f0342c76244f05ca3e12","value_sha256":"ca2d0b2f09e87fab8d0a6aede7073ec53bc37b584851f0342c76244f05ca3e12"},{"shape":[1,14,128],"sha256":"a609a4f52c9b3dc0fb5d6554019786bebecdad220bf5ae6b40d4939821f12c8b","value_sha256":"a609a4f52c9b3dc0fb5d6554019786bebecdad220bf5ae6b40d4939821f12c8b"}],[{"shape":[1,3,10240],"sha256":"0c5bdce8d9cbd2a24162139582f1889f0413632feafdba4e79cdd88353e6abc2","value_sha256":"0c5bdce8d9cbd2a24162139582f1889f0413632feafdba4e79cdd88353e6abc2"},{"shape":[1,48,128,128],"sha256":"d47c0c965fb6c41a7e1824144ef61546a7e40ead8d8606822173e21a17a4fc0f","value_sha256":"d47c0c965fb6c41a7e1824144ef61546a7e40ead8d8606822173e21a17a4fc0f"}],[{"shape":[1,3,10240],"sha256":"66014c1bce9cbbd223ec6af2f5517de66f71ce443721088167dad0c0186289fb","value_sha256":"66014c1bce9cbbd223ec6af2f5517de66f71ce443721088167dad0c0186289fb"},{"shape":[1,48,128,128],"sha256":"2593987c9ca9aeb30910e7eb69f8d6bd61c1e035861915cbe27b0838e26f3098","value_sha256":"2593987c9ca9aeb30910e7eb69f8d6bd61c1e035861915cbe27b0838e26f3098"}],[{"shape":[1,3,10240],"sha256":"45a3dbbf3dc7a79b7db631fe4c2802d6e1f4433f4b03e961df072147e8c242a2","value_sha256":"45a3dbbf3dc7a79b7db631fe4c2802d6e1f4433f4b03e961df072147e8c242a2"},{"shape":[1,48,128,128],"sha256":"e266e3456f01268de8868258a750ee18c5f8b1ff4391f647aec60a39d56f98a0","value_sha256":"e266e3456f01268de8868258a750ee18c5f8b1ff4391f647aec60a39d56f98a0"}],[{"shape":[1,2,58,256],"sha256":"ba60b2860a5717f46a4a7ffef0257799f96acb37de1a00ba88ddfc149bf44ca2","value_sha256":"ba60b2860a5717f46a4a7ffef0257799f96acb37de1a00ba88ddfc149bf44ca2"},{"shape":[1,2,58,256],"sha256":"57d040a722a1427e9b0744ee652aecca9ec30cc7b617e3db9534e772d79f42e4","value_sha256":"57d040a722a1427e9b0744ee652aecca9ec30cc7b617e3db9534e772d79f42e4"},{"shape":[1,58,128],"sha256":"84d339a2f8558893f3d73e751d5ee3d81bb1f3b1184c0b35555bf5456fdc5a22","value_sha256":"84d339a2f8558893f3d73e751d5ee3d81bb1f3b1184c0b35555bf5456fdc5a22"},{"shape":[1,14,128],"sha256":"a5378133f448c4b6380dc4711420d3b069dd8491ffc797fd16e023294405b707","value_sha256":"a5378133f448c4b6380dc4711420d3b069dd8491ffc797fd16e023294405b707"}],[{"shape":[1,3,10240],"sha256":"e6d4afab81c6a58f08c8d252991e91d25d2d60e497c3fc00614451edd0ea908c","value_sha256":"e6d4afab81c6a58f08c8d252991e91d25d2d60e497c3fc00614451edd0ea908c"},{"shape":[1,48,128,128],"sha256":"f94349b9898b0d1a5770f4aca791d4c114abad341641688012de2e2fcaa6237c","value_sha256":"f94349b9898b0d1a5770f4aca791d4c114abad341641688012de2e2fcaa6237c"}],[{"shape":[1,3,10240],"sha256":"54d2e63c68f9fc1412178f3f21bcf5897ab228967e01f709ed9d5ba0e6d0c016","value_sha256":"54d2e63c68f9fc1412178f3f21bcf5897ab228967e01f709ed9d5ba0e6d0c016"},{"shape":[1,48,128,128],"sha256":"868155f0fd6a62a46decb4f87c3e2f27310f9da38796fe0a8d926844248c7259","value_sha256":"868155f0fd6a62a46decb4f87c3e2f27310f9da38796fe0a8d926844248c7259"}],[{"shape":[1,3,10240],"sha256":"0073cc8754020915aedaad03803669a61281e18cb2dd389cb19ef7b5ed769b84","value_sha256":"0073cc8754020915aedaad03803669a61281e18cb2dd389cb19ef7b5ed769b84"},{"shape":[1,48,128,128],"sha256":"abd539c013e58f489c23c48dfc168be7a9e2145deee859432cb83d0dLine truncated
docs/issue109-verify-gate/cold-oracle-arrays.json
+1
View File
@@ -0,0 +1 @@
{"trunk": [[{"shape": [1, 3, 10240], "sha256": "1dadce7f74c7b48ad423170b8884bf5024c3d6dd91b2f9d800afca7d32679d7b", "value_sha256": "1dadce7f74c7b48ad423170b8884bf5024c3d6dd91b2f9d800afca7d32679d7b"}, {"shape": [1, 48, 128, 128], "sha256": "d7436faf9d50e278f7fa095e9c217467020a6f534a48195775c212caa0ece52d", "value_sha256": "d7436faf9d50e278f7fa095e9c217467020a6f534a48195775c212caa0ece52d"}], [{"shape": [1, 3, 10240], "sha256": "ca1b3cf93598b62575f01aea05ff0d0f087c00fdf1b63654626ef5ec3487715a", "value_sha256": "ca1b3cf93598b62575f01aea05ff0d0f087c00fdf1b63654626ef5ec3487715a"}, {"shape": [1, 48, 128, 128], "sha256": "f1364bb8165b9189279b7769e4081f213c562f4e0541acc5962cf2ae9f42a831", "value_sha256": "f1364bb8165b9189279b7769e4081f213c562f4e0541acc5962cf2ae9f42a831"}, {"shape": [1, 9, 10240], "sha256": "c9110c820fc39f8c19c770594c15e0997693015090a3c7af4b6c8ec9a31ebc4e", "value_sha256": "c9110c820fc39f8c19c770594c15e0997693015090a3c7af4b6c8ec9a31ebc4e"}, {"shape": [1, 2], "sha256": "9febaa06152dac026dba5f2c1151c11ff6a1b2f00e92278cdfd2905c87cf2c2f", "value_sha256": "9febaa06152dac026dba5f2c1151c11ff6a1b2f00e92278cdfd2905c87cf2c2f"}], [{"shape": [1, 3, 10240], "sha256": "320fdcf9c7ab899133952e093d1f3de613ba5581b3aaad45c4ad7b1b4e43f1f3", "value_sha256": "320fdcf9c7ab899133952e093d1f3de613ba5581b3aaad45c4ad7b1b4e43f1f3"}, {"shape": [1, 48, 128, 128], "sha256": "8cae4ba20fd0e54f8da130e5f8681ece7c54672b7a6404800cded8e64a977e08", "value_sha256": "8cae4ba20fd0e54f8da130e5f8681ece7c54672b7a6404800cded8e64a977e08"}], [{"shape": [1, 2, 58, 256], "sha256": "404510256f371f0bc2e7b8af2d1f7eb2ae0322d42eb9a0ee5827a9a5c8d489c0", "value_sha256": "404510256f371f0bc2e7b8af2d1f7eb2ae0322d42eb9a0ee5827a9a5c8d489c0"}, {"shape": [1, 2, 58, 256], "sha256": "f14afc7fe1c5e4c62c84121e804e14aa484db378861904d4e234ffeef10252ef", "value_sha256": "f14afc7fe1c5e4c62c84121e804e14aa484db378861904d4e234ffeef10252ef"}, {"shape": [1, 58, 128], "sha256": "ca2d0b2f09e87fab8d0a6aede7073ec53bc37b584851f0342c76244f05ca3e12", "value_sha256": "ca2d0b2f09e87fab8d0a6aede7073ec53bc37b584851f0342c76244f05ca3e12"}, {"shape": [1, 14, 128], "sha256": "a609a4f52c9b3dc0fb5d6554019786bebecdad220bf5ae6b40d4939821f12c8b", "value_sha256": "a609a4f52c9b3dc0fb5d6554019786bebecdad220bf5ae6b40d4939821f12c8b"}], [{"shape": [1, 3, 10240], "sha256": "0c5bdce8d9cbd2a24162139582f1889f0413632feafdba4e79cdd88353e6abc2", "value_sha256": "0c5bdce8d9cbd2a24162139582f1889f0413632feafdba4e79cdd88353e6abc2"}, {"shape": [1, 48, 128, 128], "sha256": "d47c0c965fb6c41a7e1824144ef61546a7e40ead8d8606822173e21a17a4fc0f", "value_sha256": "d47c0c965fb6c41a7e1824144ef61546a7e40ead8d8606822173e21a17a4fc0f"}], [{"shape": [1, 3, 10240], "sha256": "66014c1bce9cbbd223ec6af2f5517de66f71ce443721088167dad0c0186289fb", "value_sha256": "66014c1bce9cbbd223ec6af2f5517de66f71ce443721088167dad0c0186289fb"}, {"shape": [1, 48, 128, 128], "sha256": "2593987c9ca9aeb30910e7eb69f8d6bd61c1e035861915cbe27b0838e26f3098", "value_sha256": "2593987c9ca9aeb30910e7eb69f8d6bd61c1e035861915cbe27b0838e26f3098"}], [{"shape": [1, 3, 10240], "sha256": "45a3dbbf3dc7a79b7db631fe4c2802d6e1f4433f4b03e961df072147e8c242a2", "value_sha256": "45a3dbbf3dc7a79b7db631fe4c2802d6e1f4433f4b03e961df072147e8c242a2"}, {"shape": [1, 48, 128, 128], "sha256": "e266e3456f01268de8868258a750ee18c5f8b1ff4391f647aec60a39d56f98a0", "value_sha256": "e266e3456f01268de8868258a750ee18c5f8b1ff4391f647aec60a39d56f98a0"}], [{"shape": [1, 2, 58, 256], "sha256": "ba60b2860a5717f46a4a7ffef0257799f96acb37de1a00ba88ddfc149bf44ca2", "value_sha256": "ba60b2860a5717f46a4a7ffef0257799f96acb37de1a00ba88ddfc149bf44ca2"}, {"shape": [1, 2, 58, 256], "sha256": "57d040a722a1427e9b0744ee652aecca9ec30cc7b617e3db9534e772d79f42e4", "value_sha256": "57d040a722a1427e9b0744ee652aecca9ec30cc7b617e3db9534e772d79f42e4"}, {"shape": [1, 58, 128], "sha256": "84d339a2f8558893f3d73e751d5ee3d81bb1f3b1184c0b35555bf5456fdc5a22", "value_sha256": "84d339a2f8558893f3d73e751d5ee3d81bb1f3b1184c0b35555bf5456fdc5a22"}, {"shape": [1, 14, 128], "sha256": "a5378133f448c4b6380dc4711420d3b069dd8491ffc797fd16e023294405b707", "value_sha256": "a5378133f448c4b6380dc4711420d3b069dd8491ffc797fd16e023294405b707"}], [{"shape": [1, 3, 10240], "sha256": "e6d4afab81c6a58f08c8d252991e91d25d2d60e497c3fc00614451edd0ea908c", "value_sha256": "e6d4afab81c6a58f08c8d252991e91d25d2d60e497c3fc00614451edd0ea908c"}, {"shape": [1, 48, 128, 128], "sha256": "f94349b9898b0d1a5770f4aca791d4c114abad341641688012de2e2fcaa6237c", "value_sha256": "f94349b9898b0d1a5770f4aca791d4c114abad341641688012de2e2fcaa6237c"}], [{"shape": [1, 3, 10240], "sha256": "54d2e63c68f9fc1412178f3f21bcf5897ab228967e01f709ed9d5ba0e6d0c016", "value_sha256": "54d2e63c68f9fc1412178f3f21bcf5897ab228967e01f709ed9d5ba0e6d0c016"}, {"shape": [1, 48, 128, 128], "sha256": "868155f0fd6a62a46decb4f87c3e2f27310f9da38796fe0a8d926844248c7259", "value_sha256": "868155f0fd6a62a46decb4f87c3e2f27310f9da38796fe0a8d926844248c7259"}], [{"shape": [1, 3, 10240], "sha256": "0073cc87540209Line truncated
docs/issue109-verify-gate/default-route-mismatch.jsonl
+142
View File
@@ -0,0 +1,142 @@
{"case":"prefill_compare","phase":"cold","part":"logits","shape":[1,1,248320],"exact":true}
{"case":"prefill_compare","phase":"cold","part":"hidden","shape":[1,1,10240],"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":0,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1010,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1011,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1013,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1014,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1015,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":2,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1030,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1031,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1036,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1037,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1038,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1032,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1033,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1034,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1035,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":3,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":6,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1070,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1071,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1076,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1077,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1078,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1072,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1073,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1074,"exact":true}
{"case":"layer","phase":"cold","index":0,"layer":1075,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":7,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":10,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1110,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1111,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1116,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1117,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1118,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1112,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1113,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1114,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1115,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":11,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":14,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1150,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1151,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1156,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1157,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1158,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1152,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1153,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1154,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1155,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":15,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":18,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1190,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1191,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1196,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1197,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1198,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1192,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1193,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1194,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1195,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":19,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":22,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1230,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1231,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1236,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1237,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1238,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1232,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1233,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1234,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1235,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":23,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":26,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1270,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1271,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1276,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1277,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1278,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1272,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1273,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1274,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1275,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":27,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":30,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1310,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1311,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1316,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1317,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1318,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1312,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1313,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1314,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1315,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":31,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":34,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1350,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1351,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1356,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1357,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1358,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1352,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1353,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1354,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1355,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":35,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":38,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1390,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1391,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1396,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1397,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1398,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1392,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1393,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1394,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1395,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":39,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":42,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1430,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1431,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1436,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1437,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1438,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1432,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1433,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1434,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1435,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":43,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":46,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1470,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1471,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1476,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1477,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1478,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1472,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1473,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1474,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":1475,"exact":false}
{"case":"layer","phase":"cold","index":0,"layer":47,"exact":false}
{"case":"verify_forward","phase":"cold","index":0,"ids":[760,1156,369,9859],"logits_exact":false,"logits_sha256":"e3ba038c8570911a72fadeba867f1c33e5103108ef9881bfcdc008907de90839","logits_different":948664,"logits_max_abs":0.828125,"hidden_exact":false,"hidden_sha256":"b3c2ef7bb9c3dfa85748d7af883210ae4393bd1b5af5397f243d632fc437552b","hidden_different":39596,"hidden_max_abs":0.53125}
docs/issue109-verify-gate/final-primary.jsonl
+15
View File
@@ -0,0 +1,15 @@
{"case":"complete_state","depth":1,"temperature":0.0,"exit":"max_tokens","tokens":[11],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"9c41c06e776592492d6b69f6d20bad4de6c99e30c9abea1333f667755a40c49c","next_sha256":"1068b9e791272fda13d0c00cd623f7fb14807753f8536b92d9d688f7864b4f3a","old_checkpoint_rejected":true}
{"case":"complete_state","depth":1,"temperature":0.0,"exit":"stop","tokens":[11],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"9c41c06e776592492d6b69f6d20bad4de6c99e30c9abea1333f667755a40c49c","next_sha256":"1068b9e791272fda13d0c00cd623f7fb14807753f8536b92d9d688f7864b4f3a","old_checkpoint_rejected":true}
{"case":"complete_state","depth":1,"temperature":0.6,"exit":"max_tokens","tokens":[314],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"06e36aaaee43822c7fc71fd1ed554fb4cd5ab92a94dd88218894f25d56f69b89","next_sha256":"e819026b44ad53bcee190e9121bf347e1f9825171ad5c7e97a051062f2cf91b4","old_checkpoint_rejected":true}
{"case":"complete_state","depth":1,"temperature":0.6,"exit":"stop","tokens":[314],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"06e36aaaee43822c7fc71fd1ed554fb4cd5ab92a94dd88218894f25d56f69b89","next_sha256":"e819026b44ad53bcee190e9121bf347e1f9825171ad5c7e97a051062f2cf91b4","old_checkpoint_rejected":true}
{"case":"complete_state","depth":2,"temperature":0.0,"exit":"max_tokens","tokens":[11],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"9c41c06e776592492d6b69f6d20bad4de6c99e30c9abea1333f667755a40c49c","next_sha256":"1068b9e791272fda13d0c00cd623f7fb14807753f8536b92d9d688f7864b4f3a","old_checkpoint_rejected":true}
{"case":"complete_state","depth":2,"temperature":0.0,"exit":"stop","tokens":[11],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"9c41c06e776592492d6b69f6d20bad4de6c99e30c9abea1333f667755a40c49c","next_sha256":"1068b9e791272fda13d0c00cd623f7fb14807753f8536b92d9d688f7864b4f3a","old_checkpoint_rejected":true}
{"case":"complete_state","depth":2,"temperature":0.6,"exit":"max_tokens","tokens":[314],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"06e36aaaee43822c7fc71fd1ed554fb4cd5ab92a94dd88218894f25d56f69b89","next_sha256":"e819026b44ad53bcee190e9121bf347e1f9825171ad5c7e97a051062f2cf91b4","old_checkpoint_rejected":true}
{"case":"complete_state","depth":2,"temperature":0.6,"exit":"stop","tokens":[314],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"06e36aaaee43822c7fc71fd1ed554fb4cd5ab92a94dd88218894f25d56f69b89","next_sha256":"e819026b44ad53bcee190e9121bf347e1f9825171ad5c7e97a051062f2cf91b4","old_checkpoint_rejected":true}
{"case":"complete_state","depth":3,"temperature":0.0,"exit":"max_tokens","tokens":[11],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"9c41c06e776592492d6b69f6d20bad4de6c99e30c9abea1333f667755a40c49c","next_sha256":"1068b9e791272fda13d0c00cd623f7fb14807753f8536b92d9d688f7864b4f3a","old_checkpoint_rejected":true}
{"case":"complete_state","depth":3,"temperature":0.0,"exit":"stop","tokens":[11],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"9c41c06e776592492d6b69f6d20bad4de6c99e30c9abea1333f667755a40c49c","next_sha256":"1068b9e791272fda13d0c00cd623f7fb14807753f8536b92d9d688f7864b4f3a","old_checkpoint_rejected":true}
{"case":"complete_state","depth":3,"temperature":0.6,"exit":"max_tokens","tokens":[314],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"06e36aaaee43822c7fc71fd1ed554fb4cd5ab92a94dd88218894f25d56f69b89","next_sha256":"e819026b44ad53bcee190e9121bf347e1f9825171ad5c7e97a051062f2cf91b4","old_checkpoint_rejected":true}
{"case":"complete_state","depth":3,"temperature":0.6,"exit":"stop","tokens":[314],"key_length":4,"trunk_offsets":[4,4,4,4,4,4,4,4,4,4,4,4],"mtp_offset":3,"reopened_cached_tokens":4,"state_sha256":"06e36aaaee43822c7fc71fd1ed554fb4cd5ab92a94dd88218894f25d56f69b89","next_sha256":"e819026b44ad53bcee190e9121bf347e1f9825171ad5c7e97a051062f2cf91b4","old_checkpoint_rejected":true}
{"case":"cancel","safe_final_state":false}
{"case":"forward","safe_final_state":false}
{"case":"capture_disabled","safe_final_state":false}
docs/issue109-verify-gate/gate-native.jsonl
+12
View File
@@ -0,0 +1,12 @@
{"case":"gate","dtype":"bf16","phase":"prefill","values":65280,"exact":true}
{"case":"inject","phase":"prefill","values":65280,"exact":true}
{"case":"gate","dtype":"bf16","phase":"ar_decode","values":65280,"exact":true}
{"case":"inject","phase":"ar_decode","values":65280,"exact":true}
{"case":"gate","dtype":"bf16","phase":"decode_verify","values":65280,"exact":true}
{"case":"inject","phase":"decode_verify","values":65280,"exact":true}
{"case":"gate","dtype":"f16","phase":"prefill","values":63488,"exact":true}
{"case":"gate","dtype":"f16","phase":"ar_decode","values":63488,"exact":true}
{"case":"gate","dtype":"f16","phase":"decode_verify","values":63488,"exact":true}
{"case":"gate","dtype":"f32","phase":"prefill","values":65536,"exact":true}
{"case":"gate","dtype":"f32","phase":"ar_decode","values":65536,"exact":true}
{"case":"gate","dtype":"f32","phase":"decode_verify","values":65536,"exact":true}
docs/issue109-verify-gate/gate-oracle.jsonl
+12
View File
@@ -0,0 +1,12 @@
{"case":"gate","dtype":"bf16","phase":"prefill","count":65280,"sha256":"4278d0e794d68fdac18c7528d38c1f6ea2c9412117aaba69f413caddb148b252"}
{"case":"inject","dtype":"bf16","phase":"prefill","count":65280,"sha256":"253aee360f465fb04a5aae286344a5344938fbcacdf6f5d0be2a66ad566378af"}
{"case":"gate","dtype":"bf16","phase":"ar_decode","count":65280,"sha256":"4278d0e794d68fdac18c7528d38c1f6ea2c9412117aaba69f413caddb148b252"}
{"case":"inject","dtype":"bf16","phase":"ar_decode","count":65280,"sha256":"253aee360f465fb04a5aae286344a5344938fbcacdf6f5d0be2a66ad566378af"}
{"case":"gate","dtype":"bf16","phase":"decode_verify","count":65280,"sha256":"deba612e555c81b44cb06c0bdfcaf51a9beaee1ec813215d91f762a0a0f7b88c"}
{"case":"inject","dtype":"bf16","phase":"decode_verify","count":65280,"sha256":"3ce25d59ffa6aff2992424fbd04138083ce1326ed4deacdd97038e36ca9648aa"}
{"case":"gate","dtype":"f16","phase":"prefill","count":63488,"sha256":"4348e497cfa1e2d4cc5bbc56a068237bb57b8a4e644145c1eb0feefa94790154"}
{"case":"gate","dtype":"f16","phase":"ar_decode","count":63488,"sha256":"4348e497cfa1e2d4cc5bbc56a068237bb57b8a4e644145c1eb0feefa94790154"}
{"case":"gate","dtype":"f16","phase":"decode_verify","count":63488,"sha256":"4348e497cfa1e2d4cc5bbc56a068237bb57b8a4e644145c1eb0feefa94790154"}
{"case":"gate","dtype":"f32","phase":"prefill","count":65536,"sha256":"2c5e478f548c9bf676e1fdea6501484016f0555b0ae1383a962436c0b68608cd"}
{"case":"gate","dtype":"f32","phase":"ar_decode","count":65536,"sha256":"2c5e478f548c9bf676e1fdea6501484016f0555b0ae1383a962436c0b68608cd"}
{"case":"gate","dtype":"f32","phase":"decode_verify","count":65536,"sha256":"2d2d76ac7e415cf9c9ce1bd67fc26b03ea0a62a20ba91387a609e16ee6636050"}
docs/issue109-verify-gate/model-oracle.jsonl
+33
View File
@@ -0,0 +1,33 @@
{"case":"environment","overrides":{"MTPLX_QWEN4_VERIFY_SDPA_HEAD_CHUNK":"0","MTPLX_SKIP_VERIFY_SNAPSHOT":"0","MTPLX_FAMILY_CAPTURE_COMMIT":"1","MTPLX_FUSED_HC_V3":"1","MTPLX_FUSED_GDN_INPROJ":"1","MTPLX_FUSED_GATE_UP":"1","MTPLX_FUSED_GDN_CONVNORM":"1","MTPLX_FUSED_GDN_STEP":"1","MTPLX_FUSED_CONVNORM_VERIFY":"1","MTPLX_QSA_GATHER":"1","MTPLX_COMPILED_GDN":"1","MTPLX_QWEN4EXP_COMPILE":"1","MTPLX_QWEN4_BATCHED_TARGET_DISTRIBUTIONS":"1","MTPLX_QWEN4_FIXED_M4_VERIFY":"1","MTPLX_QWEN4_COMPILED_MTP_PREPARE":"1","MTPLX_QWEN4_RELAXED_DRAFT_TIES":"1","MTPLX_QWEN4_OPDIET":"1","MTPLX_QWEN4_PLE_PREFILL_LOOKAHEAD":"1","MTPLX_QWEN4_PLE_FIRST_GATHER_EARLY":"1","MTPLX_SESSION_BANK_SHED_BOUNDARIES":"1","MTPLX_SESSION_BANK_PROTECTED_TERMINAL":"1","MTPLX_NGRAM_PREWARM":"auto","MTPLX_QWEN4_VERIFY_GLUE":"1","MTPLX_QWEN4_VERIFY_GLUE_ITEMS":"qsa_rope,qsa_rope_idx","MTPLX_QSA_M4_FUSED_KV_GATHER":"1","MTPLX_QSA_GATHER_MAX_ROWS":"32","MTPLX_BATCH_TARGET_ARRAYS":"1","MTPLX_LAZY_TARGET_DISTRIBUTIONS":"0","MTPLX_COMPILED_VERIFY":"1","MTPLX_NAX_VERIFY":"0","MTPLX_QWEN4_PREFILL_WIDE_CHUNK":"4096","MTPLX_QSA_PREFILL_WIDE_MIN_CONTEXT":"16384","MTPLX_SUSTAINED_PREFILL":"1","MTPLX_PREFILL_CHUNK_SIZE":"2048","MTPLX_CONTEXT_COPY":"0"}}
{"case":"prefill_compare","phase":"cold","part":"logits","shape":[1,1,248320],"exact":true}
{"case":"prefill_compare","phase":"cold","part":"hidden","shape":[1,1,10240],"exact":true}
{"case":"tokens","phase":"cold","ids":[760]}
{"case":"verify_forward","phase":"cold","index":0,"ids":[760,1156,369,9859],"logits_exact":true,"logits_sha256":"16ff543d6e28cdbe7dbd56891e061ddd4b04291a9ca25bdf1f4437b4e3c5528d","hidden_exact":true,"hidden_sha256":"b9cc3aa8c913e508793b7486f77a10e7acd4698436060b1b6bf8abfbd59edee6"}
{"case":"tokens","phase":"cold","ids":[1156]}
{"case":"tokens","phase":"cold","ids":[6587]}
{"case":"verify_forward","phase":"cold","index":1,"ids":[6587,728,310,1698],"logits_exact":true,"logits_sha256":"24e58b3561aac59d52bf55183bc9b911432481e9613a5b98f1442945bfd6903a","hidden_exact":true,"hidden_sha256":"eddb5de99ea61034c3f0eb46ef1a515cae10c92333d069afce68cc4a5e5f9e3e"}
{"case":"tokens","phase":"cold","ids":[728,310,1698]}
{"case":"tokens","phase":"cold","ids":[494]}
{"case":"verify_forward","phase":"cold","index":2,"ids":[494,799,310,5600],"logits_exact":true,"logits_sha256":"f5f2c33a36fce4088f222ec7448389865dd74ea4c017e4e2e49f157693a33d64","hidden_exact":true,"hidden_sha256":"330d68ed62e77a12e38fe7677ffd2b9548f655c7d33e6e1607e431d1b8957e86"}
{"case":"tokens","phase":"cold","ids":[799,310,5600]}
{"case":"tokens","phase":"cold","ids":[13]}
{"case":"verify_forward","phase":"cold","index":3,"ids":[13,1061],"logits_exact":true,"logits_sha256":"f44498bbf6e6e9d8d909c0e4db24b5ef9bda5866b4a6a14e188865432538643c","hidden_exact":true,"hidden_sha256":"dec8b787a562878d091c854235120732e98f5781751b6d6a428e5cd02228deb0"}
{"case":"tokens","phase":"cold","ids":[1061]}
{"case":"cache_arrays","phase":"cold","values_exact":true,"zero_sign_differences":[["trunk",35,0]],"key35_bit_coordinates":[[0,1,0,2]]}
{"case":"turn","phase":"cold","tokens":[760,1156,6587,728,310,1698,494,799,310,5600,13,1061],"cached":0,"verify_calls":4,"final_exact":true}
{"case":"prefill_compare","phase":"warm","part":"logits","shape":[1,1,248320],"exact":true}
{"case":"prefill_compare","phase":"warm","part":"hidden","shape":[1,18,10240],"exact":false}
{"case":"tokens","phase":"warm","ids":[760]}
{"case":"verify_forward","phase":"warm","index":0,"ids":[760,1156,6587,728],"logits_exact":true,"logits_sha256":"dbab0d12f25f2be2234d58d45f86219af62deb20e22b0d73a77aa5e7b722d082","hidden_exact":true,"hidden_sha256":"8f9aebc38d1a3e550bed6a4663eedac7a5ed3ce61d836d1cecdfa11da67ebd0d"}
{"case":"tokens","phase":"warm","ids":[1156,6587,728]}
{"case":"tokens","phase":"warm","ids":[310]}
{"case":"verify_forward","phase":"warm","index":1,"ids":[310,2962,24217,494],"logits_exact":true,"logits_sha256":"d6b8e6afe3c6f5af068f240e0e69b3875311aac6929b3d5a38ab3aeaa1bf227c","hidden_exact":true,"hidden_sha256":"52f20217ed43d8e59ddbe037cb77f56cc3d8ef473fd1963948fff6faf8ad4212"}
{"case":"tokens","phase":"warm","ids":[2962,24217]}
{"case":"tokens","phase":"warm","ids":[13]}
{"case":"verify_forward","phase":"warm","index":2,"ids":[13,353,2582,3804],"logits_exact":true,"logits_sha256":"048e9343d4268d98455a01f7c73ee47af78bc811ef9e7680a72cae59336b9989","hidden_exact":true,"hidden_sha256":"8e8b07fb20baa33ad64cc76ff64403a434d520c627a182a224cf033a5a14e5d4"}
{"case":"tokens","phase":"warm","ids":[353]}
{"case":"tokens","phase":"warm","ids":[8334]}
{"case":"verify_forward","phase":"warm","index":3,"ids":[8334,29014,494],"logits_exact":true,"logits_sha256":"82d671c6ac7b3a0952e60ef554ca670562fd01ef25c37bd10bc27025768ea0f2","hidden_exact":true,"hidden_sha256":"d1bc90f284364eae8bac80a5da6ab64c9c76d6994bb09ed463a17d67e25c016e"}
{"case":"tokens","phase":"warm","ids":[29014,494]}
{"case":"cache_arrays","phase":"warm","values_exact":true,"zero_sign_differences":[["trunk",35,0]],"key35_bit_coordinates":[[0,1,0,2]]}
{"case":"turn","phase":"warm","tokens":[760,1156,6587,728,310,2962,24217,13,353,8334,29014,494],"cached":58,"verify_calls":4,"final_exact":true}
docs/issue109-verify-gate/numeric.jsonl
+3
View File
@@ -0,0 +1,3 @@
{"case":"bf16_gate","gate":-6.84375,"eager":[0.00106048583984375,0.00106048583984375,0.00106048583984375,0.00106048583984375],"compiled":[0.001068115234375,0.001068115234375,0.001068115234375,0.001068115234375]}
{"case":"qsa_divisor","operand_bits":1093993715,"eager":[0.13258253037929535,0.13258254528045654,0.0883883461356163,0.0],"compiled":[0.13258251547813416,0.13258251547813416,0.08838833123445511,0.0]}
{"case":"yarn_amplitude","operand_bits":1066516124,"eager":[1.7079442739486694,1.707944393157959,1.13862943649292,0.0],"compiled":[1.7079435586929321,1.7079436779022217,1.1386289596557617,0.0]}
docs/issue109-verify-gate/prefill-zero-sign.json
+21
View File
@@ -0,0 +1,21 @@
{
"case": "prefill_zero_sign",
"shape": [
1,
2,
46,
256
],
"layer": 35,
"coordinate": [
0,
1,
0,
2
],
"native_bits": "0x80000000",
"oracle_bits": "0x0",
"native_sha256": "d81b466dea1ff5b774e86c0af47069f555bbb59abb0f3f61165f5bc51aa28ac6",
"oracle_sha256": "520dd44032eda4789b123296b5499afb88f0d88d6cdccffa04ebe06c2bc3a853",
"issue": 114
}
docs/issue109-verify-gate/product.jsonl
+16
View File
@@ -0,0 +1,16 @@
{"case":"verify_forward","record":{"index":0,"ids":[760,1156,369,9859],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"16ff543d6e28cdbe7dbd56891e061ddd4b04291a9ca25bdf1f4437b4e3c5528d"},"hidden":{"shape":[1,4,10240],"sha256":"b9cc3aa8c913e508793b7486f77a10e7acd4698436060b1b6bf8abfbd59edee6"}}}
{"case":"verify_forward","record":{"index":1,"ids":[6587,728,310,1698],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"24e58b3561aac59d52bf55183bc9b911432481e9613a5b98f1442945bfd6903a"},"hidden":{"shape":[1,4,10240],"sha256":"eddb5de99ea61034c3f0eb46ef1a515cae10c92333d069afce68cc4a5e5f9e3e"}}}
{"case":"verify_forward","record":{"index":2,"ids":[494,799,310,5600],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"f5f2c33a36fce4088f222ec7448389865dd74ea4c017e4e2e49f157693a33d64"},"hidden":{"shape":[1,4,10240],"sha256":"330d68ed62e77a12e38fe7677ffd2b9548f655c7d33e6e1607e431d1b8957e86"}}}
{"case":"verify_forward","record":{"index":3,"ids":[13,1061],"verify":true,"logits":{"shape":[1,2,248320],"sha256":"f44498bbf6e6e9d8d909c0e4db24b5ef9bda5866b4a6a14e188865432538643c"},"hidden":{"shape":[1,2,10240],"sha256":"dec8b787a562878d091c854235120732e98f5781751b6d6a428e5cd02228deb0"}}}
{"case":"cold","prompt":[248045,8678,198,24342,286,4879,369,716,310,3238,13,13262,678,7047,9522,321,10419,11,6992,5774,310,279,16198,1973,24366,24150,362,13,248046,198,248045,846,198,2427,494,799,310,5600,13,248046,198,248045,74455,198,248068,198],"tokens":[760,1156,6587,728,310,1698,494,799,310,5600,13,1061],"cached":0,"state_sha256":"eb61c20e4a89bb55c0284d86587966af49d6c601e5421b50eff0762ecf6842b1"}
{"case":"verify_forward","record":{"index":0,"ids":[760,1156,6587,728],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"dbab0d12f25f2be2234d58d45f86219af62deb20e22b0d73a77aa5e7b722d082"},"hidden":{"shape":[1,4,10240],"sha256":"8f9aebc38d1a3e550bed6a4663eedac7a5ed3ce61d836d1cecdfa11da67ebd0d"}}}
{"case":"verify_forward","record":{"index":1,"ids":[310,2962,24217,494],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"d6b8e6afe3c6f5af068f240e0e69b3875311aac6929b3d5a38ab3aeaa1bf227c"},"hidden":{"shape":[1,4,10240],"sha256":"52f20217ed43d8e59ddbe037cb77f56cc3d8ef473fd1963948fff6faf8ad4212"}}}
{"case":"verify_forward","record":{"index":2,"ids":[13,353,2582,3804],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"048e9343d4268d98455a01f7c73ee47af78bc811ef9e7680a72cae59336b9989"},"hidden":{"shape":[1,4,10240],"sha256":"8e8b07fb20baa33ad64cc76ff64403a434d520c627a182a224cf033a5a14e5d4"}}}
{"case":"verify_forward","record":{"index":3,"ids":[8334,29014,494],"verify":true,"logits":{"shape":[1,3,248320],"sha256":"82d671c6ac7b3a0952e60ef554ca670562fd01ef25c37bd10bc27025768ea0f2"},"hidden":{"shape":[1,3,10240],"sha256":"d1bc90f284364eae8bac80a5da6ab64c9c76d6994bb09ed463a17d67e25c016e"}}}
{"case":"warm","prompt":[248045,8678,198,24342,286,4879,369,716,310,3238,13,13262,678,7047,9522,321,10419,11,6992,5774,310,279,16198,1973,24366,24150,362,13,248046,198,248045,846,198,2427,494,799,310,5600,13,248046,198,248045,74455,198,248068,198,760,1156,6587,728,310,1698,494,799,310,5600,13,1061,198,248069,271,248046,198,248045,846,198,22791,24217,13,248046,198,248045,74455,198,248068,198],"tokens":[760,1156,6587,728,310,2962,24217,13,353,8334,29014,494],"cached":58,"state_sha256":"e6cb7355aaa92a3569a365c94c55aa7a50bd731dc04f6fa320da8520d182e2f8"}
{"case":"verify_forward","record":{"index":0,"ids":[760,1156,6587,728],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"dbab0d12f25f2be2234d58d45f86219af62deb20e22b0d73a77aa5e7b722d082"},"hidden":{"shape":[1,4,10240],"sha256":"8f9aebc38d1a3e550bed6a4663eedac7a5ed3ce61d836d1cecdfa11da67ebd0d"}}}
{"case":"verify_forward","record":{"index":1,"ids":[310,2962,24217,494],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"d6b8e6afe3c6f5af068f240e0e69b3875311aac6929b3d5a38ab3aeaa1bf227c"},"hidden":{"shape":[1,4,10240],"sha256":"52f20217ed43d8e59ddbe037cb77f56cc3d8ef473fd1963948fff6faf8ad4212"}}}
{"case":"verify_forward","record":{"index":2,"ids":[13,353,2582,3804],"verify":true,"logits":{"shape":[1,4,248320],"sha256":"048e9343d4268d98455a01f7c73ee47af78bc811ef9e7680a72cae59336b9989"},"hidden":{"shape":[1,4,10240],"sha256":"8e8b07fb20baa33ad64cc76ff64403a434d520c627a182a224cf033a5a14e5d4"}}}
{"case":"verify_forward","record":{"index":3,"ids":[8334,29014,494],"verify":true,"logits":{"shape":[1,3,248320],"sha256":"82d671c6ac7b3a0952e60ef554ca670562fd01ef25c37bd10bc27025768ea0f2"},"hidden":{"shape":[1,3,10240],"sha256":"d1bc90f284364eae8bac80a5da6ab64c9c76d6994bb09ed463a17d67e25c016e"}}}
{"case":"reopened","prompt":[248045,8678,198,24342,286,4879,369,716,310,3238,13,13262,678,7047,9522,321,10419,11,6992,5774,310,279,16198,1973,24366,24150,362,13,248046,198,248045,846,198,2427,494,799,310,5600,13,248046,198,248045,74455,198,248068,198,760,1156,6587,728,310,1698,494,799,310,5600,13,1061,198,248069,271,248046,198,248045,846,198,22791,24217,13,248046,198,248045,74455,198,248068,198],"tokens":[760,1156,6587,728,310,2962,24217,13,353,8334,29014,494],"cached":58,"state_sha256":"e6cb7355aaa92a3569a365c94c55aa7a50bd731dc04f6fa320da8520d182e2f8"}
{"case":"checkpoint","actual_old_rejected":true,"warm_reopened_state_exact":true}
docs/issue109-verify-gate/reopened-native-arrays.json
+1
View File
@@ -0,0 +1 @@
{"trunk":[[{"shape":[1,3,10240],"sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84","value_sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84"},{"shape":[1,48,128,128],"sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf","value_sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf"}],[{"shape":[1,3,10240],"sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232","value_sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232"},{"shape":[1,48,128,128],"sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf","value_sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf"},{"shape":[1,9,10240],"sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08","value_sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08"},{"shape":[1,2],"sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152","value_sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152"}],[{"shape":[1,3,10240],"sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2","value_sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2"},{"shape":[1,48,128,128],"sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34","value_sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34"}],[{"shape":[1,2,88,256],"sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f","value_sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f"},{"shape":[1,2,88,256],"sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24","value_sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24"},{"shape":[1,88,128],"sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748","value_sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748"},{"shape":[1,22,128],"sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a","value_sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a"}],[{"shape":[1,3,10240],"sha256":"431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda","value_sha256":"431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda"},{"shape":[1,48,128,128],"sha256":"c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f","value_sha256":"c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f"}],[{"shape":[1,3,10240],"sha256":"f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a","value_sha256":"f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a"},{"shape":[1,48,128,128],"sha256":"8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f","value_sha256":"8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f"}],[{"shape":[1,3,10240],"sha256":"3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d","value_sha256":"3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d"},{"shape":[1,48,128,128],"sha256":"bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f","value_sha256":"bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f"}],[{"shape":[1,2,88,256],"sha256":"9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b","value_sha256":"9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b"},{"shape":[1,2,88,256],"sha256":"b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba","value_sha256":"b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba"},{"shape":[1,88,128],"sha256":"491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae","value_sha256":"491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae"},{"shape":[1,22,128],"sha256":"104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb","value_sha256":"104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb"}],[{"shape":[1,3,10240],"sha256":"e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57","value_sha256":"e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57"},{"shape":[1,48,128,128],"sha256":"0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43","value_sha256":"0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43"}],[{"shape":[1,3,10240],"sha256":"f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0","value_sha256":"f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0"},{"shape":[1,48,128,128],"sha256":"75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52","value_sha256":"75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52"}],[{"shape":[1,3,10240],"sha256":"c23b7e524522714aaf620c9e0bbb6b1d3db3585971f0cffafd7fe7e2018176dc","value_sha256":"c23b7e524522714aaf620c9e0bbb6b1d3db3585971f0cffafd7fe7e2018176dc"},{"shape":[1,48,128,128],"sha256":"4ce5d0905ec5e5ae834215c16b25d810a5ef6e54a6d7d96f95e47720Line truncated
docs/issue109-verify-gate/warm-native-arrays.json
+1
View File
@@ -0,0 +1 @@
{"trunk":[[{"shape":[1,3,10240],"sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84","value_sha256":"1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84"},{"shape":[1,48,128,128],"sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf","value_sha256":"1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf"}],[{"shape":[1,3,10240],"sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232","value_sha256":"1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232"},{"shape":[1,48,128,128],"sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf","value_sha256":"b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf"},{"shape":[1,9,10240],"sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08","value_sha256":"eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08"},{"shape":[1,2],"sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152","value_sha256":"b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152"}],[{"shape":[1,3,10240],"sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2","value_sha256":"1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2"},{"shape":[1,48,128,128],"sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34","value_sha256":"403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34"}],[{"shape":[1,2,88,256],"sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f","value_sha256":"b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f"},{"shape":[1,2,88,256],"sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24","value_sha256":"eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24"},{"shape":[1,88,128],"sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748","value_sha256":"1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748"},{"shape":[1,22,128],"sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a","value_sha256":"4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a"}],[{"shape":[1,3,10240],"sha256":"431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda","value_sha256":"431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda"},{"shape":[1,48,128,128],"sha256":"c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f","value_sha256":"c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f"}],[{"shape":[1,3,10240],"sha256":"f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a","value_sha256":"f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a"},{"shape":[1,48,128,128],"sha256":"8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f","value_sha256":"8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f"}],[{"shape":[1,3,10240],"sha256":"3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d","value_sha256":"3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d"},{"shape":[1,48,128,128],"sha256":"bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f","value_sha256":"bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f"}],[{"shape":[1,2,88,256],"sha256":"9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b","value_sha256":"9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b"},{"shape":[1,2,88,256],"sha256":"b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba","value_sha256":"b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba"},{"shape":[1,88,128],"sha256":"491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae","value_sha256":"491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae"},{"shape":[1,22,128],"sha256":"104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb","value_sha256":"104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb"}],[{"shape":[1,3,10240],"sha256":"e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57","value_sha256":"e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57"},{"shape":[1,48,128,128],"sha256":"0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43","value_sha256":"0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43"}],[{"shape":[1,3,10240],"sha256":"f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0","value_sha256":"f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0"},{"shape":[1,48,128,128],"sha256":"75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52","value_sha256":"75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52"}],[{"shape":[1,3,10240],"sha256":"c23b7e524522714aaf620c9e0bbb6b1d3db3585971f0cffafd7fe7e2018176dc","value_sha256":"c23b7e524522714aaf620c9e0bbb6b1d3db3585971f0cffafd7fe7e2018176dc"},{"shape":[1,48,128,128],"sha256":"4ce5d0905ec5e5ae834215c16b25d810a5ef6e54a6d7d96f95e47720Line truncated
docs/issue109-verify-gate/warm-oracle-arrays.json
+1
View File
@@ -0,0 +1 @@
{"trunk": [[{"shape": [1, 3, 10240], "sha256": "1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84", "value_sha256": "1df54fa91eb80b88e8b4c0d566828d36540f3508099fc1751dd700d9bd904a84"}, {"shape": [1, 48, 128, 128], "sha256": "1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf", "value_sha256": "1cc66986fdbef7d449d321ff6c558b4e862d2d9211d74038325c72c4f2f0cebf"}], [{"shape": [1, 3, 10240], "sha256": "1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232", "value_sha256": "1ba5aa461d83294641ab065813f2a27ce3a1aa280890046cdc6d839d49c86232"}, {"shape": [1, 48, 128, 128], "sha256": "b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf", "value_sha256": "b789c69d02395058a02c548725e7897e906176c9647e88b5c413a9c743a73aaf"}, {"shape": [1, 9, 10240], "sha256": "eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08", "value_sha256": "eaffcb3f7958776a888b3134ef5f9fa1285f21c242d76b0ebc19c64d75d26c08"}, {"shape": [1, 2], "sha256": "b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152", "value_sha256": "b5e055307f70d9b61bc8427b1da30e8f711b4da0c73d28a34879de8c3f253152"}], [{"shape": [1, 3, 10240], "sha256": "1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2", "value_sha256": "1ebb9fb3628cf830f4c16a815ecaa0c9ccc2049bff2b14d32cb4e3f08f1b91b2"}, {"shape": [1, 48, 128, 128], "sha256": "403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34", "value_sha256": "403226ed2eeae9becd4b498b0fa14de8152beac9138bff0c118698cd4c4fcc34"}], [{"shape": [1, 2, 88, 256], "sha256": "b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f", "value_sha256": "b4995a3d443eb57598940e4a80ece426f8e8442e07f30b8168b247baebfc422f"}, {"shape": [1, 2, 88, 256], "sha256": "eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24", "value_sha256": "eb25629ce30b96850b39bf034124fb8d627c82402baeb89d72e24869005ace24"}, {"shape": [1, 88, 128], "sha256": "1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748", "value_sha256": "1561cf66bb84117ff09036ee128d5c2e5aa21f69db459b82000dc467527df748"}, {"shape": [1, 22, 128], "sha256": "4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a", "value_sha256": "4366612cb46ecc942d06aabe6043fea79b494a34f08f797fec5e0a76ff067a1a"}], [{"shape": [1, 3, 10240], "sha256": "431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda", "value_sha256": "431c351776e8973a28d2206fee9c9262c74df879b561ecb28fdf53f1c6941dda"}, {"shape": [1, 48, 128, 128], "sha256": "c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f", "value_sha256": "c8d64ecbf9dab273a4be2d0b7b00b4fccbb59d25d99ef47ba26a3488dc1ba28f"}], [{"shape": [1, 3, 10240], "sha256": "f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a", "value_sha256": "f4ce935821cd12c5981db92dc849ca6fb25d845a8b38f0ca2ac7f98afa99666a"}, {"shape": [1, 48, 128, 128], "sha256": "8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f", "value_sha256": "8093f937ad983da7c65154990a2a163c7ab09d7e776c7d2b84a8310469433d6f"}], [{"shape": [1, 3, 10240], "sha256": "3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d", "value_sha256": "3c1051a65bbd484bd5bc7ff79919bbb0f8b99f899215efc47ad415541c4ccc7d"}, {"shape": [1, 48, 128, 128], "sha256": "bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f", "value_sha256": "bcbf58854152187ff9556062117970701ac56898d9af860d2b9b3840eaf84b1f"}], [{"shape": [1, 2, 88, 256], "sha256": "9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b", "value_sha256": "9e6bc46ee9a8170bf6bfe9a0cd592543b15d24225ba39e71bddb4aeef947555b"}, {"shape": [1, 2, 88, 256], "sha256": "b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba", "value_sha256": "b84ee35f5c3e0ac8181df0ae66e2278a03db0783c0ea83e4bbabc21bdf47f7ba"}, {"shape": [1, 88, 128], "sha256": "491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae", "value_sha256": "491cccffe6e1c810e6ce8d32fe160005762d1964da055d58746febef7000f5ae"}, {"shape": [1, 22, 128], "sha256": "104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb", "value_sha256": "104bbc74bbb75851ad6a7466e2f2e1a1421a9c93fc0d62aaaf37aa60e2a1dcdb"}], [{"shape": [1, 3, 10240], "sha256": "e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57", "value_sha256": "e87be44991fddef0510651dc23006f2a056a56cf0a61a5f8b38daa34be0c8b57"}, {"shape": [1, 48, 128, 128], "sha256": "0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43", "value_sha256": "0645fe9e0d09d4d55d3919d729ea1340b4ade128dec572010ebb07a5de306f43"}], [{"shape": [1, 3, 10240], "sha256": "f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0", "value_sha256": "f2904788ae98c816e237db5ecc398c269205f38acb680a17e7d00b53289400d0"}, {"shape": [1, 48, 128, 128], "sha256": "75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52", "value_sha256": "75f9e87f1c559b714f5d07911976152045b4b283c50d49794c68cdc66a3cea52"}], [{"shape": [1, 3, 10240], "sha256": "c23b7e52452271Line truncated
src/engine/metal/qwen_mtplx.rs
+2
View File
@@ -37,6 +37,8 @@ mod issue105;
mod issue106;
#[cfg(test)]
mod issue108;
#[cfg(test)]
mod issue109;
mod linear;
mod load;
mod mlp;
src/engine/metal/qwen_mtplx/decoder.rs
+3
View File
@@ -146,6 +146,7 @@ impl DecoderLayer<'_> {
&hidden,
options.hyper_fused,
options.hyper_v3,
options.qsa.verify_gate,
streams,
stream,
)?;
@@ -176,6 +177,7 @@ impl DecoderLayer<'_> {
&hidden,
options.hyper_fused,
options.hyper_v3,
options.qsa.verify_gate,
streams,
stream,
)?;
@@ -372,6 +374,7 @@ fn mtplx_decoder_connected_attention_and_moe_advance_caches() {
..Default::default()
},
qsa: qsa_attention::Options {
verify_gate: false,
indexer: QsaIndexerOptions {
fused: false,
compiled: false,
src/engine/metal/qwen_mtplx/execution.rs
+17 -1
View File
@@ -185,6 +185,7 @@ impl Options {
capture,
},
qsa: qsa_attention::Options {
verify_gate: false,
indexer: QsaIndexerOptions {
fused: flag("MTPLX_FUSED_QSA_INDEXER"),
compiled: flag("MTPLX_COMPILED_QSA_INDEXER"),
@@ -359,6 +360,19 @@ impl Execution {
) -> Result<Output, String> {
let ids = self.input(ids)?;
let options = self.graph_options(phase_prefill, capture)?;
self.forward_with_options(ids, &options)
}
/// Every target forward in the MTP loop (including repairs and final
/// commits) uses MTPLX's decode_verify scope. Draft/history calls do not.
pub(super) fn forward_verify(&mut self, ids: &[i32], capture: bool) -> Result<Output, String> {
let ids = self.input(ids)?;
let mut options = self.graph_options(false, capture)?;
options.qsa.verify_gate = true;
self.forward_with_options(ids, &options)
}
fn forward_with_options(&mut self, ids: Array, options: &Options) -> Result<Output, String> {
let output = self.model.with_dependent_mut(|_, model| {
model.call(
&ids,
@@ -369,13 +383,15 @@ impl Execution {
logits_keep: 0,
return_hidden: true,
},
&options,
options,
None,
&self.streams,
self.stream,
)
})?;
#[cfg(test)]
super::issue109::record(self, &ids, &output, options.qsa.verify_gate)?;
#[cfg(test)]
let output = self.inject_output(super::issue105::Stage::Target, output)?;
#[cfg(test)]
let output = self.script_output(false, output)?;
src/engine/metal/qwen_mtplx/hyper.rs
+32 -13
View File
@@ -23,6 +23,7 @@ impl GatedResidual<'_> {
input: &Array,
fused: bool,
v3: bool,
verify: bool,
streams: &Streams,
stream: Stream,
) -> Result<Read, String> {
@@ -110,18 +111,8 @@ impl GatedResidual<'_> {
let sum = super::reduce::sum(&product, &[-2], false, stream)?;
let mixed = binary(&sum, &scalar(0x3e80)?, Binary::Multiply, stream)?;
let combine = if let Some(inject) = &self.inject {
let gate = binary(
&inject.apply(&normed, stream)?,
&scalar(0x4080)?,
Binary::Divide,
stream,
)?;
let gate = unary(&gate, Unary::Sigmoid, stream)?;
Some((
input.clone(),
// Python __rmul__ keeps the array first for 2.0 * gate.
binary(&gate, &scalar(0x4000)?, Binary::Multiply, stream)?,
))
let gate = inject_gate(&inject.apply(&normed, stream)?, verify, stream)?;
Some((input.clone(), gate))
} else {
None
};
@@ -129,6 +120,34 @@ impl GatedResidual<'_> {
}
}
/// The eager HC inject site follows MTPLX _verify_inject on decode_verify.
/// Other phases retain the original standalone divide/sigmoid/multiply.
pub(super) fn inject_gate(logits: &Array, verify: bool, stream: Stream) -> Result<Array, String> {
use super::{
binary::{Binary, binary},
unary::{Unary, unary},
};
let body = |x: &Array| {
let scalar =
|bits: u16| Array::new(&[], Dtype::BF16, super::scalar_buffer(&bits.to_le_bytes())?);
let divided = binary(x, &scalar(0x4080)?, Binary::Divide, stream)?;
binary(
&unary(&divided, Unary::Sigmoid, stream)?,
&scalar(0x4000)?,
Binary::Multiply,
stream,
)
};
if !verify {
return body(logits);
}
let inputs = std::slice::from_ref(logits);
super::compiled::Replay::trace_compiled(inputs, |a| Ok(vec![body(&a[0])?]))?
.run(inputs)?
.pop()
.ok_or_else(|| "compiled HC inject omitted output".into())
}
pub(super) fn fused_read(
x: &Array,
norm: &Array,
@@ -379,7 +398,7 @@ fn mtplx_graph_hyper_forward_matches_reference() {
let mode = receipt["mode"].as_u64().unwrap();
let input = bf16(&[1, rows, 10240], 1);
let result = layer
.forward(&input, mode & 1 != 0, mode & 2 != 0, &streams, gpu)
.forward(&input, mode & 1 != 0, mode & 2 != 0, false, &streams, gpu)
.unwrap();
assert_eq!(
layer.pack.is_some(),
src/engine/metal/qwen_mtplx/issue103_analysis.rs
+23 -5
View File
@@ -175,7 +175,16 @@ fn issue103_numeric_receipts() {
ops::evaluate(&streams, &[gate.clone(), value.clone()], gpu, false).unwrap();
let body = |a: &[Array]| {
binary(
&unary::unary(&a[0], unary::Unary::Sigmoid, gpu)?,
&unary::unary(
&ops::astype(
&ops::astype(&a[0], Dtype::F32, false, gpu)?,
Dtype::BF16,
false,
gpu,
)?,
unary::Unary::Sigmoid,
gpu,
)?,
&a[1],
Binary::Multiply,
gpu,
@@ -185,7 +194,7 @@ fn issue103_numeric_receipts() {
let eager = body(&[gate.clone(), value.clone()]).unwrap();
let compiled = Replay::trace_compiled(&[gate.clone(), value.clone()], body)
.unwrap()
.run(&[gate, value])
.run(&[gate.clone(), value.clone()])
.unwrap();
let read = |a: &Array| {
ops::read_f32_as_f64(
@@ -201,7 +210,14 @@ fn issue103_numeric_receipts() {
"{}",
serde_json::json!({"case":"bf16_gate","gate":-6.84375,"eager":e,"compiled":c})
);
assert_ne!(e, c, "diagnosis no longer reproduces");
assert_ne!(
e, c,
"the two reference phase laws must remain distinguishable"
);
let actual = super::qsa_attention::attention_gate(&value, &gate, true, gpu).unwrap();
assert_eq!(read(&actual), c);
assert_eq!(e, vec![0.00106048583984375; 4]);
assert_eq!(c, vec![0.001068115234375; 4]);
for (name, operand, mode) in [
("qsa_divisor", 128_f32.sqrt(), Binary::Divide),
(
@@ -367,7 +383,8 @@ fn issue103_vision_position_receipts() {
.unwrap(),
)
.unwrap();
let options = Options {
let mut options = Options {
verify_gate: false,
indexer: super::QsaIndexerOptions {
fused: false,
compiled: false,
@@ -647,6 +664,7 @@ fn issue103_vision_position_receipts() {
.map(|i| f32::from(i / 8 < 8 && i / 8 == i % 8))
.collect::<Vec<_>>(),
);
options.verify_gate = true;
let mut gate_outputs = Vec::new();
for gate in [0., -6.84375] {
let bias = bf(&[16], &[vec![0.; 8], vec![gate; 8]].concat());
@@ -724,5 +742,5 @@ fn issue103_vision_position_receipts() {
"{}",
serde_json::json!({"case":"active_attention_gate","gate":-6.84375,"actual":actual,"expected_verify":expected})
);
assert_ne!(actual, expected, "diagnosis no longer reproduces");
assert_eq!(actual, expected, "verify must use the compiled gate law");
}
src/engine/metal/qwen_mtplx/issue106.rs
+1 -1
View File
@@ -142,7 +142,7 @@ fn advance(execution: &mut Execution, saved: &SessionSnapshot, token: i32) -> Se
.append_mtp_history(hidden.as_ref().unwrap(), &[token], false)
.unwrap();
ops::evaluate(&execution.streams, &[updated], execution.stream, false).unwrap();
let output = execution.forward(&[token], false, false).unwrap();
let output = execution.forward_verify(&[token], false).unwrap();
let logits = ar::last_row(&output.logits.unwrap(), execution.stream).unwrap();
let mut key = saved.token_ids.clone();
key.push(token);
src/engine/metal/qwen_mtplx/issue107_product.rs
+1 -1
View File
@@ -6,7 +6,7 @@ use crate::settings::{
};
use base64::Engine as _;
fn state(owner: &QwenExecutor) -> String {
pub(super) fn state(owner: &QwenExecutor) -> String {
let saved = owner.live.current.as_ref().unwrap();
let mut hashes = Vec::new();
let payload = saved
src/engine/metal/qwen_mtplx/issue109.rs
+119
View File
@@ -0,0 +1,119 @@
use super::array::Dtype;
use super::{ops, qsa_attention};
#[test]
#[ignore = "Metal and exhaustive pinned gate fixtures; external supervisor required"]
fn issue109_gate_phase_and_dtype_laws() {
super::configure_sources().unwrap();
let _context = crate::engine::metal::gpu::Context::open_qwen(0).unwrap();
let streams = super::stream::Streams::new(Some(super::allocator::Allocator::new().unwrap()));
let gpu = streams.default_stream(super::stream::Device::Gpu).unwrap();
let root = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE109_GATE").unwrap());
for (name, dtype) in [
("bf16", Dtype::BF16),
("f16", Dtype::F16),
("f32", Dtype::F32),
] {
let load = |part: &str| {
let bytes = std::fs::read(root.join(format!("{name}-{part}.f32"))).unwrap();
let array = super::array::Array::new(
&[(bytes.len() / 4) as i32],
Dtype::F32,
super::scalar_buffer(&bytes).unwrap(),
)
.unwrap();
ops::astype(&array, dtype, false, gpu).unwrap()
};
let gate = load("gate");
let output = load("output");
for phase in ["prefill", "ar_decode", "decode_verify"] {
let actual =
qsa_attention::attention_gate(&output, &gate, phase == "decode_verify", gpu)
.unwrap();
let actual = ops::read_f32_as_f64(
&ops::astype(&actual, Dtype::F32, false, gpu).unwrap(),
&streams,
gpu,
)
.unwrap()
.into_iter()
.flat_map(|v| (v as f32).to_le_bytes())
.collect::<Vec<_>>();
let expected = std::fs::read(root.join(format!("{name}-{phase}.f32"))).unwrap();
assert_eq!(actual.len(), expected.len());
let different = actual
.chunks_exact(4)
.zip(expected.chunks_exact(4))
.enumerate()
.filter_map(|(i, (a, b))| (a != b).then_some(i))
.collect::<Vec<_>>();
assert!(
different.is_empty(),
"{name}/{phase}: {} different rows; first {:?}",
different.len(),
different.first()
);
println!(
"{}",
serde_json::json!({"case":"gate","dtype":name,"phase":phase,"values":actual.len()/4,"exact":true})
);
if dtype == Dtype::BF16 {
let inject =
super::hyper::inject_gate(&gate, phase == "decode_verify", gpu).unwrap();
let actual = ops::read_f32_as_f64(
&ops::astype(&inject, Dtype::F32, false, gpu).unwrap(),
&streams,
gpu,
)
.unwrap()
.into_iter()
.flat_map(|v| (v as f32).to_le_bytes())
.collect::<Vec<_>>();
assert!(
actual == std::fs::read(root.join(format!("inject-{phase}.f32"))).unwrap(),
"inject/{phase}"
);
println!(
"{}",
serde_json::json!({"case":"inject","phase":phase,"values":actual.len()/4,"exact":true})
);
}
}
}
}
thread_local! {
static RECEIPTS: std::cell::RefCell<Option<(std::path::PathBuf, usize)>> = const { std::cell::RefCell::new(None) };
}
pub(super) fn capture(directory: Option<std::path::PathBuf>) {
RECEIPTS.with_borrow_mut(|slot| {
*slot = directory.map(|d| {
std::fs::create_dir_all(&d).unwrap();
(d, 0)
})
});
}
pub(super) fn record(
execution: &super::execution::Execution,
ids: &super::array::Array,
output: &super::model::Output,
verify: bool,
) -> Result<(), String> {
RECEIPTS.with_borrow_mut(|slot| {
let Some((directory, index)) = slot else { return Ok(()); };
let ids = ops::read_i64(ids, &execution.streams, execution.stream)?;
let mut record=serde_json::json!({"index":*index,"ids":ids,"verify":verify});
for (part, a) in [("logits",&output.logits),("hidden",&output.hidden)] {
let a=a.as_ref().ok_or("missing #109 output")?;
let bytes=execution.values_f32(a)?.into_iter().flat_map(f32::to_le_bytes).collect::<Vec<_>>();
std::fs::write(directory.join(format!("{index}-{part}.f32")),&bytes).map_err(|e|e.to_string())?;
record[part]=serde_json::json!({"shape":a.layout().shape(),"sha256":crate::engine::metal::qwen_mtplx_tests::hex_digest(&bytes)});
}
std::fs::write(directory.join(format!("{index}.json")),serde_json::to_vec(&record).unwrap()).map_err(|e|e.to_string())?;
println!("{}",serde_json::json!({"case":"verify_forward","record":record}));
*index+=1;
Ok(())
})
}
src/engine/metal/qwen_mtplx/issue109_product.rs
+184
View File
@@ -0,0 +1,184 @@
use super::issue107_product::state;
use super::*;
use crate::settings::{
GenerationPreferences, ReasoningMode, RuntimePreferences, SpeculativePreferences,
effective_settings,
};
#[test]
#[ignore = "installed model; serial external memory supervisor required"]
fn issue109_verify_rows_and_warm_checkpoint() {
assert_eq!(std::env::var("MTPLX_CONTEXT_COPY").as_deref(), Ok("0"));
let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
let directory = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE109_DIR").unwrap());
std::fs::create_dir_all(&directory).unwrap();
let settings = effective_settings(
crate::model::ModelChoice::Qwen38FlashNext,
&GenerationPreferences {
context_tokens: 8192,
max_generated_tokens: 12,
system_prompt: String::new(),
temperature: Some(0.),
top_p: Some(0.95),
min_p: Some(0.),
seed: Some(1),
reasoning_mode: ReasoningMode::Low,
},
&RuntimePreferences {
speculative: SpeculativePreferences {
glm_mtp: true,
..Default::default()
},
..Default::default()
},
root.parent().unwrap(),
)
.unwrap();
let mut owner = QwenExecutor::open(&settings.engine).unwrap();
owner.policy.store_min_suffix = Some(0);
let metrics = Metrics::new(&directory.join("metrics"));
let messages = vec![ChatTurn {
user: true,
tool: false,
system: false,
skip_previous_eos: false,
reasoning: None,
reasoning_complete: true,
content: "Count from one to ten.".into(),
}];
let run = |owner: &mut QwenExecutor, messages: &[ChatTurn], label: &str| {
super::super::issue109::capture(Some(directory.join(label)));
let (out, complete) = owner
.generate(
messages,
&settings.turn,
&AtomicBool::new(false),
&metrics,
|_, _, _| {},
|_, _| {},
)
.unwrap();
super::super::issue109::capture(None);
assert!(complete);
if let Some(prefix) = &owner.live.prefix {
if let Some(super::super::session_cache::State::Qsa(a)) = &prefix.trunk[35] {
let bytes = owner
.execution
.values_f32(a[0].as_ref().unwrap())
.unwrap()
.into_iter()
.flat_map(f32::to_le_bytes)
.collect::<Vec<_>>();
std::fs::write(directory.join(format!("{label}-prefill-key35.f32")), bytes)
.unwrap();
}
for (part, a) in [
("logits", Some(&prefix.logits)),
("hidden", prefix.hidden.as_ref()),
] {
let bytes = owner
.execution
.values_f32(a.unwrap())
.unwrap()
.into_iter()
.flat_map(f32::to_le_bytes)
.collect::<Vec<_>>();
std::fs::write(directory.join(format!("{label}-prefill-{part}.f32")), bytes)
.unwrap();
}
}
let saved = owner.live.current.as_ref().unwrap();
let states = |states: &[Option<super::super::session_cache::State>]| {
states.iter().map(|s| {
s.as_ref().map(|s| {
let arrays: &[Option<super::super::array::Array>] = match s {
super::super::session_cache::State::Recurrent(a) => a,
super::super::session_cache::State::Qsa(a) => a,
};
arrays.iter().map(|a| {
a.as_ref().map(|a| {
let values = owner.execution.values_f32(a).unwrap();
let bytes = values.iter().flat_map(|v| v.to_le_bytes()).collect::<Vec<_>>();
let canonical_zero = values.iter().flat_map(|&v| (if v == 0. { 0. } else { v }).to_le_bytes()).collect::<Vec<_>>();
let hash = crate::engine::metal::qwen_mtplx_tests::hex_digest;
serde_json::json!({"shape":a.layout().shape(),"sha256":hash(&bytes),"value_sha256":hash(&canonical_zero)})
})
}).collect::<Vec<_>>()
})
}).collect::<Vec<_>>()
};
let arrays = serde_json::json!({"trunk":states(&saved.trunk),"mtp":states(saved.mtp.as_ref().unwrap())});
std::fs::write(
directory.join(format!("{label}-arrays.json")),
serde_json::to_vec(&arrays).unwrap(),
)
.unwrap();
if let Some(super::super::session_cache::State::Qsa(a)) = &saved.trunk[35] {
let bytes = owner
.execution
.values_f32(a[0].as_ref().unwrap())
.unwrap()
.into_iter()
.flat_map(f32::to_le_bytes)
.collect::<Vec<_>>();
std::fs::write(directory.join(format!("{label}-key35.f32")), bytes).unwrap();
}
let record = serde_json::json!({"case":label,"prompt":&saved.token_ids[..out.prompt_tokens as usize],"tokens":&saved.token_ids[out.prompt_tokens as usize..],"cached":out.cached_tokens,"state_sha256":state(owner)});
std::fs::write(
directory.join(format!("{label}.json")),
serde_json::to_vec(&record).unwrap(),
)
.unwrap();
for (part, a) in [
("logits", Some(&saved.logits)),
("hidden", saved.hidden.as_ref()),
] {
let bytes = owner
.execution
.values_f32(a.unwrap())
.unwrap()
.into_iter()
.flat_map(f32::to_le_bytes)
.collect::<Vec<_>>();
std::fs::write(directory.join(format!("{label}-{part}.f32")), bytes).unwrap();
}
println!("{record}");
out
};
let first = run(&mut owner, &messages, "cold");
assert_eq!(first.cached_tokens, 0);
let original = state(&owner);
owner
.save_checkpoint(&directory.join("new.bin"), [109; 32], &mut |_| {})
.unwrap();
let old = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE109_OLD").unwrap());
assert!(
owner
.load_checkpoint(&old, &mut |_| {})
.unwrap_err()
.contains("obsolete MTP attention-gate")
);
assert_eq!(original, state(&owner));
let mut next = messages.clone();
next.push(first.message.clone());
next.push(ChatTurn {
content: "Continue counting.".into(),
..messages[0].clone()
});
let warm = run(&mut owner, &next, "warm");
assert!(warm.cached_tokens >= first.prompt_tokens);
let expected = state(&owner);
owner.reset().unwrap();
assert!(
owner
.load_checkpoint(&directory.join("new.bin"), &mut |_| {})
.unwrap()
);
assert_eq!(original, state(&owner));
run(&mut owner, &next, "reopened");
assert_eq!(expected, state(&owner));
println!(
"{}",
serde_json::json!({"case":"checkpoint","actual_old_rejected":true,"warm_reopened_state_exact":true})
);
}
src/engine/metal/qwen_mtplx/model.rs
+18 -2
View File
@@ -75,6 +75,7 @@ fn runs(layers: impl IntoIterator<Item = (bool, bool)>) -> Vec<Run> {
struct RunKey {
range: Range<usize>,
capture: bool,
verify_gate: bool,
stream: Stream,
signature: Vec<(Vec<i32>, Dtype)>,
}
@@ -223,7 +224,14 @@ impl<'a> TextModel<'a> {
let h = mtp.fuse_and_run(hidden, &emb, cache, options, vision, streams, stream)?;
let mixed = mtp
.mixer
.forward(&h, options.hyper_fused, options.hyper_v3, streams, stream)?
.forward(
&h,
options.hyper_fused,
options.hyper_v3,
options.qsa.verify_gate,
streams,
stream,
)?
.mixed;
let logits = self.head.unwrap_or(self.embedding).apply(&mixed, stream)?;
Ok(Output {
@@ -516,7 +524,14 @@ impl<'a> TextModel<'a> {
self.last_widened = Some(h.clone());
let mixed = self
.mixer
.forward(&h, options.hyper_fused, options.hyper_v3, streams, stream)?
.forward(
&h,
options.hyper_fused,
options.hyper_v3,
options.qsa.verify_gate,
streams,
stream,
)?
.mixed;
let logits = if output.emit_logits {
let mixed = if output.logits_keep != 0 {
@@ -610,6 +625,7 @@ impl<'a> TextModel<'a> {
let key = RunKey {
range: range.clone(),
capture: options.gdn.capture,
verify_gate: options.qsa.verify_gate,
stream,
signature: flat
.iter()
src/engine/metal/qwen_mtplx/owner.rs
+4
View File
@@ -1072,3 +1072,7 @@ mod issue106_product;
#[cfg(test)]
#[path = "issue107_product.rs"]
mod issue107_product;
#[cfg(test)]
#[path = "issue109_product.rs"]
mod issue109_product;
src/engine/metal/qwen_mtplx/qsa_attention.rs
+51 -8
View File
@@ -1,7 +1,5 @@
//! qwen4_exp.Attention: projection, indexer, RoPE, KV and consumer branch order.
use super::array::Array;
#[cfg(test)]
use super::array::Dtype;
use super::array::{Array, Dtype};
use super::attention::{self, EagerSelection};
use super::binary::{Binary, binary};
use super::cache::QsaCache;
@@ -34,12 +32,54 @@ pub(super) struct Attention<'a> {
/// Resolved original request-phase/environment switches, not new heuristics.
pub(super) struct Options {
pub(super) verify_gate: bool,
pub(super) indexer: QsaIndexerOptions,
pub(super) flash_min_context: u32,
pub(super) prefill_gather: bool,
pub(super) gather_tile_rows: i32,
}
/// Only decode_verify uses the fused lowering from MTPLX attention_math.py.
/// In BF16 the value-preserving cast round trip selects that lowering without
/// changing the sigmoid's dtype. All other phases and FP16 keep the stock graph.
pub(super) fn attention_gate(
output: &Array,
gate: &Array,
verify: bool,
stream: Stream,
) -> Result<Array, String> {
let dtype = gate.layout().dtype();
let body = |output: &Array, gate: &Array| {
binary(
output,
&unary::unary(gate, unary::Unary::Sigmoid, stream)?,
Binary::Multiply,
stream,
)
};
if !verify || !matches!(dtype, Dtype::BF16 | Dtype::F32) {
return body(output, gate);
}
let inputs = [output.clone(), gate.clone()];
let replay = super::compiled::Replay::trace_compiled(&inputs, |args| {
let gate = if dtype == Dtype::BF16 {
ops::astype(
&ops::astype(&args[1], Dtype::F32, false, stream)?,
Dtype::BF16,
false,
stream,
)?
} else {
args[1].clone()
};
Ok(vec![body(&args[0], &gate)?])
})?;
replay
.run(&inputs)?
.pop()
.ok_or_else(|| "compiled attention gate omitted output".into())
}
impl Attention<'_> {
fn finish(
&self,
@@ -47,13 +87,12 @@ impl Attention<'_> {
gate: &Array,
batch: i32,
rows: i32,
verify: bool,
stream: Stream,
) -> Result<Array, String> {
let out = ops::reshape(out, &[batch, rows, -1], stream)?;
let gate = unary::unary(gate, unary::Unary::Sigmoid, stream)?;
let out = self
.output
.apply(&binary(&out, &gate, Binary::Multiply, stream)?, stream)?;
let out = attention_gate(&out, gate, verify, stream)?;
let out = self.output.apply(&out, stream)?;
match self.output_bias {
Some(bias) => binary(&out, bias, Binary::Add, stream),
None => Ok(out),
@@ -211,7 +250,7 @@ impl Attention<'_> {
self.scale,
stream,
)?;
return self.finish(&out, &gate, batch, rows, stream);
return self.finish(&out, &gate, batch, rows, options.verify_gate, stream);
}
if let Some(EagerSelection::PrefillBlocks(ids, valid)) = &selected {
let keys = cache.kv.keys.as_ref().unwrap().borrow();
@@ -266,6 +305,7 @@ impl Attention<'_> {
&gate,
batch,
rows,
options.verify_gate,
stream,
);
}
@@ -279,6 +319,7 @@ impl Attention<'_> {
&gate,
batch,
rows,
options.verify_gate,
stream,
);
}
@@ -317,6 +358,7 @@ impl Attention<'_> {
&gate,
batch,
rows,
options.verify_gate,
stream,
)
}
@@ -503,6 +545,7 @@ fn mtplx_qsa_attention_module_matches_reference() {
}),
};
let options = Options {
verify_gate: false,
indexer: qsa_indexer_options(&first["options"]),
flash_min_context: first["flash_min"].as_u64().unwrap() as u32,
prefill_gather: first["prefill_gather"].as_bool().unwrap(),
src/engine/metal/qwen_mtplx/session_checkpoint.rs
+48 -1
View File
@@ -30,6 +30,8 @@ struct Manifest {
identity: Identity,
tag: [u8; 32],
#[serde(default)]
verify_gate: bool,
#[serde(default)]
images: Vec<crate::engine::qwen_vision::Identity>,
tokens: Vec<i32>,
lazy_kv: bool,
@@ -116,6 +118,7 @@ impl LiveSession {
check_abort(&abort)?;
let offset = file.stream_position().map_err(|e| e.to_string())?;
let manifest = serde_json::to_vec(&Manifest {
verify_gate: true,
identity,
tag,
images: self.current_images.clone(),
@@ -185,6 +188,11 @@ impl LiveSession {
if !crate::engine::qwen_vision::compatible(&manifest.images) {
return Err("Qwen vision cache policy differs; rebuild the cache".into());
}
if identity.mtp && !manifest.verify_gate {
return Err(
"Qwen checkpoint uses obsolete MTP attention-gate math; rebuild the cache".into(),
);
}
if !manifest.payload["mtp_history_snapshot"].is_null() != identity.mtp {
return Err("Qwen checkpoint MTP payload differs".into());
}
@@ -433,7 +441,46 @@ fn qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state() {
}
assert_eq!(live.tag, [9; 32]);
assert_eq!(fingerprint(&live), original);
// Files predating the verify-phase law lack this field. Only MTP state
// changed numerically; old AR files must remain exactly restorable.
let without_gate_policy = |bytes: &[u8]| {
let offset = u64::from_le_bytes(bytes[8..16].try_into().unwrap()) as usize;
let mut manifest: serde_json::Value = serde_json::from_slice(&bytes[offset..]).unwrap();
manifest.as_object_mut().unwrap().remove("verify_gate");
let mut result = bytes[..offset].to_vec();
result.extend(serde_json::to_vec(&manifest).unwrap());
result
};
fs::write(&broken, without_gate_policy(&valid)).unwrap();
assert!(
live.load_checkpoint(&broken, identity, &streams, stream, |_| {}, || false)
.unwrap_err()
.contains("obsolete MTP attention-gate")
);
assert_eq!(fingerprint(&live), original);
live.current.as_mut().unwrap().mtp = None;
let ar_identity = Identity {
mtp: false,
..identity
};
let ar_original = fingerprint(&live);
live.save_checkpoint(
&path,
ar_identity,
[10; 32],
&streams,
stream,
|_| {},
|| false,
)
.unwrap();
fs::write(&broken, without_gate_policy(&fs::read(&path).unwrap())).unwrap();
assert!(
live.load_checkpoint(&broken, ar_identity, &streams, stream, |_| {}, || false)
.unwrap()
);
assert_eq!(fingerprint(&live), ar_original);
println!(
"Qwen checkpoint: exact payload roundtrip, interrupted save, incompatible/corrupt restore passed"
"Qwen checkpoint: exact roundtrip, cancellation/corruption controls, old MTP rejected and old AR restored"
);
}
src/engine/metal/qwen_mtplx/speculative.rs
+9 -8
View File
@@ -436,7 +436,7 @@ impl Execution {
.collect::<Result<Vec<_>, _>>()
.map_err(|_| "context-copy token exceeds int32")?;
let started = Instant::now();
let verified = self.forward(&input, false, options.family_capture)?;
let verified = self.forward_verify(&input, options.family_capture)?;
let logits = verified.logits.ok_or("copy verify omitted logits")?;
let hidden = verified.hidden.ok_or("copy verify omitted hidden")?;
let greedy = if sampler.temperature <= 0.0 {
@@ -492,7 +492,8 @@ impl Execution {
}
let keep = accepted + 1;
let committed = if options.family_capture {
let graph = self.graph_options(false, false)?;
let mut graph = self.graph_options(false, false)?;
graph.qsa.verify_gate = true;
self.model.with_dependent_mut(|_, model| {
model.commit_verified_window(
&mut self.cache,
@@ -513,7 +514,7 @@ impl Execution {
if !committed {
self.rollback_mtp_verify(&snapshots, input.len() as i32)?;
let started = Instant::now();
let repaired = self.forward(&input[..1], false, false)?;
let repaired = self.forward_verify(&input[..1], false)?;
let logits = repaired.logits.ok_or("copy repair omitted logits")?;
let hidden = repaired.hidden.ok_or("copy repair omitted hidden")?;
ops::evaluate(
@@ -810,7 +811,7 @@ impl Execution {
.map_err(|_| "MTP token exceeds int32")?,
);
let started = Instant::now();
let verified = self.forward(&input, false, options.family_capture)?;
let verified = self.forward_verify(&input, options.family_capture)?;
let verify_logits = verified.logits.ok_or("verify omitted logits")?;
let verify_hidden = verified.hidden.ok_or("verify omitted hidden")?;
ops::evaluate(
@@ -911,7 +912,8 @@ impl Execution {
// MTPLX exits verify_capture_scope before replaying the kept
// prefix. Keeping capture armed here disables the fused GDN
// single-row path on the PLE layer's repair forward.
let graph = self.graph_options(false, false)?;
let mut graph = self.graph_options(false, false)?;
graph.qsa.verify_gate = true;
if options.family_capture {
cycle.capture_committed = self.model.with_dependent_mut(|_, model| {
model.commit_verified_window(
@@ -940,7 +942,7 @@ impl Execution {
.map(|&id| i32::try_from(id))
.collect::<Result<Vec<_>, _>>()
.map_err(|_| "MTP token exceeds int32")?;
let repaired = self.forward(&ids, false, false)?;
let repaired = self.forward_verify(&ids, false)?;
let logits = repaired.logits.ok_or("repair omitted logits")?;
let hidden = repaired.hidden.ok_or("repair omitted hidden")?;
if !options.defer_repair_eval {
@@ -999,10 +1001,9 @@ impl Execution {
result.history_seconds +=
self.append_committed(&result.hidden, &[token], options.lazy_history)?;
}
let output = self.forward(
let output = self.forward_verify(
&[i32::try_from(token).map_err(|_| "MTP token exceeds int32")?],
false,
false,
)?;
let logits = output.logits.ok_or("final commit omitted logits")?;
let hidden = output.hidden.ok_or("final commit omitted hidden")?;
tools/issue109-gate-reference.py
+36
View File
@@ -0,0 +1,36 @@
"""Exact attention gate laws for all finite BF16/F16 values and dense F32 inputs."""
import hashlib,json,subprocess,sys
from pathlib import Path
checkout,out=map(Path,sys.argv[1:3])
assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
sys.path.insert(0,str(checkout))
import mlx.core as mx
import numpy as np
from mtplx.attention_math import attention_gate
from mtplx.attention_context import attention_phase
from mtplx.models.qwen4_exp import _verify_inject
out.mkdir(parents=True,exist_ok=True)
for name,dtype in [('bf16',mx.bfloat16),('f16',mx.float16),('f32',mx.float32)]:
if name=='bf16':
values=(np.arange(65536,dtype=np.uint32)<<16).view(np.float32)
elif name=='f16':
values=np.arange(65536,dtype=np.uint16).view(np.float16).astype(np.float32)
else:
values=np.concatenate([np.linspace(-30,30,32768,dtype=np.float32),np.random.default_rng(5).normal(0,4,32768).astype(np.float32)])
values=values[np.isfinite(values)]
gate=mx.array(values).astype(dtype)
output=mx.array(((np.arange(values.size,dtype=np.int64)*17)%97-48).astype(np.float32)/32).astype(dtype)
mx.eval(gate,output)
for part,a in [('gate',gate),('output',output)]:
(out/f'{name}-{part}.f32').write_bytes(np.asarray(a.astype(mx.float32)).tobytes())
for phase in ['prefill','ar_decode','decode_verify']:
with attention_phase(phase): result=attention_gate(output,gate)
raw=np.asarray(result.astype(mx.float32)).tobytes()
(out/f'{name}-{phase}.f32').write_bytes(raw)
print(json.dumps(dict(case='gate',dtype=name,phase=phase,count=values.size,sha256=hashlib.sha256(raw).hexdigest())),flush=True)
if name=='bf16':
inject=_verify_inject(gate,4) if phase=='decode_verify' else 2.0*mx.sigmoid(gate/4)
raw=np.asarray(inject.astype(mx.float32)).tobytes()
(out/f'inject-{phase}.f32').write_bytes(raw)
print(json.dumps(dict(case='inject',dtype=name,phase=phase,count=values.size,sha256=hashlib.sha256(raw).hexdigest())),flush=True)
tools/issue109-model-reference.py
+128
View File
@@ -0,0 +1,128 @@
"""Compare actual MTP verify/repair/commit rows and warm continuation to native."""
import argparse,hashlib,json,os,subprocess,sys
from pathlib import Path
checkout,root,native,out=map(Path,sys.argv[1:5])
assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip()=='9882703f3105363ddc37eca9f97aa09a1d387112'
sys.path.insert(0,str(checkout))
from mtplx.server.openai import _server_runtime_env_overrides
os.environ.update(_server_runtime_env_overrides(argparse.Namespace(model=str(root),generation_mode='mtp'),None))
os.environ['MTPLX_SUSTAINED_PREFILL']='1'
os.environ['MTPLX_PREFILL_CHUNK_SIZE']='2048'
os.environ['MTPLX_CONTEXT_COPY']='0'
import mlx.core as mx
import numpy as np
from mlx_lm.utils import load_model
from mtplx.models import qwen4_exp as qwen
from mtplx.runtime import _load_tokenizer_resilient,MTPLXRuntime
from mtplx.mtp_patch import MTPContract
from mtplx.generation import generate_mtpk
from mtplx.sampling import SamplerConfig
from mtplx.session_bank import SessionBank
from mtplx.cache_state import snapshot_cache
from mtplx.attention_context import current_attention_phase
out.mkdir(parents=True,exist_ok=True)
print(json.dumps(dict(case='environment',overrides={k:v for k,v in os.environ.items() if k.startswith('MTPLX_')})),flush=True)
tokenizer=_load_tokenizer_resilient(root,json.loads((root/'config.json').read_text()))
model,_=load_model(root,lazy=False,strict=True,get_model_classes=lambda **_:(qwen.Model,qwen.ModelArgs))
model.post_weight_load(root)
assert model.attach_mtp(root)
rt=MTPLXRuntime(model,tokenizer,root,True,MTPContract())
phase='cold'; index=0; nested=0
def wrap(fn):
def call(input_ids,*args,**kwargs):
global index,nested
outer=nested==0
nested+=1
try: result=fn(input_ids,*args,**kwargs)
finally: nested-=1
if outer and current_attention_phase() == 'prefill' and isinstance(result,tuple) and result[0] is not None and result[1] is not None:
cache=kwargs.get('cache',args[0] if args else None)
if cache is not None:
key=np.asarray(snapshot_cache(cache).states[35][0].astype(mx.float32))
(out/f'{phase}-prefill-key35.f32').write_bytes(key.tobytes())
for part,a in zip(['logits','hidden'],result[:2]):
data=np.asarray(a.astype(mx.float32)).tobytes()
(out/f'{phase}-prefill-{part}.f32').write_bytes(data)
path=native/f'{phase}-prefill-{part}.f32'
print(json.dumps(dict(case='prefill_compare',phase=phase,part=part,shape=list(a.shape),exact=path.exists() and path.read_bytes()==data)),flush=True)
if outer and current_attention_phase()=='decode_verify':
expected=json.loads((native/phase/f'{index}.json').read_text())
ids=np.asarray(input_ids).reshape(-1).tolist()
assert expected['verify'] and ids==expected['ids'], (phase,index,ids,expected['ids'])
receipt=dict(case='verify_forward',phase=phase,index=index,ids=ids)
for part,a in zip(['logits','hidden'],result[:2]):
data=np.asarray(a.astype(mx.float32)).tobytes()
(out/f'{phase}-{index}-{part}.f32').write_bytes(data)
ref=(native/phase/f'{index}-{part}.f32').read_bytes()
receipt[part+'_exact']=data==ref
receipt[part+'_sha256']=hashlib.sha256(data).hexdigest()
if data!=ref:
x=np.frombuffer(data,np.float32); y=np.frombuffer(ref,np.float32)
receipt[part+'_different']=int(np.count_nonzero(x!=y))
receipt[part+'_max_abs']=float(np.max(np.abs(x-y)))
print(json.dumps(receipt),flush=True)
assert receipt['logits_exact'] and receipt['hidden_exact'],receipt
index+=1
return result
return call
rt.forward_ar=wrap(rt.forward_ar)
rt.forward_ar_capture=wrap(rt.forward_ar_capture)
bank=SessionBank()
for phase in ['cold','warm']:
index=0
expected=json.loads((native/f'{phase}.json').read_text())
ids=expected['prompt']
result=generate_mtpk(rt,ids,max_tokens=12,sampler=SamplerConfig(temperature=0.,top_p=.95,top_k=20),speculative_depth=3,
seed=1,stop_token_ids=set(tokenizer.eos_token_ids),mtp_history_policy='committed',verify_strategy='batched',capture_final_state=True,
session_bank=bank,session_id='issue109',store_prefix_snapshot=False,
token_callback=lambda ids:print(json.dumps(dict(case='tokens',phase=phase,ids=ids)),flush=True))
assert result.tokens==expected['tokens'],(result.tokens,expected['tokens'])
assert result.stats.cached_tokens==expected['cached'],(result.stats.cached_tokens,expected['cached'])
assert index==len(list((native/phase).glob('*.json')))
final=result.final_state
assert final.safe_to_commit
for part,a in [('logits',final.final_logits),('hidden',final.final_hidden)]:
data=np.asarray(a.astype(mx.float32)).tobytes()
assert data==(native/f'{phase}-{part}.f32').read_bytes(),(phase,part)
def states(cache):
def array(a):
if a is None: return None
values=np.asarray(a.astype(mx.float32)).copy()
bits=hashlib.sha256(values.tobytes()).hexdigest()
values[values==0]=0
return dict(shape=list(a.shape),sha256=bits,value_sha256=hashlib.sha256(values.tobytes()).hexdigest())
return [[array(a) for a in state] if state is not None else None for state in snapshot_cache(cache).states]
arrays=dict(trunk=states(final.final_trunk_cache),mtp=states(final.final_committed_mtp_cache))
(out/f'{phase}-arrays.json').write_text(json.dumps(arrays))
native_arrays=json.loads((native/f'{phase}-arrays.json').read_text())
zero_sign_differences=[]
for family in arrays:
assert len(arrays[family])==len(native_arrays[family])
for layer,(actual,want) in enumerate(zip(arrays[family],native_arrays[family])):
if actual is None or want is None:
assert actual==want
continue
assert len(actual)==len(want)
for slot,(a,b) in enumerate(zip(actual,want)):
if a is None or b is None:
assert a==b
continue
assert a['shape']==b['shape'] and a['value_sha256']==b['value_sha256'],(phase,family,layer,slot,a,b)
if a['sha256']!=b['sha256']: zero_sign_differences.append([family,layer,slot])
assert zero_sign_differences in ([], [['trunk',35,0]]), zero_sign_differences
# Retain raw evidence of the prefill signed-zero mismatch; never hide it in
# the bit-exact verify/logit checks or substitute a numerical tolerance.
key=np.asarray(snapshot_cache(final.final_trunk_cache).states[35][0].astype(mx.float32))
(out/f'{phase}-key35.f32').write_bytes(key.tobytes())
want=np.fromfile(native/f'{phase}-key35.f32',np.float32).reshape(key.shape)
different=key.view(np.uint32)!=want.view(np.uint32)
assert np.all(key[different]==0) and np.all(want[different]==0)
assert np.argwhere(different).tolist() in ([], [[0,1,0,2]])
print(json.dumps(dict(case='cache_arrays',phase=phase,values_exact=True,zero_sign_differences=zero_sign_differences,key35_bit_coordinates=np.argwhere(different).tolist())),flush=True)
head=snapshot_cache(final.final_committed_mtp_cache)
key=ids+result.tokens
assert bank.put(runtime=rt,token_ids=key,cache=final.final_trunk_cache,logits=final.final_logits,hidden=final.final_hidden,
hidden_variant='post_norm',session_id='issue109',mtp_history_policy='committed',mtp_history_snapshot=head,
snapshot_epoch=len(key),mtp_snapshot_epoch=len(key)) is not None
print(json.dumps(dict(case='turn',phase=phase,tokens=result.tokens,cached=result.stats.cached_tokens,verify_calls=index,final_exact=True)),flush=True)