Apply Qwen NFC normalization before token splitting

Cargo.lock

@@ -834,6 +834,7 @@ dependencies = [
  "shlex",
  "time",
  "turbovault-parser",
+ "unicode-normalization-alignments",
  "ureq",
  "url",
  "usvg",
@@ -3948,6 +3949,15 @@ version = "0.1.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "3b09c83c3c29d37506a3e260c08c03743a6bb66a9cd432c6934ab501a190571f"
 
+[[package]]
+name = "unicode-normalization-alignments"
+version = "0.1.12"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "43f613e4fa046e69818dd287fdc4bc78175ff20331479dab6e1b0f98d57062de"
+dependencies = [
+ "smallvec",
+]
+
 [[package]]
 name = "unicode-properties"
 version = "0.1.4"

Cargo.toml

@@ -35,6 +35,7 @@ sha2 = "0.11.0"
 shlex = "2.0.1"
 time = { version = "0.3.54", features = ["formatting", "parsing"] }
 turbovault-parser = "1.6.0"
+unicode-normalization-alignments = "0.1.12"
 ureq = { version = "3.3.0", default-features = false, features = ["rustls"] }
 url = "2.5.8"
 usvg = "0.45.1"

docs/issue111-tokenizer/README.md

@@ -87,3 +87,10 @@ denied, release app bundle with ad-hoc signing, and 310 application + 8 tool tes
 (228 opt-in tests ignored by the ordinary gate). Pinned upstream tokenizer tests:
 5 passed; the one 27B Quality-pack test skipped because that distinct model pack
 is not installed. The installed Flash Next model was exercised above.
+
+## Follow-up #116
+
+NFC is now implemented. The component-stage test still checks every original
+normalization-free split/BPE ID; it additionally checks the full tokenizer against
+all `full_ids` and the normalized decode, closing the one-case gap documented
+above. See `../issue116-nfc/` for full normalization and cache migration receipts.

docs/issue116-nfc/README.md

@@ -0,0 +1,32 @@
+# Qwen NFC normalization (#116)
+
+Qwen requires the installed artifact's `NFC` normalizer at load time and normalizes each ordinary text span before the trained Qwen split/BPE. Rendered added tokens are extracted first and remain literal boundaries. Plain text keeps its existing literal-marker semantics. DeepSeek JoyAI and GLM GPT4 token processing are unchanged.
+
+The Rust dependency `unicode-normalization-alignments 0.1.12` is the same normalization implementation used by [tokenizers 0.22.2](https://github.com/huggingface/tokenizers/blob/v0.22.2/tokenizers/Cargo.toml), the pinned oracle's installed tokenizer. Its Unicode tables are version 9.0.0; substituting newer tables would change the oracle contract. Python is only used to generate independent reference receipts, never in production.
+
+## Reference and isolated verification
+
+Pinned MTPLX: `9882703f3105363ddc37eca9f97aa09a1d387112`; Bare-Speed artifact revision: `74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`. Tokenizer SHA-256: `0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3`. DS4 revision: `ec7642cdd9ec81d01ad4b1fd8f8a3d1511533748`.
+
+- All 1,112,064 Unicode scalars in `a + scalar + acute + dot-below + separator` context match the oracle normalization digest `1905b5f60f629850aa35dfc8ccb11fa6ae4d1fb0c79efd6774d36ca8e3e64eab`.
+- 13,281 exact normalized-text/plain-ID/rendered-ID/decode cases include canonical decompositions, reordered and leading marks, long combining sequences, all 11,172 Hangul syllables, already-normalized text, compatibility characters that NFC must preserve, and every added-token boundary. Python Unicode data only enumerates inputs; expected results come from the installed oracle normalizer/tokenizer.
+- Eight three-message requests cover direct/low/medium/xhigh, with and without a decomposed system prompt. Full encoding and history plus continuation match the pinned server exactly. Context counts are 52/62, 85/91, 55/65, and 97/103 respectively.
+- The original #116 text now emits `[52033,64,18135,220,52033,220,12256,100,52033,65]` and decodes to the oracle's composed form.
+- All prior #111 component split/BPE expectations remain checked independently before normalization. The full native tokenizer now matches all 5,752 full-oracle cases, including the previously excluded NFC difference. All-number-category and exhaustive split controls remain exact.
+- GLM and DeepSeek each retain exact DS4 plain/rendered IDs on all 19 shared controls. Existing DS4 chat render controls also run unchanged.
+
+## Cache and caller review
+
+Every plain, rendered, full conversation, history, continuation, system-prompt and context-count path reaches the Qwen normalizer. Generation output bytes are not re-tokenized during a compatible completed-history continuation; its generated KV remains exact.
+
+Token-prefix comparison alone was insufficient: a completed-conversation tag can authorize appending to old token IDs without rendering history. Qwen conversation keys now include a per-field NFC discriminator only when that field changes under the pinned normalizer. Raw histories remain distinct, appended-history keys remain prefixes, and pre-NFC affected keys cannot match new keys. All producers and consumers use the actual family: live requests, persistent checkpoint selection, transient KV lookup/store, bootstrap, compaction and failure cleanup. Both DS4 families and already-normalized Qwen keys remain byte-for-byte unchanged. No checkpoint-format bump or unconditional invalidation.
+
+The installed-model test prefills the actual legacy completed-history IDs (92 tokens), assigns their pre-change conversation tag, and checks both live and disk request paths. Both miss safely and generate the normalized 91-token prompt with identical complete state. The new live and disk continuations both reuse 93 tokens and have identical complete snapshot hashes. The unchanged #111 cache test controls compatible histories.
+
+This is an MTPLX behavior correction; direct-mode nonempty reasoning-history rendering remains the separate #117 issue. Direct fixtures here retain that known boundary by using empty reasoning, while every reasoning mode includes decomposed historical reasoning.
+
+## Reproduce
+
+Generate reference files using `tools/issue116-reference.py PINNED_MTPLX MODEL_DIRECTORY OUTPUT_DIRECTORY` in the isolated reference Python environment. Set `DS4_QWEN38_ARTIFACTS` and `DS4_ISSUE116_ORACLE` to the model and receipt directories. Build `cargo test --all-features --bin ds4-server --no-run`, copy its executable outside target, then run `issue116_nfc_tokens_and_requests --ignored --nocapture --test-threads=1`. Set `DS4_ISSUE116_DIR` to a fresh writable directory and run `issue116_normalized_token_cache` serially under `test-supervisor 114688 60 120 --command`, with `MTPLX_CONTEXT_COPY=0`. Ordinary tests also cover composition, rejected normalizer kinds, affected/unchanged key discrimination and prefix compatibility without model assets.
+
+All four commit gates passed: formatting, warning-free all-target/all-feature Clippy, signed release app bundle, and 316 application + 8 tool tests (239 opt-in tests ignored). Final executable isolated controls all passed; logs and supervised resource receipts are included. The compatible #111 cache retains 95-token live/disk reuse with identical state.

docs/issue116-nfc/bundle.log

@@ -0,0 +1,9 @@
+cargo build --release --all-targets --all-features
+   Compiling unicode-normalization-alignments v0.1.12
+   Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
+    Finished `release` profile [optimized] target(s) in 38.95s
+warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
+note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
+cargo packager --release --formats app
+codesign --force --deep --sign - --timestamp=none target/release/DS4Server.app
+target/release/DS4Server.app: replacing existing signature

docs/issue116-nfc/clippy.log

@@ -0,0 +1,5 @@
+   Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
+    Checking unicode-normalization-alignments v0.1.12
+    Finished `dev` profile [unoptimized + debuginfo] target(s) in 15.51s
+warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
+note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`

docs/issue116-nfc/compatible.output.log

@@ -0,0 +1,13 @@
+
+running 1 test
+test engine::metal::qwen_mtplx::owner::issue111_product::issue111_token_prefix_cache ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
+ds4: Metal 4 tensor API enabled for Tensor kernels
+ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
+ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx
+[qwen4_exp] ngram table: streamed from SSD (file-backed pages, hot cache 1023M, prefetch on) — resident only at >=160G RAM or MTPLX_NGRAM_RESIDENT=1
+[mtplx] verify SDPA head-chunked: q_len=24 heads_per_chunk=1 chunks=24
+{"case":"token_cache","old_tokens":89,"new_tokens":93,"old_prefix_miss":true,"warm_cached":95,"disk_cached":95,"warm_disk_state_exact":true}
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 554 filtered out; finished in 16.10s
+

docs/issue116-nfc/compatible.resources.log

@@ -0,0 +1,152 @@
+{"event":"test_supervisor_started","pid":2616,"limit_bytes":120259084288,"start_seconds":60.0,"idle_seconds":120.0,"sample_ms":100}
+{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":159333,"resident_bytes":688128,"physical_bytes":213064,"peak_physical_bytes":213064,"disk_read_bytes":0,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":110,"usage":{"cpu_time_ns":103381000,"resident_bytes":52510720,"physical_bytes":31801872,"peak_physical_bytes":31801872,"disk_read_bytes":14061568,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":220,"usage":{"cpu_time_ns":213453458,"resident_bytes":71516160,"physical_bytes":50823720,"peak_physical_bytes":50823720,"disk_read_bytes":14061568,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":330,"usage":{"cpu_time_ns":323515291,"resident_bytes":89341952,"physical_bytes":67846720,"peak_physical_bytes":67846720,"disk_read_bytes":14061568,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":440,"usage":{"cpu_time_ns":431409333,"resident_bytes":97910784,"physical_bytes":76431960,"peak_physical_bytes":76431960,"disk_read_bytes":14061568,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":550,"usage":{"cpu_time_ns":530015375,"resident_bytes":121749504,"physical_bytes":95912656,"peak_physical_bytes":95912656,"disk_read_bytes":14098432,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":660,"usage":{"cpu_time_ns":689697041,"resident_bytes":831029248,"physical_bytes":785352072,"peak_physical_bytes":785302920,"disk_read_bytes":733700096,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":771,"usage":{"cpu_time_ns":893331791,"resident_bytes":2305851392,"physical_bytes":3324414448,"peak_physical_bytes":3324365296,"disk_read_bytes":2256781312,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":881,"usage":{"cpu_time_ns":1072813625,"resident_bytes":3518218240,"physical_bytes":5507304832,"peak_physical_bytes":5507304832,"disk_read_bytes":3666657280,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":991,"usage":{"cpu_time_ns":1211061125,"resident_bytes":4051468288,"physical_bytes":7010570032,"peak_physical_bytes":7010570032,"disk_read_bytes":5145395200,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1101,"usage":{"cpu_time_ns":1348695041,"resident_bytes":4583047168,"physical_bytes":8512065712,"peak_physical_bytes":8512065712,"disk_read_bytes":6632734720,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1204,"usage":{"cpu_time_ns":1472381625,"resident_bytes":5095522304,"physical_bytes":9970274864,"peak_physical_bytes":9970274864,"disk_read_bytes":8039141376,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1312,"usage":{"cpu_time_ns":1603221541,"resident_bytes":5604982784,"physical_bytes":11449406408,"peak_physical_bytes":11449406408,"disk_read_bytes":9446707200,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1421,"usage":{"cpu_time_ns":1735014458,"resident_bytes":6109708288,"physical_bytes":12923786640,"peak_physical_bytes":12923753872,"disk_read_bytes":10896543744,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1532,"usage":{"cpu_time_ns":1868273291,"resident_bytes":6643204096,"physical_bytes":14426904360,"peak_physical_bytes":14426871592,"disk_read_bytes":12358094848,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1637,"usage":{"cpu_time_ns":1985075625,"resident_bytes":7154319360,"physical_bytes":15044974712,"peak_physical_bytes":15044974712,"disk_read_bytes":13791170560,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1746,"usage":{"cpu_time_ns":2142866250,"resident_bytes":7592755200,"physical_bytes":17291909672,"peak_physical_bytes":17291827752,"disk_read_bytes":15258013696,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1854,"usage":{"cpu_time_ns":2271023583,"resident_bytes":8941387776,"physical_bytes":18695495168,"peak_physical_bytes":18695495168,"disk_read_bytes":16723083264,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1964,"usage":{"cpu_time_ns":2422701375,"resident_bytes":9112027136,"physical_bytes":20726849144,"peak_physical_bytes":20726849144,"disk_read_bytes":18166054912,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2074,"usage":{"cpu_time_ns":2571553750,"resident_bytes":10058055680,"physical_bytes":22642844000,"peak_physical_bytes":22642811232,"disk_read_bytes":19648315392,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2184,"usage":{"cpu_time_ns":2698395041,"resident_bytes":10197204992,"physical_bytes":23727694272,"peak_physical_bytes":23727661504,"disk_read_bytes":21121269760,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2294,"usage":{"cpu_time_ns":2809846958,"resident_bytes":10242228224,"physical_bytes":24744763912,"peak_physical_bytes":24744763912,"disk_read_bytes":22499573760,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2398,"usage":{"cpu_time_ns":2929312333,"resident_bytes":10501488640,"physical_bytes":25794454288,"peak_physical_bytes":25794159376,"disk_read_bytes":23878340608,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2506,"usage":{"cpu_time_ns":3059322833,"resident_bytes":11000840192,"physical_bytes":27370579112,"peak_physical_bytes":27370316968,"disk_read_bytes":25353457664,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2615,"usage":{"cpu_time_ns":3177136000,"resident_bytes":11476484096,"physical_bytes":27915871760,"peak_physical_bytes":27915838992,"disk_read_bytes":26799017984,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2717,"usage":{"cpu_time_ns":3305236791,"resident_bytes":11890180096,"physical_bytes":29299156856,"peak_physical_bytes":29299156856,"disk_read_bytes":28168097792,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2828,"usage":{"cpu_time_ns":3444665083,"resident_bytes":12396363776,"physical_bytes":30776699152,"peak_physical_bytes":30776699152,"disk_read_bytes":29653929984,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2938,"usage":{"cpu_time_ns":3587241875,"resident_bytes":13287374848,"physical_bytes":32637332376,"peak_physical_bytes":32637332376,"disk_read_bytes":31120920576,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3048,"usage":{"cpu_time_ns":3718476625,"resident_bytes":13419347968,"physical_bytes":33715055680,"peak_physical_bytes":33715055680,"disk_read_bytes":32592896000,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3158,"usage":{"cpu_time_ns":3862745000,"resident_bytes":13935034368,"physical_bytes":35305238000,"peak_physical_bytes":35305041392,"disk_read_bytes":34075648000,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3268,"usage":{"cpu_time_ns":4221988833,"resident_bytes":15545729024,"physical_bytes":38748926696,"peak_physical_bytes":38748828392,"disk_read_bytes":34964135936,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3368,"usage":{"cpu_time_ns":4520768791,"resident_bytes":16693166080,"physical_bytes":41759290952,"peak_physical_bytes":41759028808,"disk_read_bytes":35882065920,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3469,"usage":{"cpu_time_ns":4698517333,"resident_bytes":16840884224,"physical_bytes":42929010344,"peak_physical_bytes":42929010344,"disk_read_bytes":37211987968,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3570,"usage":{"cpu_time_ns":4818584041,"resident_bytes":16932339712,"physical_bytes":43779225328,"peak_physical_bytes":43778963184,"disk_read_bytes":38526349312,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3671,"usage":{"cpu_time_ns":4960413583,"resident_bytes":17113464832,"physical_bytes":44463863728,"peak_physical_bytes":44463503280,"disk_read_bytes":39893979136,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3780,"usage":{"cpu_time_ns":5109427791,"resident_bytes":17600004096,"physical_bytes":45710293344,"peak_physical_bytes":45710244192,"disk_read_bytes":41359953920,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3883,"usage":{"cpu_time_ns":5259375583,"resident_bytes":18051858432,"physical_bytes":47025060504,"peak_physical_bytes":47025011352,"disk_read_bytes":42747498496,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3983,"usage":{"cpu_time_ns":5408296625,"resident_bytes":18550669312,"physical_bytes":48495131696,"peak_physical_bytes":48495131696,"disk_read_bytes":44065443840,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4087,"usage":{"cpu_time_ns":5602608375,"resident_bytes":19682885632,"physical_bytes":50595413904,"peak_physical_bytes":50595348368,"disk_read_bytes":45456543744,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4196,"usage":{"cpu_time_ns":5770821208,"resident_bytes":20535820288,"physical_bytes":52396327376,"peak_physical_bytes":52396327376,"disk_read_bytes":46919229440,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4306,"usage":{"cpu_time_ns":5921878458,"resident_bytes":20966178816,"physical_bytes":53794374408,"peak_physical_bytes":53794374408,"disk_read_bytes":48413712384,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4415,"usage":{"cpu_time_ns":6067369583,"resident_bytes":21013250048,"physical_bytes":54810903400,"peak_physical_bytes":54810903400,"disk_read_bytes":49855766528,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4524,"usage":{"cpu_time_ns":6211329875,"resident_bytes":21122859008,"physical_bytes":55889937400,"peak_physical_bytes":55889937400,"disk_read_bytes":51292209152,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4626,"usage":{"cpu_time_ns":6384336875,"resident_bytes":21641871360,"physical_bytes":57354798456,"peak_physical_bytes":57354765688,"disk_read_bytes":52606058496,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4730,"usage":{"cpu_time_ns":6567795333,"resident_bytes":22091104256,"physical_bytes":58773587632,"peak_physical_bytes":58773554864,"disk_read_bytes":53949251584,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4830,"usage":{"cpu_time_ns":6769414916,"resident_bytes":22279389184,"physical_bytes":59806173280,"peak_physical_bytes":59806189664,"disk_read_bytes":55228735488,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4932,"usage":{"cpu_time_ns":6988532541,"resident_bytes":22465511424,"physical_bytes":61706635792,"peak_physical_bytes":61706635792,"disk_read_bytes":56615804928,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5035,"usage":{"cpu_time_ns":7222857666,"resident_bytes":23527800832,"physical_bytes":63211031992,"peak_physical_bytes":63210999224,"disk_read_bytes":57900920832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5136,"usage":{"cpu_time_ns":7428005166,"resident_bytes":23248404480,"physical_bytes":64513920560,"peak_physical_bytes":64513805872,"disk_read_bytes":59220209664,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5237,"usage":{"cpu_time_ns":7640257708,"resident_bytes":23509811200,"physical_bytes":65805291584,"peak_physical_bytes":65805291584,"disk_read_bytes":60609933312,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5338,"usage":{"cpu_time_ns":7893449166,"resident_bytes":23703486464,"physical_bytes":67129790816,"peak_physical_bytes":67129790816,"disk_read_bytes":61883437056,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5439,"usage":{"cpu_time_ns":8069151958,"resident_bytes":23393878016,"physical_bytes":68120351144,"peak_physical_bytes":68120351144,"disk_read_bytes":63226310656,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5539,"usage":{"cpu_time_ns":8279660416,"resident_bytes":24032002048,"physical_bytes":69797450752,"peak_physical_bytes":69797401600,"disk_read_bytes":64541749248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5640,"usage":{"cpu_time_ns":8490036291,"resident_bytes":24638685184,"physical_bytes":71413814848,"peak_physical_bytes":71413798464,"disk_read_bytes":65923428352,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5748,"usage":{"cpu_time_ns":8727857625,"resident_bytes":24056414208,"physical_bytes":72730597120,"peak_physical_bytes":72730597120,"disk_read_bytes":67275468800,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5858,"usage":{"cpu_time_ns":8991819625,"resident_bytes":25029771264,"physical_bytes":74902035040,"peak_physical_bytes":74902035040,"disk_read_bytes":68625006592,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5965,"usage":{"cpu_time_ns":9121539500,"resident_bytes":25102336000,"physical_bytes":75179124592,"peak_physical_bytes":75179124592,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6075,"usage":{"cpu_time_ns":9242528083,"resident_bytes":25102057472,"physical_bytes":75196868464,"peak_physical_bytes":75196868464,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6182,"usage":{"cpu_time_ns":9349408541,"resident_bytes":25102073856,"physical_bytes":75202422640,"peak_physical_bytes":75202422640,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6292,"usage":{"cpu_time_ns":9474436375,"resident_bytes":25102073856,"physical_bytes":75220346736,"peak_physical_bytes":75220346736,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6402,"usage":{"cpu_time_ns":9583537500,"resident_bytes":25102024704,"physical_bytes":75228505968,"peak_physical_bytes":75228505968,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6512,"usage":{"cpu_time_ns":9695956666,"resident_bytes":25102024704,"physical_bytes":75234535280,"peak_physical_bytes":75234535280,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6614,"usage":{"cpu_time_ns":9795948333,"resident_bytes":25102024704,"physical_bytes":75236403056,"peak_physical_bytes":75236403056,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6724,"usage":{"cpu_time_ns":9909111041,"resident_bytes":25102024704,"physical_bytes":75239532400,"peak_physical_bytes":75239548784,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6832,"usage":{"cpu_time_ns":10020431833,"resident_bytes":25102041088,"physical_bytes":75246348144,"peak_physical_bytes":75246348144,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6932,"usage":{"cpu_time_ns":10129025583,"resident_bytes":25102057472,"physical_bytes":75252885360,"peak_physical_bytes":75252885360,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7043,"usage":{"cpu_time_ns":10241415750,"resident_bytes":25100091392,"physical_bytes":75266762608,"peak_physical_bytes":75266762608,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7153,"usage":{"cpu_time_ns":10373631791,"resident_bytes":25083215872,"physical_bytes":75319994224,"peak_physical_bytes":75319994224,"disk_read_bytes":68647342080,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7253,"usage":{"cpu_time_ns":10522411041,"resident_bytes":25129091072,"physical_bytes":75515062344,"peak_physical_bytes":75514947656,"disk_read_bytes":68980805632,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7363,"usage":{"cpu_time_ns":10790531958,"resident_bytes":25126961152,"physical_bytes":75710245008,"peak_physical_bytes":75710245008,"disk_read_bytes":70304993280,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7474,"usage":{"cpu_time_ns":10865107875,"resident_bytes":25128386560,"physical_bytes":76157790352,"peak_physical_bytes":76157790352,"disk_read_bytes":70304993280,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7584,"usage":{"cpu_time_ns":10946926166,"resident_bytes":25082789888,"physical_bytes":76402518160,"peak_physical_bytes":76402518160,"disk_read_bytes":70305026048,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7694,"usage":{"cpu_time_ns":11068180291,"resident_bytes":25067274240,"physical_bytes":76430190736,"peak_physical_bytes":76430190736,"disk_read_bytes":70305026048,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7804,"usage":{"cpu_time_ns":11173154791,"resident_bytes":25081643008,"physical_bytes":76525906112,"peak_physical_bytes":76525906112,"disk_read_bytes":70305042432,"disk_write_bytes":8388608}}
+{"event":"test_resource_sample","elapsed_ms":7914,"usage":{"cpu_time_ns":11273477041,"resident_bytes":25181995008,"physical_bytes":76625209704,"peak_physical_bytes":76625209704,"disk_read_bytes":70305050624,"disk_write_bytes":100663296}}
+{"event":"test_resource_sample","elapsed_ms":8015,"usage":{"cpu_time_ns":11366888333,"resident_bytes":25279889408,"physical_bytes":76639070616,"peak_physical_bytes":76639070616,"disk_read_bytes":70306758656,"disk_write_bytes":201326592}}
+{"event":"test_resource_sample","elapsed_ms":8125,"usage":{"cpu_time_ns":11469821625,"resident_bytes":25329893376,"physical_bytes":76646214088,"peak_physical_bytes":76646214088,"disk_read_bytes":70306762752,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":8235,"usage":{"cpu_time_ns":11579868041,"resident_bytes":25362726912,"physical_bytes":76647393736,"peak_physical_bytes":76647524808,"disk_read_bytes":70306762752,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":8345,"usage":{"cpu_time_ns":11689904583,"resident_bytes":25399459840,"physical_bytes":76649490888,"peak_physical_bytes":76649490888,"disk_read_bytes":70306762752,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":8455,"usage":{"cpu_time_ns":11799819875,"resident_bytes":25432522752,"physical_bytes":76651063776,"peak_physical_bytes":76651063776,"disk_read_bytes":70306762752,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":8566,"usage":{"cpu_time_ns":11909971166,"resident_bytes":25440600064,"physical_bytes":76652734944,"peak_physical_bytes":76652734944,"disk_read_bytes":70306762752,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":8676,"usage":{"cpu_time_ns":12020029750,"resident_bytes":25441632256,"physical_bytes":76653865440,"peak_physical_bytes":76653865440,"disk_read_bytes":70306762752,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":8786,"usage":{"cpu_time_ns":12130060375,"resident_bytes":25442582528,"physical_bytes":76654914016,"peak_physical_bytes":76654914016,"disk_read_bytes":70306762752,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":8886,"usage":{"cpu_time_ns":12233129875,"resident_bytes":25444220928,"physical_bytes":76654995936,"peak_physical_bytes":76655192544,"disk_read_bytes":70306762752,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":8996,"usage":{"cpu_time_ns":12358889083,"resident_bytes":25461506048,"physical_bytes":76841004520,"peak_physical_bytes":76841004520,"disk_read_bytes":70320033792,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":9105,"usage":{"cpu_time_ns":12422631208,"resident_bytes":25461506048,"physical_bytes":76863368680,"peak_physical_bytes":76863368680,"disk_read_bytes":70320033792,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":9215,"usage":{"cpu_time_ns":12551216333,"resident_bytes":25461506048,"physical_bytes":76881374696,"peak_physical_bytes":76881374696,"disk_read_bytes":70320033792,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":9325,"usage":{"cpu_time_ns":12671464458,"resident_bytes":25457999872,"physical_bytes":76894023144,"peak_physical_bytes":76894023144,"disk_read_bytes":70320033792,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":9432,"usage":{"cpu_time_ns":12807240250,"resident_bytes":25419776000,"physical_bytes":76912913896,"peak_physical_bytes":76912913896,"disk_read_bytes":70320033792,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":9542,"usage":{"cpu_time_ns":12932071333,"resident_bytes":25419857920,"physical_bytes":76928577000,"peak_physical_bytes":76928577000,"disk_read_bytes":70320033792,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":9648,"usage":{"cpu_time_ns":13046913750,"resident_bytes":25435455488,"physical_bytes":76950138512,"peak_physical_bytes":76950138512,"disk_read_bytes":70333255680,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":9752,"usage":{"cpu_time_ns":13150957666,"resident_bytes":25437454336,"physical_bytes":76951498408,"peak_physical_bytes":76951498408,"disk_read_bytes":70333255680,"disk_write_bytes":234967040}}
+{"event":"test_resource_sample","elapsed_ms":9854,"usage":{"cpu_time_ns":13257489500,"resident_bytes":25480970240,"physical_bytes":77005303512,"peak_physical_bytes":77005303512,"disk_read_bytes":70333255680,"disk_write_bytes":276910080}}
+{"event":"test_resource_sample","elapsed_ms":9964,"usage":{"cpu_time_ns":13358843291,"resident_bytes":25557008384,"physical_bytes":77077835552,"peak_physical_bytes":77077835552,"disk_read_bytes":70333263872,"disk_write_bytes":369184768}}
+{"event":"test_resource_sample","elapsed_ms":10074,"usage":{"cpu_time_ns":13461909708,"resident_bytes":25558122496,"physical_bytes":77078376224,"peak_physical_bytes":77078376224,"disk_read_bytes":70333267968,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":10187,"usage":{"cpu_time_ns":13581440833,"resident_bytes":25569329152,"physical_bytes":77130297528,"peak_physical_bytes":77130297528,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":10297,"usage":{"cpu_time_ns":13702236666,"resident_bytes":25569509376,"physical_bytes":77171241144,"peak_physical_bytes":77171241144,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":10408,"usage":{"cpu_time_ns":13816639250,"resident_bytes":25569984512,"physical_bytes":77272903912,"peak_physical_bytes":77272903912,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":10518,"usage":{"cpu_time_ns":13925399333,"resident_bytes":25570869248,"physical_bytes":77272985880,"peak_physical_bytes":77272985880,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":10620,"usage":{"cpu_time_ns":14025605583,"resident_bytes":25571229696,"physical_bytes":77272985880,"peak_physical_bytes":77272985880,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":10730,"usage":{"cpu_time_ns":14132666416,"resident_bytes":25571606528,"physical_bytes":77272985880,"peak_physical_bytes":77272985880,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":10841,"usage":{"cpu_time_ns":14239541666,"resident_bytes":25572311040,"physical_bytes":77272789296,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":10951,"usage":{"cpu_time_ns":14347124375,"resident_bytes":25572884480,"physical_bytes":77272805680,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":11061,"usage":{"cpu_time_ns":14454963541,"resident_bytes":25573490688,"physical_bytes":77272822064,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":11171,"usage":{"cpu_time_ns":14561838833,"resident_bytes":25573785600,"physical_bytes":77272822064,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":11281,"usage":{"cpu_time_ns":14668830083,"resident_bytes":25574260736,"physical_bytes":77272723784,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":11391,"usage":{"cpu_time_ns":14778161125,"resident_bytes":25575030784,"physical_bytes":77272609096,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":11502,"usage":{"cpu_time_ns":14890348666,"resident_bytes":25575047168,"physical_bytes":77272625480,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":11612,"usage":{"cpu_time_ns":15002012041,"resident_bytes":25575047168,"physical_bytes":77272625480,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":11722,"usage":{"cpu_time_ns":15113066541,"resident_bytes":25575063552,"physical_bytes":77272641864,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":11832,"usage":{"cpu_time_ns":15224825458,"resident_bytes":25575063552,"physical_bytes":77269578056,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":11938,"usage":{"cpu_time_ns":15331116583,"resident_bytes":25575079936,"physical_bytes":77269594440,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":12048,"usage":{"cpu_time_ns":15442872541,"resident_bytes":25575079936,"physical_bytes":77269594440,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":12158,"usage":{"cpu_time_ns":15554611208,"resident_bytes":25575112704,"physical_bytes":77269627208,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":12268,"usage":{"cpu_time_ns":15664505041,"resident_bytes":25560989696,"physical_bytes":77234762056,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":12378,"usage":{"cpu_time_ns":15774243916,"resident_bytes":25561055232,"physical_bytes":77234762056,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":12489,"usage":{"cpu_time_ns":15884788333,"resident_bytes":25561137152,"physical_bytes":77234762056,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":12599,"usage":{"cpu_time_ns":15990170250,"resident_bytes":25561350144,"physical_bytes":77100544328,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":12709,"usage":{"cpu_time_ns":16100233833,"resident_bytes":25589661696,"physical_bytes":77100544328,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":12815,"usage":{"cpu_time_ns":16206177125,"resident_bytes":25621118976,"physical_bytes":77100544328,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":12925,"usage":{"cpu_time_ns":16316104000,"resident_bytes":25655721984,"physical_bytes":77099987272,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":13035,"usage":{"cpu_time_ns":16434337666,"resident_bytes":25668321280,"physical_bytes":77227569480,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":13140,"usage":{"cpu_time_ns":16549080708,"resident_bytes":25668583424,"physical_bytes":77238792520,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":13250,"usage":{"cpu_time_ns":16666574958,"resident_bytes":25668599808,"physical_bytes":77239759176,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":13360,"usage":{"cpu_time_ns":16802376708,"resident_bytes":25668599808,"physical_bytes":77243625800,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":13471,"usage":{"cpu_time_ns":16941121000,"resident_bytes":25668599808,"physical_bytes":77243920712,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":13581,"usage":{"cpu_time_ns":17082610250,"resident_bytes":25668599808,"physical_bytes":77244363080,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":13691,"usage":{"cpu_time_ns":17217226291,"resident_bytes":25668616192,"physical_bytes":77244920136,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":13800,"usage":{"cpu_time_ns":17335255875,"resident_bytes":25668616192,"physical_bytes":77246148936,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":13910,"usage":{"cpu_time_ns":17443025166,"resident_bytes":25669156864,"physical_bytes":77246165344,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14021,"usage":{"cpu_time_ns":17549570458,"resident_bytes":25669812224,"physical_bytes":77246165344,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14131,"usage":{"cpu_time_ns":17656631000,"resident_bytes":25670336512,"physical_bytes":77246165344,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14241,"usage":{"cpu_time_ns":17764435333,"resident_bytes":25671122944,"physical_bytes":77246165344,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14351,"usage":{"cpu_time_ns":17869649500,"resident_bytes":25671614464,"physical_bytes":77246165344,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14461,"usage":{"cpu_time_ns":17978232375,"resident_bytes":25672007680,"physical_bytes":77246165344,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14571,"usage":{"cpu_time_ns":18084128250,"resident_bytes":25672597504,"physical_bytes":77246181752,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14681,"usage":{"cpu_time_ns":18191435958,"resident_bytes":25672761344,"physical_bytes":77246067088,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14782,"usage":{"cpu_time_ns":18289298375,"resident_bytes":25677266944,"physical_bytes":77245936016,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14892,"usage":{"cpu_time_ns":18400232041,"resident_bytes":25677414400,"physical_bytes":77245936016,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":14998,"usage":{"cpu_time_ns":18508431958,"resident_bytes":25677561856,"physical_bytes":77245936016,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":15108,"usage":{"cpu_time_ns":18619657958,"resident_bytes":25677692928,"physical_bytes":77242806672,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":15215,"usage":{"cpu_time_ns":18727824000,"resident_bytes":25677889536,"physical_bytes":77242806672,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":15325,"usage":{"cpu_time_ns":18838781250,"resident_bytes":25678053376,"physical_bytes":77242806672,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":15435,"usage":{"cpu_time_ns":18950425166,"resident_bytes":25678266368,"physical_bytes":77242806672,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":15545,"usage":{"cpu_time_ns":19062836125,"resident_bytes":25678299136,"physical_bytes":77242806672,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":15655,"usage":{"cpu_time_ns":19163880000,"resident_bytes":25671516160,"physical_bytes":77105487832,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":15765,"usage":{"cpu_time_ns":19274966083,"resident_bytes":20484456448,"physical_bytes":65867206072,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":15875,"usage":{"cpu_time_ns":19384641250,"resident_bytes":13847871488,"physical_bytes":55429313304,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":15985,"usage":{"cpu_time_ns":19494484375,"resident_bytes":157024256,"physical_bytes":36127721104,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_sample","elapsed_ms":16085,"usage":{"cpu_time_ns":19592187125,"resident_bytes":157024256,"physical_bytes":5089314448,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304}}
+{"event":"test_resource_summary","elapsed_ms":16196,"peak":{"cpu_time_ns":19592187125,"resident_bytes":25678299136,"physical_bytes":77272985880,"peak_physical_bytes":77273018672,"disk_read_bytes":70339035136,"disk_write_bytes":470114304},"error":null}

docs/issue116-nfc/deepseek-chat.output.log

@@ -0,0 +1,6 @@
+
+running 1 test
+test engine::tokenizer::tests::ds4_chat_matches_original_session_tokens ... ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 554 filtered out; finished in 0.14s
+

docs/issue116-nfc/deepseek-chat.resources.log

@@ -0,0 +1,4 @@
+{"event":"test_supervisor_started","pid":2559,"limit_bytes":4294967296,"start_seconds":60.0,"idle_seconds":120.0,"sample_ms":100}
+{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":214916,"resident_bytes":688128,"physical_bytes":213064,"peak_physical_bytes":213064,"disk_read_bytes":0,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":110,"usage":{"cpu_time_ns":107828625,"resident_bytes":65110016,"physical_bytes":42648128,"peak_physical_bytes":42648128,"disk_read_bytes":200704,"disk_write_bytes":0}}
+{"event":"test_resource_summary","elapsed_ms":220,"peak":{"cpu_time_ns":107828625,"resident_bytes":65110016,"physical_bytes":42648128,"peak_physical_bytes":42648128,"disk_read_bytes":200704,"disk_write_bytes":0},"error":null}

docs/issue116-nfc/glm-chat.output.log

@@ -0,0 +1,6 @@
+
+running 1 test
+test engine::tokenizer::tests::ds4_chat_matches_original_session_tokens ... ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 554 filtered out; finished in 0.22s
+

docs/issue116-nfc/glm-chat.resources.log

@@ -0,0 +1,5 @@
+{"event":"test_supervisor_started","pid":2556,"limit_bytes":4294967296,"start_seconds":60.0,"idle_seconds":120.0,"sample_ms":100}
+{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":167291,"resident_bytes":671744,"physical_bytes":196680,"peak_physical_bytes":213064,"disk_read_bytes":0,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":107,"usage":{"cpu_time_ns":105019333,"resident_bytes":84639744,"physical_bytes":60375616,"peak_physical_bytes":60375616,"disk_read_bytes":188416,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":217,"usage":{"cpu_time_ns":215093833,"resident_bytes":84000768,"physical_bytes":66290240,"peak_physical_bytes":66306648,"disk_read_bytes":188416,"disk_write_bytes":0}}
+{"event":"test_resource_summary","elapsed_ms":323,"peak":{"cpu_time_ns":215093833,"resident_bytes":84639744,"physical_bytes":66290240,"peak_physical_bytes":66306648,"disk_read_bytes":188416,"disk_write_bytes":0},"error":null}

docs/issue116-nfc/nfc.output.log

@@ -0,0 +1,15 @@
+
+running 1 test
+test engine::tokenizer::issue116::issue116_nfc_tokens_and_requests ... {"effort":"direct","system":"","context_tokens":52,"exact":true}
+{"effort":"direct","system":"Café 각.","context_tokens":62,"exact":true}
+{"effort":"low","system":"","context_tokens":85,"exact":true}
+{"effort":"low","system":"Café 각.","context_tokens":91,"exact":true}
+{"effort":"medium","system":"","context_tokens":55,"exact":true}
+{"effort":"medium","system":"Café 각.","context_tokens":65,"exact":true}
+{"effort":"xhigh","system":"","context_tokens":97,"exact":true}
+{"effort":"xhigh","system":"Café 각.","context_tokens":103,"exact":true}
+{"case":"nfc","scalar_sha256":"1905b5f60f629850aa35dfc8ccb11fa6ae4d1fb0c79efd6774d36ca8e3e64eab","token_cases":13281,"requests":8,"exact":true}
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 554 filtered out; finished in 2.30s
+

docs/issue116-nfc/nfc.resources.log

@@ -0,0 +1,44 @@
+{"event":"test_supervisor_started","pid":2513,"limit_bytes":4294967296,"start_seconds":60.0,"idle_seconds":120.0,"sample_ms":100}
+{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":155333,"resident_bytes":688128,"physical_bytes":213064,"peak_physical_bytes":213064,"disk_read_bytes":0,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":102,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":203,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":310,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":418,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":528,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":636,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":746,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":857,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":966,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1068,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1179,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1283,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1390,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1500,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1602,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1712,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1822,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1928,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2036,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2138,"usage":{"cpu_time_ns":2514083,"resident_bytes":32768,"physical_bytes":81992,"peak_physical_bytes":98376,"disk_read_bytes":1576960,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2249,"usage":{"cpu_time_ns":33891666,"resident_bytes":30785536,"physical_bytes":17121784,"peak_physical_bytes":17121784,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2359,"usage":{"cpu_time_ns":143979708,"resident_bytes":49102848,"physical_bytes":35455504,"peak_physical_bytes":35455504,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2466,"usage":{"cpu_time_ns":251499166,"resident_bytes":69959680,"physical_bytes":56230416,"peak_physical_bytes":56230416,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2577,"usage":{"cpu_time_ns":361222166,"resident_bytes":83099648,"physical_bytes":67863104,"peak_physical_bytes":67863104,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2687,"usage":{"cpu_time_ns":471158250,"resident_bytes":103497728,"physical_bytes":88146496,"peak_physical_bytes":88146496,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2797,"usage":{"cpu_time_ns":581650583,"resident_bytes":108052480,"physical_bytes":91734592,"peak_physical_bytes":91734592,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2907,"usage":{"cpu_time_ns":691689083,"resident_bytes":112984064,"physical_bytes":95847000,"peak_physical_bytes":95847000,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3017,"usage":{"cpu_time_ns":801739416,"resident_bytes":114311168,"physical_bytes":97174104,"peak_physical_bytes":97174104,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3127,"usage":{"cpu_time_ns":911768708,"resident_bytes":115621888,"physical_bytes":98484824,"peak_physical_bytes":98484824,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3237,"usage":{"cpu_time_ns":1021745500,"resident_bytes":116948992,"physical_bytes":99811928,"peak_physical_bytes":99811928,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3347,"usage":{"cpu_time_ns":1131864083,"resident_bytes":118439936,"physical_bytes":101302872,"peak_physical_bytes":101302872,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3458,"usage":{"cpu_time_ns":1241895833,"resident_bytes":121094144,"physical_bytes":103957080,"peak_physical_bytes":103957080,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3568,"usage":{"cpu_time_ns":1351928666,"resident_bytes":123355136,"physical_bytes":106218072,"peak_physical_bytes":106218072,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3678,"usage":{"cpu_time_ns":1461970458,"resident_bytes":124665856,"physical_bytes":107528792,"peak_physical_bytes":107528792,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3788,"usage":{"cpu_time_ns":1572019500,"resident_bytes":130154496,"physical_bytes":112788056,"peak_physical_bytes":112788056,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3898,"usage":{"cpu_time_ns":1682065750,"resident_bytes":160366592,"physical_bytes":140034648,"peak_physical_bytes":140034648,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4000,"usage":{"cpu_time_ns":1783676416,"resident_bytes":161054720,"physical_bytes":140296792,"peak_physical_bytes":140296792,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4108,"usage":{"cpu_time_ns":1891915958,"resident_bytes":161808384,"physical_bytes":141050456,"peak_physical_bytes":141050456,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4216,"usage":{"cpu_time_ns":2000209583,"resident_bytes":161906688,"physical_bytes":141148760,"peak_physical_bytes":141148760,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4327,"usage":{"cpu_time_ns":2110130125,"resident_bytes":162037760,"physical_bytes":141279832,"peak_physical_bytes":141279832,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4437,"usage":{"cpu_time_ns":2219777333,"resident_bytes":162037760,"physical_bytes":141279832,"peak_physical_bytes":141279832,"disk_read_bytes":14389248,"disk_write_bytes":0}}
+{"event":"test_resource_summary","elapsed_ms":4547,"peak":{"cpu_time_ns":2219777333,"resident_bytes":162037760,"physical_bytes":141279832,"peak_physical_bytes":141279832,"disk_read_bytes":14389248,"disk_write_bytes":0},"error":null}

docs/issue116-nfc/oracle.log

@@ -0,0 +1,4 @@
+[transformers] You are using a model of type `qwen4_exp` to instantiate a model of type ``. This may be expected if you are loading a checkpoint that shares a subset of the architecture (e.g., loading a `sam2_video` checkpoint into `Sam2Model`), but is otherwise not supported and can yield errors. Please verify that the checkpoint is compatible with the model you are instantiating.
+[tokenizer] restored the pre-tokenizer regex from tokenizer.json: the loaded tokenizer split combining marks off their letters (Devanagari, Thai, vowelled Arabic tokenized 25-90% longer than the model was trained on)
+[mtplx] chat turn segmentation {"enabled": true, "reason": "structure holds and all 12 probe renders give the single-call ids when cut", "tokenizer": "TokenizerWrapper"}
+{"cases": 13281, "requests": 8, "scalar_sha256": "1905b5f60f629850aa35dfc8ccb11fa6ae4d1fb0c79efd6774d36ca8e3e64eab", "tokenizer_sha256": "0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3", "tokenizers": "0.22.2"}

docs/issue116-nfc/prior.output.log

@@ -0,0 +1,16 @@
+
+running 3 tests
+test engine::tokenizer::issue111::issue111_ds4_families ... {"case":"ds4_control","family":"glm","cases":19,"plain_rendered_exact":true}
+{"case":"ds4_control","family":"deepseek","cases":19,"plain_rendered_exact":true}
+ok
+test engine::tokenizer::issue111::issue111_every_unicode_scalar_split ... {"case":"unicode_split","scalars":1112064,"pieces":2082513,"sha256":"7ef8755fb56062d9695b38bc132f325ea59c3c65f198e1b1ab20bff1afef632a","exact":true}
+ok
+test engine::tokenizer::issue111::issue111_qwen_tokens_and_requests ... {"case":"request","effort":"direct","context_tokens":59,"exact":true}
+{"case":"request","effort":"low","context_tokens":93,"exact":true}
+{"case":"request","effort":"medium","context_tokens":63,"exact":true}
+{"case":"request","effort":"xhigh","context_tokens":105,"exact":true}
+{"case":"tokenizer","number_count":1911,"cases":5752,"plain_rendered_roundtrip_exact":true}
+ok
+
+test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 552 filtered out; finished in 6.32s
+

docs/issue116-nfc/prior.resources.log

@@ -0,0 +1,60 @@
+{"event":"test_supervisor_started","pid":2533,"limit_bytes":4294967296,"start_seconds":60.0,"idle_seconds":120.0,"sample_ms":100}
+{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":169125,"resident_bytes":688128,"physical_bytes":213064,"peak_physical_bytes":213064,"disk_read_bytes":0,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":110,"usage":{"cpu_time_ns":101856083,"resident_bytes":83492864,"physical_bytes":59195968,"peak_physical_bytes":59195968,"disk_read_bytes":9576448,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":220,"usage":{"cpu_time_ns":211938000,"resident_bytes":67551232,"physical_bytes":29278808,"peak_physical_bytes":64815704,"disk_read_bytes":9576448,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":330,"usage":{"cpu_time_ns":319535416,"resident_bytes":81854464,"physical_bytes":47514176,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":440,"usage":{"cpu_time_ns":429264291,"resident_bytes":81444864,"physical_bytes":45072960,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":550,"usage":{"cpu_time_ns":539635250,"resident_bytes":81444864,"physical_bytes":45072960,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":660,"usage":{"cpu_time_ns":649102541,"resident_bytes":81625088,"physical_bytes":45253184,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":771,"usage":{"cpu_time_ns":758016916,"resident_bytes":81674240,"physical_bytes":45302336,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":881,"usage":{"cpu_time_ns":868806333,"resident_bytes":81723392,"physical_bytes":45351488,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":991,"usage":{"cpu_time_ns":979884750,"resident_bytes":81756160,"physical_bytes":45384256,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1101,"usage":{"cpu_time_ns":1089786458,"resident_bytes":81772544,"physical_bytes":45400640,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1211,"usage":{"cpu_time_ns":1199707291,"resident_bytes":82214912,"physical_bytes":45843008,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1321,"usage":{"cpu_time_ns":1308623125,"resident_bytes":82264064,"physical_bytes":45892160,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1430,"usage":{"cpu_time_ns":1418420333,"resident_bytes":82427904,"physical_bytes":46056000,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1540,"usage":{"cpu_time_ns":1528490000,"resident_bytes":82771968,"physical_bytes":46400064,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1649,"usage":{"cpu_time_ns":1637371458,"resident_bytes":83001344,"physical_bytes":46629440,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1759,"usage":{"cpu_time_ns":1747650375,"resident_bytes":83263488,"physical_bytes":46891584,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1869,"usage":{"cpu_time_ns":1857203958,"resident_bytes":83525632,"physical_bytes":47153728,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1980,"usage":{"cpu_time_ns":1967118125,"resident_bytes":83771392,"physical_bytes":47399488,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2090,"usage":{"cpu_time_ns":2077281458,"resident_bytes":83951616,"physical_bytes":47579712,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2200,"usage":{"cpu_time_ns":2186295208,"resident_bytes":84131840,"physical_bytes":47759936,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2310,"usage":{"cpu_time_ns":2297883791,"resident_bytes":84312064,"physical_bytes":47940160,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2413,"usage":{"cpu_time_ns":2400806333,"resident_bytes":84475904,"physical_bytes":48104000,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2523,"usage":{"cpu_time_ns":2511110583,"resident_bytes":84672512,"physical_bytes":48300608,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2631,"usage":{"cpu_time_ns":2619294625,"resident_bytes":84885504,"physical_bytes":48513600,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2735,"usage":{"cpu_time_ns":2722820833,"resident_bytes":85082112,"physical_bytes":48710208,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2845,"usage":{"cpu_time_ns":2832876750,"resident_bytes":85295104,"physical_bytes":48923200,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2955,"usage":{"cpu_time_ns":2942863083,"resident_bytes":85508096,"physical_bytes":49136192,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3065,"usage":{"cpu_time_ns":3052332708,"resident_bytes":85721088,"physical_bytes":49349184,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3175,"usage":{"cpu_time_ns":3162369708,"resident_bytes":85934080,"physical_bytes":49562176,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3285,"usage":{"cpu_time_ns":3272410416,"resident_bytes":86147072,"physical_bytes":49775168,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3395,"usage":{"cpu_time_ns":3382444375,"resident_bytes":86360064,"physical_bytes":49988160,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3505,"usage":{"cpu_time_ns":3492013958,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3615,"usage":{"cpu_time_ns":3600157500,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3726,"usage":{"cpu_time_ns":3711857333,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3836,"usage":{"cpu_time_ns":3816674791,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3946,"usage":{"cpu_time_ns":3931704375,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4056,"usage":{"cpu_time_ns":4039640541,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4166,"usage":{"cpu_time_ns":4152580125,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4276,"usage":{"cpu_time_ns":4262739333,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4386,"usage":{"cpu_time_ns":4372437083,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4496,"usage":{"cpu_time_ns":4482362291,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4606,"usage":{"cpu_time_ns":4592292791,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4717,"usage":{"cpu_time_ns":4702056708,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4827,"usage":{"cpu_time_ns":4812138458,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4930,"usage":{"cpu_time_ns":4915473958,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5037,"usage":{"cpu_time_ns":5022994208,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5147,"usage":{"cpu_time_ns":5132479000,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5258,"usage":{"cpu_time_ns":5243562583,"resident_bytes":86556672,"physical_bytes":50184768,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5368,"usage":{"cpu_time_ns":5353654166,"resident_bytes":89341952,"physical_bytes":52085312,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5478,"usage":{"cpu_time_ns":5463690166,"resident_bytes":98025472,"physical_bytes":60768832,"peak_physical_bytes":64815704,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5588,"usage":{"cpu_time_ns":5573756416,"resident_bytes":112656384,"physical_bytes":75203160,"peak_physical_bytes":78283352,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5696,"usage":{"cpu_time_ns":5682128375,"resident_bytes":118374400,"physical_bytes":80675416,"peak_physical_bytes":80675416,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5807,"usage":{"cpu_time_ns":5788370500,"resident_bytes":122044416,"physical_bytes":84263512,"peak_physical_bytes":84263512,"disk_read_bytes":9580544,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5913,"usage":{"cpu_time_ns":5898118125,"resident_bytes":137035776,"physical_bytes":98255472,"peak_physical_bytes":98255472,"disk_read_bytes":10334208,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6020,"usage":{"cpu_time_ns":6004703666,"resident_bytes":137068544,"physical_bytes":98288240,"peak_physical_bytes":98288240,"disk_read_bytes":10334208,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6130,"usage":{"cpu_time_ns":6113680875,"resident_bytes":137199616,"physical_bytes":98419312,"peak_physical_bytes":98419312,"disk_read_bytes":10334208,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6240,"usage":{"cpu_time_ns":6224595958,"resident_bytes":137314304,"physical_bytes":98534000,"peak_physical_bytes":98534000,"disk_read_bytes":10334208,"disk_write_bytes":0}}
+{"event":"test_resource_summary","elapsed_ms":6350,"peak":{"cpu_time_ns":6224595958,"resident_bytes":137314304,"physical_bytes":98534000,"peak_physical_bytes":98534000,"disk_read_bytes":10334208,"disk_write_bytes":0},"error":null}

docs/issue116-nfc/product.output.log

@@ -0,0 +1,13 @@
+
+running 1 test
+test engine::metal::qwen_mtplx::owner::issue116_product::issue116_normalized_token_cache ... ds4: Metal device Apple M5 Max, 128.00 GiB RAM
+ds4: Metal 4 tensor API enabled for Tensor kernels
+ds4: drift-patch flags hc_stable=on norm_unify=on kv_raw_f32=off rope_exp2_log2=off math_safe=off tensor_matmul=on
+ds4: Qwen reference kernels: Powered by MTPLX — https://github.com/youssofal/mtplx
+[qwen4_exp] ngram table: streamed from SSD (file-backed pages, hot cache 1023M, prefetch on) — resident only at >=160G RAM or MTPLX_NGRAM_RESIDENT=1
+[mtplx] verify SDPA head-chunked: q_len=17 heads_per_chunk=1 chunks=24
+{"case":"nfc_cache","legacy_history_tokens":92,"normalized_prompt_tokens":91,"live_old_miss":true,"disk_old_miss":true,"cold_state_exact":true,"warm_cached":93,"disk_cached":93,"warm_disk_state_exact":true}
+ok
+
+test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 554 filtered out; finished in 20.55s
+

docs/issue116-nfc/product.resources.log

@@ -0,0 +1,192 @@
+{"event":"test_supervisor_started","pid":2561,"limit_bytes":120259084288,"start_seconds":60.0,"idle_seconds":120.0,"sample_ms":100}
+{"event":"test_resource_sample","elapsed_ms":0,"usage":{"cpu_time_ns":186333,"resident_bytes":671744,"physical_bytes":196680,"peak_physical_bytes":213064,"disk_read_bytes":0,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":110,"usage":{"cpu_time_ns":103017333,"resident_bytes":52477952,"physical_bytes":31818232,"peak_physical_bytes":31818232,"disk_read_bytes":1228800,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":219,"usage":{"cpu_time_ns":212234208,"resident_bytes":70402048,"physical_bytes":49758736,"peak_physical_bytes":49758736,"disk_read_bytes":1228800,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":329,"usage":{"cpu_time_ns":322309458,"resident_bytes":89341952,"physical_bytes":67895848,"peak_physical_bytes":67895848,"disk_read_bytes":1228800,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":431,"usage":{"cpu_time_ns":423519625,"resident_bytes":97976320,"physical_bytes":76546624,"peak_physical_bytes":76546624,"disk_read_bytes":1228800,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":541,"usage":{"cpu_time_ns":526630625,"resident_bytes":121192448,"physical_bytes":95929016,"peak_physical_bytes":95929016,"disk_read_bytes":1249280,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":641,"usage":{"cpu_time_ns":652712750,"resident_bytes":526925824,"physical_bytes":480576832,"peak_physical_bytes":480576832,"disk_read_bytes":412278784,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":747,"usage":{"cpu_time_ns":837644291,"resident_bytes":1952022528,"physical_bytes":2875607296,"peak_physical_bytes":2875508992,"disk_read_bytes":1856856064,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":857,"usage":{"cpu_time_ns":1006487208,"resident_bytes":3419193344,"physical_bytes":5384539544,"peak_physical_bytes":5384539544,"disk_read_bytes":3332587520,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":964,"usage":{"cpu_time_ns":1139420416,"resident_bytes":3901145088,"physical_bytes":6837915440,"peak_physical_bytes":6837849904,"disk_read_bytes":4718886912,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1072,"usage":{"cpu_time_ns":1269260041,"resident_bytes":4378591232,"physical_bytes":8286441624,"peak_physical_bytes":8286376088,"disk_read_bytes":6198706176,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1182,"usage":{"cpu_time_ns":1400213875,"resident_bytes":4875862016,"physical_bytes":9750680088,"peak_physical_bytes":9750598168,"disk_read_bytes":7662292992,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1292,"usage":{"cpu_time_ns":1526673500,"resident_bytes":5384437760,"physical_bytes":11205120944,"peak_physical_bytes":11205088176,"disk_read_bytes":9108443136,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1403,"usage":{"cpu_time_ns":1660769791,"resident_bytes":5898354688,"physical_bytes":12683220296,"peak_physical_bytes":12682909000,"disk_read_bytes":10606104576,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1511,"usage":{"cpu_time_ns":1787359708,"resident_bytes":6390054912,"physical_bytes":13852759776,"peak_physical_bytes":13852530400,"disk_read_bytes":12082991104,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1615,"usage":{"cpu_time_ns":1911929750,"resident_bytes":6825705472,"physical_bytes":14716409928,"peak_physical_bytes":14716409928,"disk_read_bytes":13454528512,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1725,"usage":{"cpu_time_ns":2049463750,"resident_bytes":7305281536,"physical_bytes":16477657544,"peak_physical_bytes":16477444552,"disk_read_bytes":14941081600,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1831,"usage":{"cpu_time_ns":2203858750,"resident_bytes":8624406528,"physical_bytes":18378530208,"peak_physical_bytes":18378497440,"disk_read_bytes":16307822592,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":1941,"usage":{"cpu_time_ns":2348704375,"resident_bytes":9077276672,"physical_bytes":19978673856,"peak_physical_bytes":19978477248,"disk_read_bytes":17837506560,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2048,"usage":{"cpu_time_ns":2487334166,"resident_bytes":9728589824,"physical_bytes":21422088424,"peak_physical_bytes":21421990120,"disk_read_bytes":19281575936,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2158,"usage":{"cpu_time_ns":2623040041,"resident_bytes":10170204160,"physical_bytes":22755320352,"peak_physical_bytes":22755320352,"disk_read_bytes":20734246912,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2264,"usage":{"cpu_time_ns":2747426083,"resident_bytes":10208067584,"physical_bytes":23762526848,"peak_physical_bytes":23762526848,"disk_read_bytes":22154592256,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2368,"usage":{"cpu_time_ns":2870511958,"resident_bytes":10324836352,"physical_bytes":24848737040,"peak_physical_bytes":24848737040,"disk_read_bytes":23516073984,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2478,"usage":{"cpu_time_ns":3004248500,"resident_bytes":10826432512,"physical_bytes":26320168080,"peak_physical_bytes":26320168080,"disk_read_bytes":25026793472,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2588,"usage":{"cpu_time_ns":3140130208,"resident_bytes":11346706432,"physical_bytes":27786094144,"peak_physical_bytes":27786094144,"disk_read_bytes":26502942720,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2697,"usage":{"cpu_time_ns":3274878833,"resident_bytes":11872894976,"physical_bytes":29282002880,"peak_physical_bytes":29282002880,"disk_read_bytes":27931267072,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2806,"usage":{"cpu_time_ns":3413523375,"resident_bytes":12383354880,"physical_bytes":30763821400,"peak_physical_bytes":30763821400,"disk_read_bytes":29408137216,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":2914,"usage":{"cpu_time_ns":3551261000,"resident_bytes":13285687296,"physical_bytes":32634088440,"peak_physical_bytes":32634088440,"disk_read_bytes":30853943296,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3022,"usage":{"cpu_time_ns":3679326750,"resident_bytes":13402750976,"physical_bytes":33698770080,"peak_physical_bytes":33698770080,"disk_read_bytes":32302587904,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3132,"usage":{"cpu_time_ns":3826058083,"resident_bytes":13920256000,"physical_bytes":35290721800,"peak_physical_bytes":35290721800,"disk_read_bytes":33783980032,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3241,"usage":{"cpu_time_ns":3962733166,"resident_bytes":14432157696,"physical_bytes":36665552848,"peak_physical_bytes":36665552848,"disk_read_bytes":35204882432,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3351,"usage":{"cpu_time_ns":4396706791,"resident_bytes":16343465984,"physical_bytes":40218506608,"peak_physical_bytes":40218260848,"disk_read_bytes":35756236800,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3461,"usage":{"cpu_time_ns":4672814583,"resident_bytes":17017307136,"physical_bytes":42699650992,"peak_physical_bytes":42699421616,"disk_read_bytes":36833828864,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3564,"usage":{"cpu_time_ns":4829710208,"resident_bytes":17539874816,"physical_bytes":43628542256,"peak_physical_bytes":43628509488,"disk_read_bytes":38207451136,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3674,"usage":{"cpu_time_ns":4961475208,"resident_bytes":17826889728,"physical_bytes":43915672072,"peak_physical_bytes":43915672072,"disk_read_bytes":39665913856,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3781,"usage":{"cpu_time_ns":5174336958,"resident_bytes":18564235264,"physical_bytes":46675147896,"peak_physical_bytes":46675115128,"disk_read_bytes":41091174400,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3881,"usage":{"cpu_time_ns":5330718708,"resident_bytes":19718799360,"physical_bytes":48693493696,"peak_physical_bytes":48693493696,"disk_read_bytes":42437709824,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":3991,"usage":{"cpu_time_ns":5464810583,"resident_bytes":19797393408,"physical_bytes":49743118344,"peak_physical_bytes":49743118344,"disk_read_bytes":43886546944,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4092,"usage":{"cpu_time_ns":5593206833,"resident_bytes":19887325184,"physical_bytes":50800590952,"peak_physical_bytes":50800590952,"disk_read_bytes":45231984640,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4197,"usage":{"cpu_time_ns":5743990458,"resident_bytes":20281475072,"physical_bytes":52142457320,"peak_physical_bytes":52142457320,"disk_read_bytes":46659510272,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4297,"usage":{"cpu_time_ns":5920055000,"resident_bytes":20966424576,"physical_bytes":53795177368,"peak_physical_bytes":53795177368,"disk_read_bytes":47996067840,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4406,"usage":{"cpu_time_ns":6070675583,"resident_bytes":21012398080,"physical_bytes":54810592248,"peak_physical_bytes":54810592248,"disk_read_bytes":49459863552,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4512,"usage":{"cpu_time_ns":6209962875,"resident_bytes":21120204800,"physical_bytes":55887823984,"peak_physical_bytes":55887823984,"disk_read_bytes":50853036032,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4619,"usage":{"cpu_time_ns":6401150250,"resident_bytes":21619228672,"physical_bytes":57332663768,"peak_physical_bytes":57332663768,"disk_read_bytes":52247789568,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4720,"usage":{"cpu_time_ns":6586663583,"resident_bytes":22090285056,"physical_bytes":58773325656,"peak_physical_bytes":58773325656,"disk_read_bytes":53572993024,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4828,"usage":{"cpu_time_ns":6758714666,"resident_bytes":22603563008,"physical_bytes":60256225520,"peak_physical_bytes":60256225520,"disk_read_bytes":54990299136,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":4929,"usage":{"cpu_time_ns":6938745625,"resident_bytes":22875537408,"physical_bytes":61461711520,"peak_physical_bytes":61461498528,"disk_read_bytes":56334114816,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5029,"usage":{"cpu_time_ns":7130200291,"resident_bytes":22654025728,"physical_bytes":62426696616,"peak_physical_bytes":62426614696,"disk_read_bytes":57499234304,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5140,"usage":{"cpu_time_ns":7411766541,"resident_bytes":23621894144,"physical_bytes":64369479456,"peak_physical_bytes":64369479456,"disk_read_bytes":58965127168,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5240,"usage":{"cpu_time_ns":7614785500,"resident_bytes":23271899136,"physical_bytes":65608634480,"peak_physical_bytes":65608634480,"disk_read_bytes":60112482304,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5342,"usage":{"cpu_time_ns":7836605458,"resident_bytes":23123935232,"physical_bytes":67139195352,"peak_physical_bytes":67139195352,"disk_read_bytes":61189525504,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5442,"usage":{"cpu_time_ns":8023652833,"resident_bytes":23552032768,"physical_bytes":68036498216,"peak_physical_bytes":68036498216,"disk_read_bytes":62540705792,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5551,"usage":{"cpu_time_ns":8325105416,"resident_bytes":23936483328,"physical_bytes":70058824968,"peak_physical_bytes":70058824968,"disk_read_bytes":63845953536,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5653,"usage":{"cpu_time_ns":8474620833,"resident_bytes":24227954688,"physical_bytes":70726899384,"peak_physical_bytes":70726899384,"disk_read_bytes":65114632192,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5763,"usage":{"cpu_time_ns":8762165875,"resident_bytes":23464361984,"physical_bytes":71909267200,"peak_physical_bytes":71909267200,"disk_read_bytes":66529603584,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5864,"usage":{"cpu_time_ns":8961558875,"resident_bytes":23842045952,"physical_bytes":73334839616,"peak_physical_bytes":73334790464,"disk_read_bytes":67817291776,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":5974,"usage":{"cpu_time_ns":9235729083,"resident_bytes":24448860160,"physical_bytes":74817169000,"peak_physical_bytes":74817169000,"disk_read_bytes":68354199552,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6078,"usage":{"cpu_time_ns":9358443416,"resident_bytes":24384094208,"physical_bytes":75087800104,"peak_physical_bytes":75087800104,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6181,"usage":{"cpu_time_ns":9462210958,"resident_bytes":24384110592,"physical_bytes":75097958184,"peak_physical_bytes":75097974568,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6289,"usage":{"cpu_time_ns":9577317041,"resident_bytes":24371544064,"physical_bytes":75103577896,"peak_physical_bytes":75103577896,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6399,"usage":{"cpu_time_ns":9695786041,"resident_bytes":24371560448,"physical_bytes":75126351656,"peak_physical_bytes":75126368040,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6509,"usage":{"cpu_time_ns":9808355833,"resident_bytes":24371560448,"physical_bytes":75131447080,"peak_physical_bytes":75131447080,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6620,"usage":{"cpu_time_ns":9920527416,"resident_bytes":24371625984,"physical_bytes":75135199040,"peak_physical_bytes":75135199040,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6730,"usage":{"cpu_time_ns":10033224458,"resident_bytes":24371707904,"physical_bytes":75137197888,"peak_physical_bytes":75137214272,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6839,"usage":{"cpu_time_ns":10145385958,"resident_bytes":24371707904,"physical_bytes":75142244160,"peak_physical_bytes":75142293312,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":6947,"usage":{"cpu_time_ns":10268287916,"resident_bytes":24367611904,"physical_bytes":75161265984,"peak_physical_bytes":75161265984,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7051,"usage":{"cpu_time_ns":10364680833,"resident_bytes":24367841280,"physical_bytes":75176666944,"peak_physical_bytes":75176666944,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7161,"usage":{"cpu_time_ns":10506995500,"resident_bytes":24369364992,"physical_bytes":75239778112,"peak_physical_bytes":75239778112,"disk_read_bytes":68354936832,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7271,"usage":{"cpu_time_ns":10706446291,"resident_bytes":24215519232,"physical_bytes":75547535480,"peak_physical_bytes":75547535480,"disk_read_bytes":69422927872,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7381,"usage":{"cpu_time_ns":10849571583,"resident_bytes":24152588288,"physical_bytes":75889977464,"peak_physical_bytes":75889977464,"disk_read_bytes":70012588032,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7483,"usage":{"cpu_time_ns":10938977625,"resident_bytes":24154243072,"physical_bytes":76307409016,"peak_physical_bytes":76307409016,"disk_read_bytes":70012604416,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7593,"usage":{"cpu_time_ns":11048945333,"resident_bytes":24154783744,"physical_bytes":76329019512,"peak_physical_bytes":76329019512,"disk_read_bytes":70012620800,"disk_write_bytes":0}}
+{"event":"test_resource_sample","elapsed_ms":7697,"usage":{"cpu_time_ns":11160402125,"resident_bytes":24169168896,"physical_bytes":76431075520,"peak_physical_bytes":76431075520,"disk_read_bytes":70012637184,"disk_write_bytes":8388608}}
+{"event":"test_resource_sample","elapsed_ms":7804,"usage":{"cpu_time_ns":11257911916,"resident_bytes":24259362816,"physical_bytes":76520892728,"peak_physical_bytes":76520892728,"disk_read_bytes":70012641280,"disk_write_bytes":92274688}}
+{"event":"test_resource_sample","elapsed_ms":7915,"usage":{"cpu_time_ns":11360467625,"resident_bytes":24360927232,"physical_bytes":76544633216,"peak_physical_bytes":76544633216,"disk_read_bytes":70014349312,"disk_write_bytes":192937984}}
+{"event":"test_resource_sample","elapsed_ms":8025,"usage":{"cpu_time_ns":11474780291,"resident_bytes":24425283584,"physical_bytes":76657946280,"peak_physical_bytes":76657946280,"disk_read_bytes":70014365696,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":8135,"usage":{"cpu_time_ns":11523067833,"resident_bytes":24425152512,"physical_bytes":76677312168,"peak_physical_bytes":76677312168,"disk_read_bytes":70014365696,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":8245,"usage":{"cpu_time_ns":11580837791,"resident_bytes":24425218048,"physical_bytes":76690009768,"peak_physical_bytes":76690009768,"disk_read_bytes":70014365696,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":8355,"usage":{"cpu_time_ns":11696350541,"resident_bytes":24440832000,"physical_bytes":76818198376,"peak_physical_bytes":76818198376,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":8465,"usage":{"cpu_time_ns":11807333166,"resident_bytes":24442306560,"physical_bytes":76832583552,"peak_physical_bytes":76832583552,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":8576,"usage":{"cpu_time_ns":11925251458,"resident_bytes":24442765312,"physical_bytes":76946255768,"peak_physical_bytes":76946255768,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":8681,"usage":{"cpu_time_ns":12027777666,"resident_bytes":24443256832,"physical_bytes":76946272176,"peak_physical_bytes":76946272176,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":8791,"usage":{"cpu_time_ns":12132194708,"resident_bytes":24443535360,"physical_bytes":76946272176,"peak_physical_bytes":76946272176,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":8897,"usage":{"cpu_time_ns":12235883416,"resident_bytes":24443781120,"physical_bytes":76945354672,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9007,"usage":{"cpu_time_ns":12341485458,"resident_bytes":24444256256,"physical_bytes":76945387464,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9118,"usage":{"cpu_time_ns":12448669125,"resident_bytes":24444551168,"physical_bytes":76945403848,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9228,"usage":{"cpu_time_ns":12554719666,"resident_bytes":24444960768,"physical_bytes":76945436640,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9338,"usage":{"cpu_time_ns":12661662333,"resident_bytes":24445157376,"physical_bytes":76945436640,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9448,"usage":{"cpu_time_ns":12769259500,"resident_bytes":24445370368,"physical_bytes":76945453024,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9558,"usage":{"cpu_time_ns":12865381666,"resident_bytes":24455954432,"physical_bytes":76945387512,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9668,"usage":{"cpu_time_ns":12964266666,"resident_bytes":24468799488,"physical_bytes":76945387512,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9778,"usage":{"cpu_time_ns":13070865333,"resident_bytes":24484872192,"physical_bytes":76945403896,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9886,"usage":{"cpu_time_ns":13172361625,"resident_bytes":24497684480,"physical_bytes":76942127096,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":9996,"usage":{"cpu_time_ns":13279555541,"resident_bytes":24510545920,"physical_bytes":76942143480,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":10106,"usage":{"cpu_time_ns":13387692083,"resident_bytes":24526602240,"physical_bytes":76942143480,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":10216,"usage":{"cpu_time_ns":13496277208,"resident_bytes":24539463680,"physical_bytes":76942192680,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":10327,"usage":{"cpu_time_ns":13604604500,"resident_bytes":24555536384,"physical_bytes":76942209064,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":10437,"usage":{"cpu_time_ns":13714110750,"resident_bytes":24564842496,"physical_bytes":76945010728,"peak_physical_bytes":76946288560,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":10547,"usage":{"cpu_time_ns":13824159541,"resident_bytes":24568561664,"physical_bytes":76947009576,"peak_physical_bytes":76947009576,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":10657,"usage":{"cpu_time_ns":13934210666,"resident_bytes":24569954304,"physical_bytes":76948336680,"peak_physical_bytes":76948336680,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":10764,"usage":{"cpu_time_ns":14041072666,"resident_bytes":24571641856,"physical_bytes":76949991464,"peak_physical_bytes":76949991464,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":10874,"usage":{"cpu_time_ns":14150483750,"resident_bytes":24586600448,"physical_bytes":76951597096,"peak_physical_bytes":76951597096,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":10981,"usage":{"cpu_time_ns":14257617458,"resident_bytes":24622219264,"physical_bytes":76952711208,"peak_physical_bytes":76952711208,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":11091,"usage":{"cpu_time_ns":14367651333,"resident_bytes":24654610432,"physical_bytes":76953743400,"peak_physical_bytes":76953743400,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":11197,"usage":{"cpu_time_ns":14474129916,"resident_bytes":24690458624,"physical_bytes":76954972200,"peak_physical_bytes":76954972200,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":11307,"usage":{"cpu_time_ns":14594827041,"resident_bytes":24693489664,"physical_bytes":76961820712,"peak_physical_bytes":76961820712,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":11418,"usage":{"cpu_time_ns":14727955500,"resident_bytes":24693374976,"physical_bytes":76964180008,"peak_physical_bytes":76964180008,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":11527,"usage":{"cpu_time_ns":14859490625,"resident_bytes":24693374976,"physical_bytes":76964180008,"peak_physical_bytes":76964180008,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":11632,"usage":{"cpu_time_ns":14995528833,"resident_bytes":24693374976,"physical_bytes":76964180008,"peak_physical_bytes":76964180008,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":11742,"usage":{"cpu_time_ns":15119880500,"resident_bytes":24693391360,"physical_bytes":76964769832,"peak_physical_bytes":76964769832,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":11852,"usage":{"cpu_time_ns":15245852250,"resident_bytes":24693374976,"physical_bytes":76964753448,"peak_physical_bytes":76964786216,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":11962,"usage":{"cpu_time_ns":15355517416,"resident_bytes":24693440512,"physical_bytes":76965048360,"peak_physical_bytes":76965048360,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":12064,"usage":{"cpu_time_ns":15453650208,"resident_bytes":24693653504,"physical_bytes":76965048360,"peak_physical_bytes":76965048360,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":12174,"usage":{"cpu_time_ns":15562757875,"resident_bytes":24693800960,"physical_bytes":76965163048,"peak_physical_bytes":76965163048,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":12284,"usage":{"cpu_time_ns":15669178916,"resident_bytes":24693800960,"physical_bytes":76965163048,"peak_physical_bytes":76965163048,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":12394,"usage":{"cpu_time_ns":15777314875,"resident_bytes":24693833728,"physical_bytes":76965163048,"peak_physical_bytes":76965163048,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":12504,"usage":{"cpu_time_ns":15884031291,"resident_bytes":24694079488,"physical_bytes":76965163048,"peak_physical_bytes":76965163048,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":12614,"usage":{"cpu_time_ns":15989999291,"resident_bytes":24694439936,"physical_bytes":76965163048,"peak_physical_bytes":76965163048,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":12724,"usage":{"cpu_time_ns":16096967000,"resident_bytes":24694816768,"physical_bytes":76965163048,"peak_physical_bytes":76965163048,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":12825,"usage":{"cpu_time_ns":16194386041,"resident_bytes":24695144448,"physical_bytes":76962820160,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":12933,"usage":{"cpu_time_ns":16297925000,"resident_bytes":24695701504,"physical_bytes":76962705472,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":13044,"usage":{"cpu_time_ns":16406089458,"resident_bytes":24695701504,"physical_bytes":76962705472,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":13147,"usage":{"cpu_time_ns":16490333375,"resident_bytes":24695701504,"physical_bytes":76962705472,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":13257,"usage":{"cpu_time_ns":16590709208,"resident_bytes":24695701504,"physical_bytes":76962705472,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":13363,"usage":{"cpu_time_ns":16694347291,"resident_bytes":24702009344,"physical_bytes":76962721856,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":13473,"usage":{"cpu_time_ns":16801952125,"resident_bytes":24717967360,"physical_bytes":76962738264,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":13581,"usage":{"cpu_time_ns":16906488458,"resident_bytes":24730746880,"physical_bytes":76962754648,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":13691,"usage":{"cpu_time_ns":17014983958,"resident_bytes":24746754048,"physical_bytes":76962771032,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":13801,"usage":{"cpu_time_ns":17125307625,"resident_bytes":24753192960,"physical_bytes":76962787416,"peak_physical_bytes":76965179456,"disk_read_bytes":70027587584,"disk_write_bytes":235057152}}
+{"event":"test_resource_sample","elapsed_ms":13911,"usage":{"cpu_time_ns":17251575750,"resident_bytes":24753192960,"physical_bytes":76961886296,"peak_physical_bytes":76965179456,"disk_read_bytes":70027599872,"disk_write_bytes":386052096}}
+{"event":"test_resource_sample","elapsed_ms":14013,"usage":{"cpu_time_ns":17353567416,"resident_bytes":24753258496,"physical_bytes":76962984024,"peak_physical_bytes":76965179456,"disk_read_bytes":70027603968,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":14114,"usage":{"cpu_time_ns":17467517041,"resident_bytes":24758550528,"physical_bytes":76981760184,"peak_physical_bytes":76981760184,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":14224,"usage":{"cpu_time_ns":17591384625,"resident_bytes":24758550528,"physical_bytes":76997554360,"peak_physical_bytes":76997554360,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":14334,"usage":{"cpu_time_ns":17726413666,"resident_bytes":24758550528,"physical_bytes":77004026040,"peak_physical_bytes":77004026040,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":14444,"usage":{"cpu_time_ns":17854729583,"resident_bytes":24758550528,"physical_bytes":77008892088,"peak_physical_bytes":77008892088,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":14555,"usage":{"cpu_time_ns":17995622416,"resident_bytes":24758550528,"physical_bytes":77014331576,"peak_physical_bytes":77014331576,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":14664,"usage":{"cpu_time_ns":18139482791,"resident_bytes":24758075392,"physical_bytes":77022736568,"peak_physical_bytes":77022736568,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":14774,"usage":{"cpu_time_ns":18254686750,"resident_bytes":24758272000,"physical_bytes":77026652368,"peak_physical_bytes":77026652368,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":14884,"usage":{"cpu_time_ns":18361526458,"resident_bytes":24758599680,"physical_bytes":77026701544,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":14994,"usage":{"cpu_time_ns":18468610458,"resident_bytes":24759025664,"physical_bytes":77026701544,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":15104,"usage":{"cpu_time_ns":18574070541,"resident_bytes":24759664640,"physical_bytes":77026701544,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":15215,"usage":{"cpu_time_ns":18681760375,"resident_bytes":24760107008,"physical_bytes":77026701544,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":15325,"usage":{"cpu_time_ns":18787468416,"resident_bytes":24760991744,"physical_bytes":77026701544,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":15435,"usage":{"cpu_time_ns":18894399625,"resident_bytes":24761532416,"physical_bytes":77026701544,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":15545,"usage":{"cpu_time_ns":19001490708,"resident_bytes":24761925632,"physical_bytes":77026701544,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":15655,"usage":{"cpu_time_ns":19107999791,"resident_bytes":24762384384,"physical_bytes":77026570472,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":15765,"usage":{"cpu_time_ns":19213392875,"resident_bytes":24762892288,"physical_bytes":77026439400,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":15870,"usage":{"cpu_time_ns":19321485625,"resident_bytes":24762892288,"physical_bytes":77026439400,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":15980,"usage":{"cpu_time_ns":19432533958,"resident_bytes":24762892288,"physical_bytes":77026439400,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":16089,"usage":{"cpu_time_ns":19540743708,"resident_bytes":24762892288,"physical_bytes":77026373864,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":16197,"usage":{"cpu_time_ns":19651683500,"resident_bytes":24762892288,"physical_bytes":77023310056,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":16307,"usage":{"cpu_time_ns":19762796083,"resident_bytes":24762892288,"physical_bytes":77023310056,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":16417,"usage":{"cpu_time_ns":19871382833,"resident_bytes":24762892288,"physical_bytes":77023310056,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":16528,"usage":{"cpu_time_ns":19986119500,"resident_bytes":24762892288,"physical_bytes":77023310056,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":16631,"usage":{"cpu_time_ns":20090779958,"resident_bytes":24744869888,"physical_bytes":76984791272,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":16741,"usage":{"cpu_time_ns":20200831958,"resident_bytes":24748064768,"physical_bytes":76987920616,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":16851,"usage":{"cpu_time_ns":20310932333,"resident_bytes":24748326912,"physical_bytes":76987920616,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":16961,"usage":{"cpu_time_ns":20418826583,"resident_bytes":24748490752,"physical_bytes":76853145832,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":17064,"usage":{"cpu_time_ns":20521872000,"resident_bytes":24748965888,"physical_bytes":76853145832,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":17174,"usage":{"cpu_time_ns":20631928958,"resident_bytes":24748965888,"physical_bytes":76853145832,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":17284,"usage":{"cpu_time_ns":20741983458,"resident_bytes":24748965888,"physical_bytes":76853145832,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":17394,"usage":{"cpu_time_ns":20854965125,"resident_bytes":24777293824,"physical_bytes":76971569384,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":17504,"usage":{"cpu_time_ns":20970847375,"resident_bytes":24777555968,"physical_bytes":76980859112,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":17615,"usage":{"cpu_time_ns":21090651250,"resident_bytes":24777523200,"physical_bytes":76992196840,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":17725,"usage":{"cpu_time_ns":21225478000,"resident_bytes":24777523200,"physical_bytes":76992786664,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":17831,"usage":{"cpu_time_ns":21351726666,"resident_bytes":24777523200,"physical_bytes":76992901352,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":17933,"usage":{"cpu_time_ns":21477807750,"resident_bytes":24777523200,"physical_bytes":76993130728,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":18043,"usage":{"cpu_time_ns":21621495041,"resident_bytes":24777523200,"physical_bytes":76993589480,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":18154,"usage":{"cpu_time_ns":21750403416,"resident_bytes":24777523200,"physical_bytes":76993917160,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":18264,"usage":{"cpu_time_ns":21866906625,"resident_bytes":24777523200,"physical_bytes":76993982696,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":18374,"usage":{"cpu_time_ns":21975773583,"resident_bytes":24777965568,"physical_bytes":76993982696,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":18484,"usage":{"cpu_time_ns":22083089875,"resident_bytes":24778113024,"physical_bytes":76993982696,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":18594,"usage":{"cpu_time_ns":22190155583,"resident_bytes":24778407936,"physical_bytes":76993982696,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":18704,"usage":{"cpu_time_ns":22296297000,"resident_bytes":24778440704,"physical_bytes":76993982696,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":18814,"usage":{"cpu_time_ns":22402388333,"resident_bytes":24778457088,"physical_bytes":76993982696,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":18924,"usage":{"cpu_time_ns":22508419958,"resident_bytes":24778866688,"physical_bytes":76993982696,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":19034,"usage":{"cpu_time_ns":22615267750,"resident_bytes":24779112448,"physical_bytes":76993982696,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":19144,"usage":{"cpu_time_ns":22722949541,"resident_bytes":24792006656,"physical_bytes":76993851624,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":19254,"usage":{"cpu_time_ns":22831700666,"resident_bytes":24792121344,"physical_bytes":76993720552,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":19365,"usage":{"cpu_time_ns":22942598958,"resident_bytes":24792334336,"physical_bytes":76993720552,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":19475,"usage":{"cpu_time_ns":23054300083,"resident_bytes":24792481792,"physical_bytes":76993720552,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":19583,"usage":{"cpu_time_ns":23164380708,"resident_bytes":24792530944,"physical_bytes":76991492328,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":19685,"usage":{"cpu_time_ns":23267293916,"resident_bytes":24792563712,"physical_bytes":76990591208,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":19795,"usage":{"cpu_time_ns":23378134291,"resident_bytes":24792596480,"physical_bytes":76990591208,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":19905,"usage":{"cpu_time_ns":23488377416,"resident_bytes":24792596480,"physical_bytes":76990591208,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":20014,"usage":{"cpu_time_ns":23599816416,"resident_bytes":24792596480,"physical_bytes":76990591208,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":20124,"usage":{"cpu_time_ns":23701991125,"resident_bytes":24251940864,"physical_bytes":75714960472,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":20234,"usage":{"cpu_time_ns":23806395291,"resident_bytes":18448171008,"physical_bytes":62144022288,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":20344,"usage":{"cpu_time_ns":23920864583,"resident_bytes":8783462400,"physical_bytes":50393867312,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_sample","elapsed_ms":20454,"usage":{"cpu_time_ns":24032068000,"resident_bytes":156762112,"physical_bytes":29417162384,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072}}
+{"event":"test_resource_summary","elapsed_ms":20559,"peak":{"cpu_time_ns":24032068000,"resident_bytes":24792596480,"physical_bytes":77026701544,"peak_physical_bytes":77026701544,"disk_read_bytes":70029180928,"disk_write_bytes":470147072},"error":null}

docs/issue116-nfc/requests.json

@@ -0,0 +1 @@
+[{"legacy_history_ids": [248045, 846, 198, 52033, 64, 264, 52033, 220, 52033, 220, 12256, 100, 52033, 65, 83117, 226, 222, 157, 227, 94, 157, 228, 101, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271, 34, 5468, 52033, 220, 46701, 254, 13, 248046, 198], "effort": "direct", "system": "", "messages": [{"role": "user", "content": " ́a á ́ १́b 각\n "}, {"role": "assistant", "content": "Café ①.", "reasoning_content": ""}, {"role": "user", "content": "Continue é and 가.\n   x"}], "ids": [248045, 846, 198, 52033, 64, 18135, 220, 52033, 220, 12256, 100, 52033, 65, 155180, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271, 34, 2492, 933, 220, 46701, 254, 13, 248046, 198, 248045, 846, 198, 22791, 3825, 321, 34335, 13, 198, 256, 830, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271], "legacy_ids": [248045, 846, 198, 52033, 64, 264, 52033, 220, 52033, 220, 12256, 100, 52033, 65, 83117, 226, 222, 157, 227, 94, 157, 228, 101, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271, 34, 5468, 52033, 220, 46701, 254, 13, 248046, 198, 248045, 846, 198, 22791, 378, 52033, 321, 83117, 226, 222, 157, 227, 94, 13, 198, 256, 830, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271], "decoded": "<|im_start|>user\ńa á ́ १́b 각<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nCafé ①.<|im_end|>\n<|im_start|>user\nContinue é and 가.\n   x<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"}, {"legacy_history_ids": [248045, 8678, 198, 34, 5468, 52033, 83117, 226, 222, 157, 227, 94, 157, 228, 101, 13, 248046, 198, 248045, 846, 198, 52033, 64, 264, 52033, 220, 52033, 220, 12256, 100, 52033, 65, 83117, 226, 222, 157, 227, 94, 157, 228, 101, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271, 34, 5468, 52033, 220, 46701, 254, 13, 248046, 198], "effort": "direct", "system": "Café 각.", "messages": [{"role": "user", "content": " ́a á ́ १́b 각\n "}, {"role": "assistant", "content": "Café ①.", "reasoning_content": ""}, {"role": "user", "content": "Continue é and 가.\n   x"}], "ids": [248045, 8678, 198, 34, 2492, 933, 155180, 13, 248046, 198, 248045, 846, 198, 52033, 64, 18135, 220, 52033, 220, 12256, 100, 52033, 65, 155180, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271, 34, 2492, 933, 220, 46701, 254, 13, 248046, 198, 248045, 846, 198, 22791, 3825, 321, 34335, 13, 198, 256, 830, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271], "legacy_ids": [248045, 8678, 198, 34, 5468, 52033, 83117, 226, 222, 157, 227, 94, 157, 228, 101, 13, 248046, 198, 248045, 846, 198, 52033, 64, 264, 52033, 220, 52033, 220, 12256, 100, 52033, 65, 83117, 226, 222, 157, 227, 94, 157, 228, 101, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271, 34, 5468, 52033, 220, 46701, 254, 13, 248046, 198, 248045, 846, 198, 22791, 378, 52033, 321, 83117, 226, 222, 157, 227, 94, 13, 198, 256, 830, 248046, 198, 248045, 74455, 198, 248068, 271, 248069, 271], "decoded": "<|im_start|>system\nCafé 각.<|im_end|>\n<|im_start|>user\ńa á ́ १́b 각<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nCafé ①.<|im_end|>\n<|im_start|>user\nContinue é and 가.\n   x<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"}, {"legacy_history_ids": [248045, 8678, 198, 24342, 286, 4879, 369, 716, 310, 3238, 13, 13262, 678, 7047, 9522, 321, 10419, 11, 6992, 5774, 310, 279, 16198, 1973, 24366, 24150, 362, 13, 248046, 198, 248045, 846, 198, 52033, 64, 264, 52033, 220, 52033, 220, 12256, 100, 52033, 65, 83117, 226, 222, 157, 227, 94, 157, 228, 101, 248046, 198, 248045, 74455, 198, 248068, 198, 64, 52033, 136, 96, 3272, 13, 198, 248069, 271, 34, 5468, 52033, 220, 46701, 254, 13, 248046, 198], "effort": "low", "system": "", "messages": [{"role": "user", "content": " ́a á ́ १́b 각\n "}, {"role": "assistant", "content": "Café ①.", "reasoning_content": "ạ́ thought."}, {"role": "user", "content": "Continue é and 가.\n   x"}], "ids": [248045, 8678, 198, 24342, 286, 4879, 369, 716, 310, 3238, 13, 13262, 678, 7047, 9522, 321, 10419, 11, 6992, 5774, 310, 279, 16198, 1973, 24366, 24150, 362, 13, 248046, 198, 248045, 846, 198, 52033, 64, 18135, 220, 52033, 220, 12256, 100, 52033, 65, 155180, 248046, 198, 248045, 74455, 198, 248068, 198, 19614, 52033, 3272, 13, 198, 248069, 271, 34, 2492, 933, 220, 46701, 254, 13, 248046, 198, 248045, 846, 198, 22791, 3825, 321, 34335, 13, 198, 256, 830, 248046, 198, 248045, 74455, 198, 248068, 198], "legacy_ids": [248045, 8678, 198, 24342, 286, 4879, 369, 716, 310, 3238, 13, 13262, 678, 7047, 9522, 321, 10419, 11, 6992, 5774, 310, 279, 16198, 1973, 24366, 24150, 362, 13, 248046, 198, 248045, 846, 198, 52033, 64, 264, 52033, 220, 52033, 220, 12256, 100, 52033, 65, 83117, 226, 222, 157, 227, 94, 157, 228, 101, 248046, 198, 248045, 74455, 198, 248068, 198, 64, 52033, 136, 96, 3272, 13, 198, 248069, 271, 34, 5468, 52033, 220, 46701, 254, 13, 248046, 198, 248045, 846, 198, 22791, 378, 52033Line truncated

docs/issue116-nfc/scalar-nfc.json

@@ -0,0 +1 @@
+{"scalars": 1112064, "bytes": 11054695, "sha256": "1905b5f60f629850aa35dfc8ccb11fa6ae4d1fb0c79efd6774d36ca8e3e64eab", "tokenizers": "0.22.2"}

docs/issue116-nfc/tests.log

@@ -0,0 +1,599 @@
+   Compiling ds4-server v0.1.0 (/Users/gb/Projects/DS4Server)
+    Finished `test` profile [unoptimized + debuginfo] target(s) in 5.80s
+warning: the following packages contain code that will be rejected by a future version of Rust: block v0.1.6
+note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
+     Running unittests src/main.rs (target/debug/deps/ds4_server-d15236c42a8179f4)
+
+running 555 tests
+test agent::tests::anchored_edits_require_unique_head_and_tail ... ok
+test agent::tests::ai_risk_response_requires_structured_risk_and_reason ... ok
+test a2ui::tests::streaming_parser_only_accepts_complete_a2ui_jsonl_lines ... ok
+test agent::tests::ai_shell_risk_assessment_receives_the_dev_brain_root ... ok
+test a2ui::tests::embedded_catalog_and_validator_cover_the_same_components ... ok
+test agent::tests::dev_brain_paths_are_trusted_by_shell_risk_assessment ... ok
+test a2ui::tests::basic_formatters_follow_catalog_examples ... ok
+test a2ui::tests::catalog_validation_rejects_invalid_variants_and_function_arguments ... ok
+test a2ui::tests::catalog_accepts_pie_donut_and_heatmap_charts ... ok
+test a2ui::tests::lifecycle_updates_incrementally_and_json_pointer_upserts ... ok
+test a2ui::tests::dismissal_boundaries_replay_history_and_a_fresh_active_epoch ... ok
+test a2ui::tests::latest_created_surface_is_the_active_surface ... ok
+test a2ui::tests::catalog_validation_and_actions_use_current_local_data ... ok
+test agent::tests::ralph_loop_stops_on_cap_blocker_failure_and_cancellation ... ok
+test agent::tests::present_svg_parsers_recover_exact_source_for_both_model_transports ... ok
+test a2ui::tests::all_basics_compose_into_one_surface ... ok
+test a2ui::tests::v1_action_and_function_round_trips_follow_current_schema ... ok
+test agent::tests::ralph_report_semantics_and_handoff_bound_are_enforced ... ok
+test agent::tests::risky_shell_commands_require_one_time_approval ... ok
+test agent::tests::model_adapters_preserve_primitives_and_report_recoverable_syntax_errors ... ok
+test agent::tests::scripted_ralph_backend_gets_fresh_prompts_and_only_validated_handoffs ... ok
+test agent::tests::generated_schemas_match_the_executable_tool_contracts ... ok
+test agent::tests::project_boundary_rejects_parent_paths ... ok
+test agent::tests::repeated_call_advisories_use_canonical_arguments_and_reset_per_user_turn ... ok
+test a2ui::export::tests::static_svg_renderer_covers_the_complete_catalog ... ok
+test agent::tests::ralph_report_schemas_are_child_only_and_parse_for_both_models ... ok
+test agent::tests::stored_tool_cards_restore_ai_approval_reasons_by_call ... ok
+test agent::tests::present_svg_executes_without_side_effects_and_maps_only_completed_cards ... ok
+test agent::tests::tool_output_keeps_terminal_lines_but_parameter_summaries_do_not ... ok
+test agent::web::tests::browser_helpers_preserve_ds4_wire_values ... ok
+test agent::tests::glob_confinement_skips_symlinks_and_honors_cancellation ... ok
+test agent::web::tests::headless_browser_extracts_a_local_page ... ignored, requires an installed Chrome browser
+test agent::tests::present_svg_validation_enforces_the_static_vector_boundary ... ok
+test agent::tests::validation_is_precise_and_prevents_side_effects_until_corrected ... ok
+test app::generation::tests::attached_image_marker_is_model_only ... ok
+test agent::tests::prompts_and_parsers_expose_the_reference_tool_set ... ok
+test agent::tests::successful_structured_file_tools_report_project_paths ... ok
+test agent::tests::glob_finds_all_regular_file_names_deterministically_without_reading_them ... ok
+test agent::tests::standard_agent_skills_are_discovered_prompted_and_read_only ... ok
+test agent::tests::shell_commands_skip_user_startup_files ... ok
+test app::generation::tests::plain_chat_uses_the_same_unmodified_prompt_in_ui_and_headless ... ok
+test app::generation::tests::queued_guidance_is_one_reference_style_user_turn ... ok
+test app::generation::tests::title_context_starts_at_the_first_user_and_ignores_system_rows ... ok
+test app::generation::tests::turn_summary_accumulates_model_continuations ... ok
+test agent::tests::standard_file_tools_can_maintain_only_managed_dev_brain_pages ... ok
+test agent::tests::ralph_round_contexts_are_unique_and_child_jobs_are_cleaned_up ... ok
+test agent::tests::shell_environment_is_loaded_once_from_login_and_interactive_startup_files ... ok
+test app::git::tests::unified_diff_parser_aligns_replacements_for_split_view ... ok
+test app::git::tests::configured_git_diff_options_change_the_rendered_patch ... ok
+test app::git::tests::unstage_works_before_the_first_commit ... ok
+test app::preferences::tests::runtime_config_updates_after_lock_poisoning ... ok
+test app::git::tests::libgit2_status_merges_staged_and_worktree_changes ... ok
+test app::projects::tests::discarding_a_session_removes_every_checkpoint_stage ... ok
+test app::projects::tests::draft_titles_follow_the_stored_session_count ... ok
+test agent::tests::denial_and_stop_cancel_commands_awaiting_approval ... ok
+test app::projects::tests::session_states_are_exclusive_and_ordered ... ok
+test app::projects::tests::sweeping_removes_checkpoints_of_sessions_that_no_longer_exist ... ok
+test app::projects::tests::git_state_lists_and_switches_local_branches ... ok
+test app::tests::background_activity_drives_polling_indicators_and_quit_confirmation ... ok
+test app::git::tests::origin_fetch_pull_and_push_use_the_current_branch ... ok
+test agent::tests::shell_environment_probe_stops_blocked_startup_files ... ok
+test app::tests::chat_html_export_embeds_loaded_a2ui_images_as_data_urls ... ok
+test agent::tests::oversized_shell_output_is_bounded_and_retained_until_cleanup ... ok
+test app::tests::chat_only_follows_updates_while_near_the_tail ... ok
+test app::tests::ds4_gib_and_streaming_cache_inputs_are_typed ... ok
+test app::tests::one_shot_titles_are_reduced_to_a_single_sidebar_line ... ok
+test app::tests::owned_runtime_paths_stay_in_application_support ... ok
+test app::tests::preferences_shortcut_and_dspark_support_are_explicit ... ok
+test app::tests::source_does_not_depend_on_a_sibling_ds4_checkout ... ok
+test app::tests::stored_session_always_selects_its_model ... ok
+test app::tests::tab_scrolls_a_field_back_into_the_preferences_viewport ... ok
+test app::view::a2ui::tests::renderer_input_helpers_cover_dates_numbers_choices_and_weights ... ok
+test app::view::chat::tests::chat_divider_groups_token_counts_by_thousands ... ok
+test app::view::chat::tests::composer_uses_shift_enter_for_line_breaks ... ok
+test app::view::git::tests::split_diff_width_covers_the_longest_side_and_tabs ... ok
+test app::tests::chat_export_preserves_completed_svg_and_keeps_failures_generic ... ok
+test app::view::tests::action_buttons_share_geometry ... ok
+test app::view::tests::context_pie_wedge_follows_the_fill_grade ... ok
+test app::view::tests::download_measurements_are_readable ... ok
+test app::view::tests::raised_surfaces_stay_dark ... ok
+test compaction::tests::a_new_summary_replaces_the_previous_rebuild_summary ... ok
+test compaction::tests::hard_trigger_and_retry_reserve_match_the_reference ... ok
+test compaction::tests::soft_trigger_matches_reference_thresholds ... ok
+test compaction::tests::summary_budget_keeps_answer_room_and_rejects_exhausted_contexts ... ok
+test compaction::tests::summary_control_markup_is_private ... ok
+test compaction::tests::tail_prefers_the_first_user_boundary_after_target ... ok
+test config::tests::defaults_write_an_empty_file_and_reload_unchanged ... ok
+test config::tests::deprecated_flash_config_migrates_and_persists_only_0731 ... ok
+test config::tests::explicit_0731_dspark_opt_out_survives_the_default ... ok
+test config::tests::invalid_files_are_reported_instead_of_reset ... ok
+test app::preferences::tests::selectors_rehydrate_their_generation_and_model_profiles ... ok
+test config::tests::legacy_flat_model_settings_are_dropped_but_the_prompt_is_kept ... ok
+test config::tests::model_profiles_expose_only_their_upstream_thinking_modes ... ok
+test config::tests::only_changed_values_are_written_and_read_back ... ok
+test config::tests::qwen_selection_and_native_defaults_survive_restart ... ok
+test app::generation::tests::model_history_uses_persisted_content_without_changing_the_transcript ... ok
+test app::preferences::tests::multiline_system_prompt_round_trips_through_the_editor ... ok
+test app::generation::tests::legacy_generation_rows_collapse_into_their_user_turn ... ok
+test app::generation::tests::assistant_markdown_keeps_llm_tables ... ok
+test app::view::stats::tests::current_session_summary_uses_persisted_chat_facts ... ok
+test app::generation::tests::last_compaction_selects_its_tail_without_hiding_history ... ok
+test app::tests::chat_export_formats_visible_turns_reasoning_and_bash_tools ... ok
+test app::tests::assistant_stream_keeps_reasoning_separate_from_the_answer ... ok
+test app::generation::tests::forced_compaction_requires_new_visible_chat ... ok
+test app::generation::tests::extension_context_deduplicates_session_start_and_rearms_after_compaction ... ok
+test app::generation::tests::complete_tool_call_in_reasoning_gets_one_correction ... ok
+test app::tests::chat_html_export_is_safe_standalone_and_renders_embedded_surfaces ... ok
+test app::generation::tests::a2ui_auto_switch_waits_for_a_renderable_surface ... ok
+test dev_brain::tests::revision_fixture_preserves_numeric_looking_strings ... ok
+test dev_brain::tests::path_confinement_and_unrelated_notes_are_preserved ... ok
+test dev_brain::tests::skill_index_migration_does_not_overwrite_an_unrelated_root_note ... ok
+test database::tests::context_work_reactivates_an_archived_session ... ok
+test dsml::tests::parses_reference_and_legacy_dsml_syntaxes ... ok
+test dsml::tests::recovers_complete_invokes_before_an_incomplete_one ... ok
+test dsml::tests::recovers_complete_invokes_without_the_outer_closing_tag ... ok
+test database::tests::a2ui_dismissal_persists_a_fresh_surface_boundary ... ok
+test engine::ds4_sampling::tests::heap_fallback_preserves_rng_and_signed_zero_ties ... ok
+test engine::gguf::tests::keeps_unknown_tensor_kinds_rejected ... ok
+test engine::gguf::tests::accepts_exact_mxfp4_block_layout ... ok
+test engine::gguf::tests::mxfp4_scalar_dot_matches_independent_values ... ok
+test engine::gguf::tests::maps_metadata_and_tensor_payload_without_copying_weights ... ok
+test engine::gguf::tests::rejects_mxfp4_dimension_overflow ... ok
+test engine::gguf::tests::rejects_mxfp4_rows_that_are_not_block_aligned ... ok
+test engine::gguf::tests::rejects_truncated_mxfp4_payload ... ok
+test database::tests::inline_svg_tool_call_survives_session_switch_compaction_and_reopen ... ok
+test engine::metal::glm::tests::dsa_indexer_schedule_matches_the_reference ... ok
+test engine::metal::glm::tests::glm53_m5_text_performance_gate ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm53_m5_vision_performance_and_oracle_gate ... ignored, requires GLM 5.3 Q2, its vision sidecar, a fixed image, and Apple Metal
+test engine::metal::glm::tests::glm53_reference_logits_replay_separates_sampling_from_execution ... ignored, requires the installed GLM 5.3 Flash Q2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm53_reference_prompt_tokens_match_shared_runtime ... ignored, installed GLM and DS4SERVER_GLM_REFERENCE_LOG from the standalone reference
+test engine::metal::glm::tests::glm53_two_row_verifier_restores_recurrent_state ... ignored, installed GLM 5.3 and Metal; run under test-supervisor
+test engine::metal::glm::tests::glm_byte_budget_reserves_prefill_before_dynamic_experts ... ok
+test engine::metal::glm::tests::glm_mtp_preserves_target_tokens_and_drafts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm_prefill_schedule_matches_ds4_boundaries ... ok
+test database::tests::sessions_are_ordered_by_recent_use_within_sidebar_groups ... ok
+test engine::metal::glm::tests::repeated_glm_prompt_rewinds_one_token_for_logits ... ok
+test engine::metal::glm::tests::resident_and_streamed_glm_match_ds4_decode_oracles ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
+test engine::metal::glm::tests::streamed_glm_uses_ds4_indexed_prefill_for_long_prompts ... ignored, requires the 197 GiB GLM 5.2 checkpoint and Apple Metal
+test engine::metal::glm::tests::glm_resident_decode_flushes_periodically_without_a_final_or_ssd_flush ... ok
+test engine::metal::gpu::qwen_mtplx_allocation_tests::backend_allocation_policy_is_explicit_inside_and_outside_qwen_scope ... ignored, requires Apple Metal; allocation only, no GPU dispatch or model
+test engine::metal::gpu::qwen_mtplx_allocation_tests::dispatch_retains_allocation_without_changing_data_donation ... ignored, requires Apple Metal, no model weights
+test engine::metal::gpu::qwen_mtplx_allocation_tests::empty_storage_preserves_data_identity_views_and_zero_allocator_usage ... ignored, requires Apple Metal context; empty descriptors, no GPU allocation
+test engine::metal::gpu::qwen_mtplx_allocation_tests::mtplx_tensor_cpu_access_releases_native_storage ... ignored, requires Apple Metal; CPU buffer access only, no GPU dispatch or model
+test engine::metal::markov::tests::installed_markov_worker_dispatch ... ignored, requires installed 0731 DSpark support; CPU dispatch diagnostic, not parity acceptance
+test engine::metal::power_throttle_keeps_ds4_ewma_and_full_power_bypass ... ok
+test engine::metal::markov::tests::markov_workers_reuse_threads_and_buffers_with_ordered_ties ... ok
+test engine::metal::qwen::tests::affine_split_k_matches_reference_shapes_and_scratch_bound ... ok
+test engine::metal::qwen::tests::dense_attention_crossover_is_phase_specific ... ok
+test engine::metal::qwen::tests::qsa_selection_and_capacity_cover_the_full_native_window ... ok
+test engine::metal::qwen::tests::qwen_affine_qmv_and_qmm_match_mlx_reference ... ignored, requires the pinned Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_affine_qmv_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_core_boundary_and_checkpoint_are_stable ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_fused_gdn_input_rows_match_separate_projections ... ignored, requires the pinned Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_gdn_conv_norm_rows_matches_mtplx ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_gdn_delta_rows_matches_mtplx ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_lazy_target_probabilities_match_batched_sampling ... ignored, requires the pinned Qwen metadata and Apple Metal; no model inference
+test engine::metal::qwen::tests::qwen_m5_performance_gate ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_metal_primitives_match_reference_vectors ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_mtp_matches_target_and_restores_its_cache ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_ple_gather_copies_only_the_requested_row ... ok
+test engine::metal::qwen::tests::qwen_ple_hash_contract_matches_golden_boundaries ... ok
+test engine::metal::qwen::tests::qwen_qsa_dense_mask_accepts_unsorted_blocks ... ignored, requires Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_full_context_preallocation_succeeds ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_logical_depth_oracles_cover_prefill_decode_and_state ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_real_layer_executes_the_first_sparse_token ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_qsa_selection_and_attention_remain_bounded_at_native_depths ... ignored, requires Apple Metal and allocates the full QSA score window
+test engine::metal::qwen::tests::qwen_reference_layer_trace ... ignored, diagnostic trace against the pinned MLX-LM reference
+test engine::metal::qwen::tests::qwen_reference_sdpa_trace ... ignored, diagnostic comparison with captured MTPLX SDPA inputs
+test engine::metal::qwen::tests::qwen_reference_stateful_trace ... ignored, diagnostic trace against the pinned MLX-LM reference
+test engine::metal::qwen::tests::qwen_reference_verify_trace ... ignored, diagnostic trace against the pinned MTPLX verifier
+test engine::metal::qwen::tests::qwen_sorted_routed_prefill_matches_grouped_output ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_stage_counters_preserve_async_batches ... ignored, requires pinned Qwen embeddings and Apple Metal; stage profiler opt-in
+test engine::metal::qwen::tests::qwen_target_submission_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen::tests::qwen_verify_layer_profile ... ignored, requires the pinned 105 GB Qwen artifact set and Apple Metal
+test engine::metal::qwen_mtplx::allocator::mtplx_allocations_preserve_logical_bounds_with_reference_rounding ... ignored, requires Apple Metal, no model weights
+test engine::kvstore::tests::opening_reclaims_checkpoints_the_index_cannot_reach ... ok
+test engine::metal::qwen_mtplx::allocator::mtplx_allocator_cache_matches_live_reference_trace ... ok
+test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_heap_limits_and_gpu_lifetime ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx::allocator::mtplx_native_allocator_replays_reference_allocations ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::ar::mtplx_ar_driver_matches_reference ... ignored, requires Apple Metal; unchanged MTPLX generate_ar orchestration oracle
+test engine::metal::qwen_mtplx::array::mtplx_array_detach_clears_every_siblings_inputs_and_primitive ... ok
+test engine::metal::qwen_mtplx::array::mtplx_array_materialization_keeps_geometry_and_rejects_short_storage ... ignored, requires Apple Metal; typed materialized descriptors without model weights
+test engine::metal::qwen_mtplx::array::mtplx_array_layout_preserves_dtype_shape_stride_and_materialization_flags ... ok
+test database::tests::migrations_and_crud_work ... ok
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_dense_mask_graph_matches_reference ... ignored, requires Apple Metal; original MTPLX block mask including sentinel and tails
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_eager_array_graph_matches_reference ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_prefill_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX tiled prefill gather
+test engine::metal::qwen_mtplx::attention::mtplx_qsa_rows_gather_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX QSA rows-gather attention
+test engine::metal::qwen_mtplx::attention::mtplx_sdpa_fallback_graph_matches_reference ... ignored, requires Apple M5 Max; actual Qwen SDPA fallback reference
+test engine::metal::qwen_mtplx::binary::mtplx_graph_binary_preserves_reference_branches ... ignored, requires Apple Metal; binary graph broadcast/layout and donation branches
+test engine::metal::qwen_mtplx::binding::mtplx_installed_module_binding_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS and SSD policy; only small norms and four table rows evaluated
+test engine::metal::qwen_mtplx::cache::mtplx_kv_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX KVCache growth, trim and state receipts
+test engine::metal::qwen_mtplx::cache::mtplx_qsa_cache_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX positional cache, mirror and held state aliases
+test engine::metal::qwen_mtplx::array::mtplx_dtype_promotion_matches_all_reference_pairs ... ok
+test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_scalar_bits_and_custom_kernel_identity ... ignored, requires Metal buffers and external supervisor; scalar ownership only, no GPU dispatch
+test engine::metal::qwen_mtplx::compiled::mtplx_replay_replaces_descriptors_without_rebuilding_primitives ... ok
+test engine::metal::qwen_mtplx::context_copy::prompt_copy_match_and_backoff_preserve_reference_policy ... ok
+test engine::metal::qwen_mtplx::decoder::mtplx_decoder_connected_attention_and_moe_advance_caches ... ignored, requires Apple Metal and external test-supervisor; complete decoder/cache smoke
+test engine::metal::qwen_mtplx::decoder::mtplx_decoder_residual_preserves_graph_order ... ok
+test engine::metal::qwen_mtplx::dense::mtplx_dense_attention_batches_match_reference ... ignored, requires Apple M5 Max; actual BF16 batched matmul reference
+test engine::metal::qwen_mtplx::dense::mtplx_dense_float_graph_matches_reference ... ignored, requires Apple M5 Max; floating Matmul factory and original dispatch
+test engine::metal::qwen_mtplx::dense::mtplx_qsa_score_array_graph_matches_reference ... ignored, requires Apple M5 Max; canonical QSA F32 graph with the original TF32 setting
+test engine::metal::qwen_mtplx::encoder::mtplx_empty_bindings_preserve_nil_dependencies_without_dummy_storage ... ignored, requires Apple Metal; empty original-kernel grids and nil bindings
+test engine::kvstore::tests::longest_compatible_prefix_wins_and_hits_survive_reopen ... ok
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_barrier_epochs_and_commit_thresholds ... ok
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_commits_after_primitives_and_balances_completion ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_scope_routes_operators_and_allocator ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_shared_events_and_error_completion_order ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::encoder::mtplx_encoder_untracked_dependencies_across_encoders_and_commands ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_drains_failure_preserves_cause_and_retries_only_failed_nodes ... ignored, requires Apple Metal; supervised evaluator failure and retry
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_releases_intermediates_and_preserves_descriptor_donation ... ignored, requires Apple Metal; graph ownership and last-consumer release
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_retains_tracer_topology_without_input_donation ... ignored, requires Apple Metal; evaluator tracer donation and retained topology
+test engine::metal::qwen_mtplx::eval::execution_tests::mtplx_eval_runs_cpu_gpu_siblings_and_async_continuations ... ignored, requires Apple Metal; tiny supervised CPU/GPU evaluator graph
+test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_scheduled_detach_and_tracer_errors ... ok
+test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_preserves_bfs_width_and_sibling_choice ... ok
+test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ar_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_AR_REFERENCE required
+test engine::metal::qwen_mtplx::execution::mtplx_full_installed_mtp_prefill_matches_reference ... ignored, full installed MTP model; external guard and DS4_QWEN38_MTP_PREFILL_REFERENCE required
+test engine::metal::qwen_mtplx::eval::mtplx_eval_tape_tracks_stream_fences_per_leaf_and_cross_device_union ... ok
+test engine::metal::qwen_mtplx::execution::mtplx_full_installed_ping_forward_matches_reference ... ignored, full installed model; requires external memory guard and DS4_QWEN38_FORWARD_REFERENCE
+test engine::metal::qwen_mtplx::execution::qwen_power_throttle_drains_before_sleep_and_bypasses_at_full_power ... ignored, requires Apple Metal; run under test-supervisor
+test engine::metal::qwen_mtplx::fused::mtplx_compiled_constant_spelling ... ok
+test engine::kvstore::tests::budget_evicts_the_least_valuable_unprotected_entry ... ok
+test engine::metal::qwen_mtplx::fused::mtplx_compiled_fusion_gpu_matches_reference ... ignored, requires Metal and external supervisor; generated fusion vs pinned SiLU plus multi-output/scalar dispatch
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_forward_matches_model_reference ... ignored, requires Apple Metal; actual MTPLX GDN forward and cache receipts
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_prefill_and_verify_match_model_reference ... ignored, requires Apple Metal; complete GDN Conv1d prefill and fused verify graphs
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_staged_primitives_match_reference ... ignored, requires Apple Metal; 80 actual delta-update receipts and six conv-normalization kernels
+test engine::metal::qwen_mtplx::gdn::mtplx_graph_gdn_step_preserves_views_and_siblings ... ignored, requires Apple Metal; GDN custom primitive strided copies and sibling roots
+test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_event_snapshots_preserve_values_streams_and_shared_errors ... ignored, requires Apple Metal events, no model weights
+test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fence_coherence_preserves_empty_and_partial_word_spans ... ignored, requires Apple Metal3, model-free empty coherent dispatch
+test engine::metal::qwen_mtplx::graph_sync::mtplx_graph_fences_transfer_cpu_gpu_gpu_cpu_in_both_modes ... ignored, requires Apple Metal3, tiny model-free coherent fence kernels
+test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_forward_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX GatedResidual reference receipts
+test engine::metal::qwen_mtplx::hyper::mtplx_graph_hyper_pack_and_reads_match_reference ... ignored, requires Apple Metal; original hyper primitives and runtime pack
+test engine::metal::qwen_mtplx::indexing::mtplx_graph_indexing_preserves_layouts_and_offsets ... ignored, requires Apple Metal; Select layout branches, GatherAxis views and empty arguments
+test engine::metal::qwen_mtplx::indexing::mtplx_scatter_axis_graph_matches_reference ... ignored, requires Apple Metal; pinned Bool ScatterAxis layout and broadcast paths
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_numeric_receipts ... ignored, #103 diagnostic; Metal required, eager versus compiled numeric operands
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_sampling_receipts ... ignored, #103 diagnostic; Metal required, pins currently broken sampling law and invalid rows
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_sorted_gather_receipts ... ignored, #103 diagnostic; M5 and pinned runtime shader required; initialized sentinels
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_tokenizer_receipts ... ignored, #103 diagnostic; installed tokenizer and latest oracle JSONL; no GPU
+test engine::metal::qwen_mtplx::issue103_analysis::issue103_vision_position_receipts ... ignored, #103 diagnostic; small active attention module on Metal; mixed position boundary
+test engine::metal::qwen_mtplx::issue104::issue104_installed_ar_stream_and_state ... ignored, full pinned Bare-Speed model; external memory guard required
+test engine::metal::qwen_mtplx::issue105::issue105_batched_greedy_rejects_each_invalid_row ... ignored, Metal required; run serially under test-supervisor
+test engine::metal::qwen_mtplx::issue106::issue106_complete_state_and_checkpoint_recovery ... ignored, installed model and actual pre-fix checkpoint; external memory guard required
+test engine::metal::qwen_mtplx::issue106::issue106_upstream_exit_matrix ... ignored, installed model, scripted logits on real caches; external memory guard required
+test engine::metal::qwen_mtplx::issue108::issue108_attention_chunk_positions ... ignored, Metal and pinned MTPLX chunk fixtures; external supervisor required
+test engine::metal::qwen_mtplx::compile_graph::mtplx_compile_simplify_preserves_reference_parent_rules ... ok
+test engine::metal::qwen_mtplx::issue109::issue109_gate_phase_and_dtype_laws ... ignored, Metal and exhaustive pinned gate fixtures; external supervisor required
+test engine::metal::qwen_mtplx::issue110::issue110_actual_projection_rows_match_oracle ... ignored, installed layer weights and pinned oracle receipts; external memory supervisor required
+test database::tests::session_model_migration_preserves_data_and_accepts_catalog ... ok
+test engine::metal::qwen_mtplx::issue110::issue110_guard_allocation_refusal_releases_graph_and_retries ... ignored, Metal allocation refusal and retry; external supervisor required
+test engine::metal::qwen_mtplx::issue110::issue110_large_cold_and_warm_prefill ... ignored, full installed model and large partial chunks; serial memory supervisor required
+test engine::metal::qwen_mtplx::issue112_bench::issue112_hidden_prefill ... ignored, full installed model hidden-state prefill; serial supervisor required
+test engine::metal::qwen_mtplx::issue112_bench::issue112_paired_prefill ... ignored, full installed model; serial memory supervisor and explicit output directory
+test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_dequantize_matches_reference ... ignored, requires Apple Metal; original table dequantize dtype/group/bit/layout receipts
+test engine::metal::qwen_mtplx::linear::mtplx_graph_affine_quantize_matches_reference ... ignored, requires Apple Metal; original affine quantize dtype/group/bit/layout receipts
+test engine::metal::qwen_mtplx::linear::mtplx_graph_gather_qmm_matches_reference ... ignored, requires Apple Metal; original GatherQMM array layouts and expert routing
+test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_batched_identity ... ignored, requires Apple Metal and external supervisor; strided/broadcast batched affine dispatch
+test engine::metal::qwen_mtplx::linear::mtplx_graph_quantized_linear_matches_reference ... ignored, requires Apple M5 Max Metal; 78 pinned affine fixtures through lazy graph
+test engine::metal::qwen_mtplx::load::mtplx_safetensors_load_laziness_boundaries_and_lifetime ... ignored, requires Apple Metal; bounded CPU Load and dependent GPU graph
+test engine::metal::qwen_mtplx::mlp::mtplx_graph_shared_mlp_matches_reference ... ignored, requires Apple M5 Max; actual MTPLX shared MLP reference receipts
+test engine::metal::qwen_mtplx::mlp::mtplx_graph_split_and_compiled_silu_layouts ... ignored, requires Apple Metal; Split sibling ownership and compiled SiLU layouts
+test engine::metal::qwen_mtplx::linear::sorted_gather_guard_matches_pinned_call_predicates ... ok
+test engine::metal::qwen_mtplx::moe::mtplx_graph_sparse_moe_matches_reference ... ignored, requires Apple M5 Max; full MoE Array graph, no model weights; use test-supervisor
+test engine::metal::qwen_mtplx::moe_sorted::issue112_all_kernel_variants ... ignored, Metal synthetic all supported row-map variants; external supervisor
+test engine::metal::qwen_mtplx::moe_sorted::issue112_canary_failures ... ignored, Metal canary rejection and sticky fallback; external supervisor
+test engine::metal::qwen_mtplx::model::mtplx_model_compile_policy_and_run_boundaries ... ok
+test engine::metal::qwen_mtplx::moe_sorted::issue112_full_model_state ... ignored, full installed model; validates all cold/warm state receipts against the pre-port oracle
+test engine::metal::qwen_mtplx::moe_sorted::issue112_installed_gate_up ... ignored, installed projection weights; external supervisor required
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_embedding_matches_reference ... ignored, requires Apple Metal; full resident NGram graph and lazy decode history
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_gpu_rows_and_history_match_reference ... ignored, requires Apple Metal; original lazy NGram GPU row graph, no table
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_resident_matches_reference ... ignored, requires Apple Metal; original NGramTable resident gather/dequantize receipts
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_table_priority ... ignored, requires Apple Metal; NGram table routing and missing-sidecar behavior
+test engine::metal::qwen_mtplx::ngram_sidecar::mtplx_ngram_sidecar_matches_reference ... ignored, requires Apple Metal; actual file-backed MTPLX sidecar/LRU/dequantization receipts
+test engine::metal::qwen_mtplx::moe_sorted::row_map_applicability ... ok
+test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_allclose_branches ... ignored, requires Apple Metal; stage verification special values and original reduction branches
+test engine::metal::qwen_mtplx::ngram_stage::mtplx_ngram_stage_matches_reference ... ignored, requires Apple Metal; original complete stage lifecycle and file-backed tables
+test engine::metal::qwen_mtplx::normalization::mtplx_graph_norm_modules_match_reference ... ignored, requires Apple Metal; connected Qwen normalization graph golden fixtures
+test engine::metal::qwen_mtplx::normalization::mtplx_graph_rms_norm_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph RMSNorm donation, copy, broadcast and looped kernels
+test engine::metal::qwen_mtplx::normalization::mtplx_logsumexp_graph_matches_reference ... ignored, requires Apple Metal; original sampler LogSumExp branches
+test engine::metal::qwen_mtplx::normalization::mtplx_softmax_graph_storage_matches_reference ... ignored, requires Apple Metal; original Softmax type and storage branches
+test engine::metal::qwen_mtplx::ops::mtplx_concatenate_preserves_general_general_copy_semantics ... ignored, requires Apple Metal; Concatenate promotion, offsets, strided writers and validation
+test engine::metal::qwen_mtplx::ops::mtplx_owned_operations_build_and_execute_qsa_layout_graph ... ignored, requires Apple Metal; self-contained lazy QSA-layout recipe, no callback map
+test engine::metal::qwen_mtplx::ops::mtplx_slice_and_contiguous_preserve_cache_window_semantics ... ignored, requires Apple Metal; normalized Slice and Contiguous cache-window graph
+test engine::metal::qwen_mtplx::owner::issue105_product::issue105_product_ar_cleanup_and_http_retry ... ignored, installed Qwen model; serial test-supervisor with memory guard required
+test engine::metal::qwen_mtplx::owner::issue105_product::issue105_product_mtp_cleanup_and_http_retry ... ignored, installed Qwen MTP model; serial test-supervisor with memory guard required
+test engine::metal::qwen_mtplx::owner::issue106_product::issue106_product_discards_old_checkpoint_and_recovers_cold ... ignored, installed model and actual pre-fix checkpoint; external memory guard required
+test engine::metal::qwen_mtplx::owner::issue107_product::issue107_vision_product ... ignored, installed model plus vision tower; serial external memory supervisor required
+test engine::metal::qwen_mtplx::owner::issue109_product::issue109_verify_rows_and_warm_checkpoint ... ignored, installed model; serial external memory supervisor required
+test engine::metal::qwen_mtplx::owner::issue109_product::issue113_default_verify_and_checkpoint ... ignored, installed model and actual pre-head-chunk checkpoint; serial external memory supervisor
+test engine::metal::qwen_mtplx::owner::issue111_product::issue111_token_prefix_cache ... ignored, installed model; serial external memory supervisor required
+test engine::metal::qwen_mtplx::owner::issue116_product::issue116_normalized_token_cache ... ignored, installed model; serial external memory supervisor required
+test engine::metal::qwen_mtplx::owner::qwen_product_generator_chat_matches_reference ... ignored, full installed model through product Generator; external supervisor and chat reference required
+test engine::metal::qwen_mtplx::owner::qwen_vision_chat_checkpoint_preserves_image_identity ... ignored, requires installed Qwen, Metal, and DS4_QWEN_VISION_IMAGE; run under test-supervisor
+test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_connected_matches_reference ... ignored, requires Apple M5 Max; complete original PLELayer with real lookup
+test engine::metal::qwen_mtplx::ple::mtplx_graph_ple_projection_and_cache_match_reference ... ignored, requires Apple M5 Max; actual MTPLX PLELayer after embedding lookup
+test engine::metal::qwen_mtplx::policy::qwen_server_policy_matches_original_functions ... ignored, requires DS4_QWEN38_POLICY_REFERENCE from pinned MTPLX; CPU-only
+test engine::metal::qwen_mtplx::owner::qwen_mtp_progress_publishes_without_turn_completion_or_double_counting ... ok
+test engine::metal::qwen_mtplx::fused::mtplx_compile_fusion_preserves_parent_and_output_boundaries ... ok
+test engine::metal::qwen_mtplx::prefill::mtplx_mtp_prefill_driver_matches_reference ... ignored, requires Apple Metal; original streaming MTP history prefill
+test engine::metal::qwen_mtplx::prefill::mtplx_prefill_driver_matches_reference ... ignored, requires Apple Metal; original _prefill control flow and raw cache roots
+test engine::metal::qwen_mtplx::qsa_attention::mtplx_qsa_attention_module_matches_reference ... ignored, requires Metal and unchanged whole-Attention receipts; no model download
+test engine::metal::qwen_mtplx::policy::sorted_moe_switch_and_family_rehearsal ... ok
+test engine::metal::qwen_mtplx::qsa_compile::mtplx_compiled_indexer_array_replay_matches_reference ... ignored, requires Metal and original compiled indexer receipts; no model weights
+test engine::metal::qwen_mtplx::qsa_core::mtplx_qsa_core_host_matches_reference ... ignored, requires Metal and original host-core receipts; no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_cache_donation_requires_descriptor_data_and_allocation_contract ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_completion_owns_buffers_until_gpu_finishes ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_evaluator_retains_inputs_and_siblings_not_primary_data ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::qsa_indexer::issue107::issue107_real_geometry_query_and_block_start_positions ... ignored, Metal and independently verified pinned indexer fixtures; external supervisor required
+test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_canonical_indexer_routes_match_reference ... ignored, requires Metal and original ongoing indexer routing receipts; no weights
+test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_compiled_commit_matches_reference ... ignored, requires Metal and original indexer lifecycle receipts; no model weights
+test engine::metal::qwen_mtplx::qsa_graph::qsa_schedule_preserves_the_selected_sibling ... ok
+test engine::metal::qwen_mtplx::qsa_indexer::mtplx_qsa_indexer_prefix_matches_reference ... ignored, requires Apple Metal, original stateful QSAIndexer receipts, no model weights
+test engine::metal::qwen_mtplx::fused::mtplx_compiled_emitter_matches_all_pinned_silu_bodies ... ok
+test engine::metal::qwen_mtplx::qsa_prefill::mtplx_qsa_prefill_graph_matches_reference ... ignored, requires Metal and original two-stage prefill receipts, no weights
+test engine::metal::qwen_mtplx::qsa_prepare::mtplx_qsa_prepare_graph_matches_reference ... ignored, requires Apple Metal; parameterized original QSA preparation and traced positions
+test engine::metal::qwen_mtplx::policy::retired_ar_pipeline_flag_cannot_change_server_or_turn_policy ... ok
+test engine::metal::qwen_mtplx::qsa_select::mtplx_qsa_fused_graph_matches_reference ... ignored, requires Apple Metal and original QSA selector receipts; no model weights
+test engine::metal::qwen_mtplx::random::mtplx_sampler_prefix_matches_reference ... ignored, requires Apple Metal; original target-prefix sampler and default RNG
+test engine::metal::qwen_mtplx::random::mtplx_sampler_resident_matches_reference ... ignored, requires Apple Metal; original MTPLX resident sampling and keyed RNG
+test engine::metal::qwen_mtplx::reduce::mtplx_sampler_argmax_matches_reference ... ignored, requires Apple Metal; original MTPLX greedy ArgReduce and storage
+test engine::metal::qwen_mtplx::reduce::mtplx_sum_graph_matches_reference ... ignored, requires Apple Metal; original shared sum dispatch and QSA mean
+test engine::metal::qwen_mtplx::residency::mtplx_residency_matches_reference_lifecycle_cases ... ignored, requires Apple Metal 3 / macOS 15, no model weights
+test engine::metal::qwen_mtplx::rope::mtplx_rope_graph_matches_reference ... ignored, requires Apple Metal; actual MTPLX RoPE, static YaRN and multi-axis receipts
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_batched_matches_reference ... ignored, requires Apple Metal; original MTPLX bound and serial batch distributions
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_entry_matches_reference ... ignored, requires Apple Metal; actual MTPLX generator sampling entry
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_fallback_matches_reference ... ignored, requires Apple Metal; original sparse fallback and host dense sampler
+test engine::metal::qwen_mtplx::request::qwen_request_progress_uses_ui_frontiers_and_decode_timer ... ok
+test engine::metal::qwen_mtplx::request::qwen_request_settings_override_backend_policy ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_penalties_match_reference ... ignored, requires Apple Metal; original GPU and CPU raw-logit penalties
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_selection_matches_reference ... ignored, requires Apple Metal; original MTPLX sampler selection, no model weights
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_serial_support_matches_reference ... ignored, requires Apple Metal; actual MTPLX serial candidate route, no weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_primitive_holds_data_and_temporaries_in_fifo_order ... ignored, requires Apple Metal shared storage, no GPU dispatch or model weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_cpu_dispatch_counts_tenth_and_completes_after_failed_work ... ok
+test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_cpu_events_propagate_errors_without_consuming_source ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_streams_preserve_fifo_errors_and_shutdown ... ok
+test engine::metal::qwen_mtplx::sdpa::mtplx_sdpa_vector_matches_reference ... ignored, requires Apple M5 Max; actual pinned fused SDPA kernels
+test engine::metal::qwen_mtplx::session_cache::issue114_snapshot_stage_preserves_zero_bits ... ignored, Metal; exact pinned lazy/deep snapshot zero-sign law
+test engine::metal::qwen_mtplx::session_cache::mtplx_full_installed_chat_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_CHAT_REFERENCE required
+test engine::metal::qwen_mtplx::session_cache::mtplx_session_boundary_policy ... ok
+test engine::metal::qwen_mtplx::session_cache::mtplx_session_final_state_requires_capture_and_safe_cache ... ok
+test engine::metal::qwen_mtplx::session_checkpoint::qwen_checkpoint_roundtrip_and_failed_restore_preserve_live_state ... ignored, model-free Metal checkpoint; external supervisor and DS4_QWEN38_CHECKPOINT_DIR required
+test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_host_import_matches_reference ... ignored, model-free Metal imports; external supervisor and DS4_HOST_IMPORT_REFERENCE required
+test engine::metal::qwen_mtplx::session_codec::mtplx_session_codec_matches_original_blobs_and_restore ... ignored, tiny GPU arrays; external supervisor and DS4_SESSION_CODEC_REFERENCE required
+test engine::metal::qwen_mtplx::sort::mtplx_sort_graph_matches_reference ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::sparse_attention::mtplx_sparse_attention_graph_matches_reference ... ignored, requires Apple M5; actual MTPLX sparse Decode and TensorOps Prefill consumers
+test engine::metal::qwen_mtplx::speculative::mtplx_full_installed_mtp_generation_matches_reference ... ignored, full installed model; external guard and DS4_QWEN38_MTP_GENERATION_REFERENCE required
+test engine::metal::qwen_mtplx::stream::mtplx_stream_registry_preserves_thread_defaults_templates_and_clear ... ok
+test engine::metal::qwen_mtplx::stream::mtplx_stream_routes_queues_events_and_finalizes_all_under_pressure ... ignored, requires Apple Metal, tiny model-free two-queue copy
+test database::tests::chat_and_compaction_history_survive_reopen_and_session_deletion ... ok
+test engine::metal::qwen_mtplx::submission::mtplx_submission_attaches_at_flush_eval_finish_and_drop ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx::unary::mtplx_graph_unary_preserves_reference_layout_and_values ... ignored, requires Apple Metal; graph unary layout, donation and kernel boundaries
+test engine::metal::qwen_mtplx::verify_sdpa::issue113_head_chunk_outputs_and_dispatch ... ignored, Metal, pinned head-chunk numerical and dispatch fixtures; serial supervisor
+test engine::metal::profile::tests::profile_reports_locality_and_lru_hits ... ok
+test engine::metal::qwen_mtplx::views::mtplx_dtype_conversion_and_views_preserve_values_layout_and_donation ... ignored, requires Apple Metal; original conversion/view kernels and donation in lazy eval
+test engine::metal::qwen_mtplx::views::mtplx_dynamic_slice_graph_matches_reference ... ignored, requires Metal and original dynamic slice receipts; no model weights
+test engine::metal::qwen_mtplx::views::mtplx_reshape_shares_or_copies_in_the_connected_evaluator ... ignored, requires Apple Metal; original Reshape copy kernels and shared backing
+test engine::metal::qwen_mtplx::views::mtplx_views_preserve_aliases_offsets_strides_and_gpu_output ... ignored, requires Apple Metal; connected zero-copy views and offset-aware original copy kernel
+test engine::metal::qwen_mtplx::vision::qwen_vision_tower_matches_mtplx_image ... ignored, requires Metal and pinned Qwen vision artifacts/reference values
+test engine::metal::qwen_mtplx::weights::mtplx_checkpoint_projection_fusions_match_reference ... ignored, requires Apple Metal; bounded lazy checkpoint Load and fusion
+test engine::metal::qwen_mtplx::weights::mtplx_installed_checkpoint_fusion_metadata_matches_reference ... ignored, requires installed DS4_QWEN38_ARTIFACTS; headers only, never evaluate weights
+test engine::metal::qwen_mtplx_tests::mtplx_affine_selection_preserves_reference_boundaries ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_canonical_hyper_and_gdn_kernels_match_reference ... ignored, requires Apple Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_canonical_source_bodies_preserve_pinned_hashes ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_connected_delta_update_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gather_sort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_decode_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_forward_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_gdn_staged_matches_model_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_norm_modules_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_router_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_sparse_moe_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_connected_switch_glu_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_gate_up_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_cache_lifecycle_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_backings_and_dynamic_copies_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_routing_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_cache_window_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_capacity_matches_reference ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_indexer_modes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_compiled_scalar_sources_match_reference ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_connected_cache_selection_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_eager_topk_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_f32_scores_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_preparation_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_indexer_routes_match_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_producer_dispatch_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx::verify_sdpa::issue113_plan_and_policy_match_pinned_oracle ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_selector_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_affine_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_argsort_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_gather_matches_reference ... ignored, requires Apple M5 Max Metal, no model weights
+test engine::metal::qwen_mtplx::submission::mtplx_all_native_command_commits_pass_the_submission_hook ... ok
+test engine::metal::tests::batch_raw_span_includes_the_chunk_and_previous_window ... ok
+test engine::metal::tests::compression_schedule_tracks_the_deepseek_model_shape ... ok
+test engine::metal::tests::decode_feature_policy_keeps_device_generations_and_rollbacks_separate ... ok
+test engine::metal::tests::decode_raw_span_tracks_the_logical_sliding_window ... ok
+test engine::metal::tests::directional_steering_matches_the_ds4_token_oracle ... ignored, requires the installed Flash GGUF, DS4 steering fixture, and a Metal device
+test engine::metal::tests::dspark_confidence_quantizes_q8_input_like_ds4 ... ok
+test engine::metal::tests::dspark_markov_activation_matches_q8_rounding ... ok
+test engine::metal::tests::dspark_matches_original_summary_cycles ... ignored, requires installed 0731/DSpark and DS4SERVER_DSPARK_REFERENCE cycle recording
+test engine::metal::tests::dspark_runs_a_target_owned_greedy_cycle ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures
+test engine::metal::tests::dspark_scheduler_matches_the_ds4_default_window ... ok
+test engine::metal::tests::dspark_verifier_selects_only_the_committed_row ... ok
+test engine::metal::tests::flash_0731_long_context_crosses_indexed_prefill_boundary ... ignored, requires a 0731 Flash GGUF and an Apple M5 device
+test engine::metal::tests::flash_0731_m5_decode_performance_gate ... ignored, requires a 0731 Flash GGUF and an Apple M5 device
+test engine::metal::tests::flash_0731_resident_and_ssd_streaming_choose_the_same_tokens ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
+test engine::metal::tests::flash_0731_runs_exact_sampled_dspark ... ignored, requires the installed 0731 target and checkpoint-specific DSpark GGUF fixtures
+test engine::metal::tests::flash_0731_ssd_streaming_maps_and_seeds_batched_prefill_layers ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
+test engine::metal::tests::generated_hotlists_match_each_model_shape ... ok
+test engine::metal::tests::half_conversion_handles_normal_and_subnormal_values ... ok
+test engine::metal::tests::pro_q4_model_spans_remain_isolated ... ok
+test engine::metal::tests::pro_resident_admission_includes_weights_context_and_scratch ... ok
+test engine::metal::tests::pro_ssd_streaming_full_layer_and_selected_batch_agree ... ignored, requires the installed DeepSeek V4 Pro GGUF and Apple Metal
+test engine::metal::tests::resident_multi_session_switching_preserves_each_kv_frontier ... ignored, requires the installed 81 GiB Flash GGUF fixture and a Metal device
+test engine::metal::tests::ssd_streaming_supports_dspark ... ignored, requires the installed 0731 Flash and checkpoint-specific DSpark GGUF fixtures
+test engine::metal::tests::streaming_memory_plan_matches_ds4_graph_formulas ... ok
+test engine::metal::vision::tests::glm53_resize_matches_reference_token_grids ... ok
+test engine::metal::qwen_mtplx::scheduler::mtplx_scheduler_activity_preserves_reference_pressure_and_wait_rules ... ok
+test engine::qwen::tests::metadata_and_safetensors_boundaries_fail_closed ... ok
+test engine::qwen::tests::pinned_artifact_set_loads_and_renders_goldens ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
+test engine::qwen::tests::pinned_qwen_metadata_matches_product_without_executor_state ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
+test engine::qwen_vision::qwen_image_cache_keys_distinguish_pixels_and_preserve_prior_text ... ok
+test engine::qwen::tests::frozen_inventory_and_memory_categories_are_exact ... ok
+test engine::qwen_vision::qwen_vision_positions_and_resize_contract ... ok
+test engine::sampling_tests::checkpoint_headers_identify_the_exact_model ... ok
+test engine::sampling_tests::checkpoint_rebuilds_explain_compatibility_and_history_misses ... ok
+test engine::sampling_tests::checkpoint_tag_accepts_an_unchanged_prefix_before_reminders ... ok
+test engine::sampling_tests::checkpoint_tag_covers_the_canonical_chat_state ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_distribution_and_rng_match_reference ... ok
+test engine::sampling_tests::ds4_sampler_real_logits_cpu_benchmark ... ignored, CPU-only; requires DS4_SAMPLER_RECORDING with the recorded GLM logits
+test engine::sampling_tests::exact_delta_sampling_accepts_or_corrects_the_draft ... ok
+test engine::sampling_tests::generated_text_drains_incomplete_bytes_before_phase_changes_and_finish ... ok
+test engine::sampling_tests::generated_text_preserves_thinking_utf8_and_split_stops ... ok
+test engine::sampling_tests::metal_executes_real_flash_token ... ignored, requires the 80 GiB Flash checkpoint and Apple Metal
+test engine::metal::qwen_mtplx_tests::mtplx_qsa_prefill_workspace_matches_reference ... ok
+test engine::sampling_tests::qwen_rng_matches_numpy_pcg64 ... ok
+test engine::sampling_tests::qwen_sampling_uses_mtplx_global_nucleus_mass_before_top_k ... ok
+test engine::sampling_tests::resumed_prefill_crossover_matches_each_ds4_graph ... ok
+test engine::sampling_tests::sampling_probabilities_match_ds4_filter_order ... ok
+test engine::sampling_tests::split_utf8_token_bytes_are_joined_before_decoding ... ok
+test engine::sampling_tests::top_k_and_stream_stops_are_applied_before_output ... ok
+test engine::sampling_tests::workspace_instruction_identity_participates_in_append_only_kv_keys ... ok
+test engine::sampling_tests::zero_temperature_is_greedy ... ok
+test engine::tokenizer::issue111::issue111_ds4_families ... ignored, CPU-only; installed DS4 GGUFs and exact oracle IDs
+test engine::tokenizer::issue111::issue111_every_unicode_scalar_split ... ignored, CPU-only; exhaustive pinned Qwen split receipt
+test engine::tokenizer::issue111::issue111_qwen_tokens_and_requests ... ignored, CPU-only; installed Qwen tokenizer and pinned exact IDs
+test engine::metal::qwen_mtplx::ngram::mtplx_ngram_cpu_rows_and_raw_history_match_reference ... ok
+test engine::tokenizer::issue116::issue116_nfc_tokens_and_requests ... ignored, CPU-only; pinned NFC receipts and installed Qwen tokenizer
+test engine::tokenizer::issue116::nfc_contract_and_composition ... ok
+test engine::tokenizer::issue116::only_affected_qwen_history_changes_checkpoint_tags ... ok
+test engine::tokenizer::tests::character_helpers_are_total_at_every_byte_offset ... ok
+test engine::tokenizer::tests::ds4_chat_matches_original_session_tokens ... ignored, CPU-only; requires DS4SERVER_CHAT_REFERENCE and its installed GGUF
+test engine::tokenizer::tests::qwen_checkpoint_continuation_matches_full_conversation ... ignored, requires the installed Qwen tokenizer, but no GPU or model weights
+test engine::tokenizer::tests::qwen_plain_chat_matches_mtplx_server_tokens ... ignored, requires local tokenizer and tools/qwen-chat-reference.py output; no inference
+test engine::tokenizer::tests::reasoning_prefixes_match_each_model_family_contract ... ok
+test engine::validation::tests::configured_glm53_main_fixture_passes_the_exact_layout ... ok
+test engine::validation::tests::configured_glm53_vision_fixture_passes_the_exact_layout ... ok
+test engine::tokenizer::issue111::qwen_split_boundaries ... ok
+test engine::validation::tests::installed_dspark_fixture_passes_the_target_layout ... ok
+test engine::validation::tests::mxfp4_is_limited_to_routed_experts ... ok
+test dev_brain::tests::parser_and_index_rebuild_ignore_links_in_code ... ok
+test extensions::tests::duplicate_skill_names_and_update_rollback_are_deterministic ... ok
+test dev_brain::tests::validation_names_every_drifted_source_and_requires_semantic_refresh ... ok
+test extensions::tests::hook_output_is_bounded_typed_and_event_matched ... ok
+test engine::sampling_tests::probability_ratio_speculation_preserves_the_target_distribution ... ok
+test extensions::tests::manifest_and_paths_are_validated_without_following_escapes ... ok
+test extensions::tests::pinned_ponytail_runs_all_lifecycle_modes ... ignored, downloads the pinned Ponytail reference integration
+test extensions::tests::registry_persists_enable_disable_errors_and_uninstall ... ok
+test engine::metal::qwen_mtplx::array::mtplx_array_releases_sibling_cycles_and_deep_input_graphs ... ok
+test extensions::tests::symlink_and_variable_escapes_are_rejected ... ok
+test instructions::tests::changes_removals_resume_and_compaction_are_append_only ... ok
+test instructions::tests::empty_and_repaired_baselines_do_not_repeat_prompt_content ... ok
+test instructions::tests::discovery_is_global_to_local_deduplicated_and_confined ... ok
+test instructions::tests::newly_crossed_scope_is_added_once_and_unavailable_content_is_retained ... ok
+test metrics::tests::cache_report_splits_buckets_and_folds_index_files_into_their_checkpoint ... ok
+test metrics::tests::decode_timer_excludes_later_work_and_resets_between_requests ... ok
+test metrics::tests::prompt_timing_keeps_unmeasured_components_distinct_from_zero ... ok
+test metrics::tests::removed_preview_metric_code_stays_decodable ... ok
+test metrics::tests::snapshots_track_runtime_and_server_counters ... ok
+test model::qwen::qwen_vision_is_an_independent_managed_artifact_set ... ok
+test model::qwen::tests::embedded_manifest_is_the_frozen_complete_set ... ok
+test model::qwen::tests::verification_rejects_hash_mismatch_and_honors_cancellation ... ok
+test model::transfer::tests::cancellation_keeps_the_partial_file_for_the_next_run ... ok
+test model::transfer::tests::catalog_and_checksum_verification_are_explicit ... ok
+test instructions::tests::frames_escape_control_tags_and_enforce_utf8_byte_limits ... ok
+test model::transfer::tests::qwen_set_verifies_reloads_and_removes_as_one_managed_artifact ... ignored, requires DS4_QWEN38_ARTIFACTS to point at the pinned 105 GB source
+test model::transfer::tests::managed_artifact_inventory_and_delete_include_partial_files ... ok
+test model::transfer::tests::restart_resumes_at_the_existing_partial_byte ... ok
+test model::transfer::tests::verification_reports_bytes_read ... ok
+test model_eval::tests::acceleration_receipt_reports_effective_sampling_contract ... ok
+test model_eval::tests::acceleration_switch_uses_each_models_native_path ... ok
+test model_eval::tests::canary_cross_phase_samples_keep_their_start_phase ... ok
+test dev_brain::tests::skill_validation_and_default_guide_restore_are_strict ... ok
+test model_eval::tests::evaluations_exclude_overlapping_workers_and_release_on_exit ... ok
+test model_eval::tests::external_canary_observes_phases_and_finishes_on_eof ... ignored, Metal probe only; DS4SERVER_CANARY_BINARY must point to the current release binary
+test model_eval::tests::external_canary_protocol_accepts_only_known_phases ... ok
+test model_eval::tests::parses_safe_defaults_and_overrides ... ok
+test model_eval::tests::rejects_unbounded_or_invalid_limits ... ok
+test model_eval::tests::repeated_prompts_form_one_ordered_conversation ... ok
+test model_eval::tests::warmup_timing_separates_loading_without_inventing_missing_durations ... ok
+test model_eval::tests::worker_progress_events_advance_the_watchdog ... ok
+test native_edit::tests::edit_history_supports_undo_and_redo ... ok
+test native_edit::tests::paste_replays_a_complete_command_shortcut ... ok
+test native_media::tests::media_player_only_accepts_remote_http_urls ... ok
+test native_menu::tests::edit_items_are_custom_menu_events ... ok
+test native_menu::tests::user_guide_uses_unmodified_f1 ... ok
+test native_menu::tests::view_items_keep_the_selected_pane_checked ... ok
+test runtime::tests::dropping_active_generation_cancels_it ... ok
+test runtime::tests::operation_failures_use_the_matching_event ... ok
+test runtime::tests::operation_tracking_is_consistent ... ok
+test runtime::tests::runtime_panic_recovery_accepts_the_next_operation ... ok
+test server::request::tests::reasoning_requests_preserve_each_models_exact_modes ... ok
+test server::tests::canonical_tool_replay_preserves_argument_types ... ok
+test server::tests::connection_slots_enforce_the_limit_and_release_on_drop ... ok
+test server::tests::content_arrays_match_ds4_text_projection ... ok
+test server::tests::context_limit_errors_use_protocol_standard_fields ... ok
+test server::tests::cors_headers_are_opt_in_and_cover_preflight ... ok
+test server::tests::dsml_tool_stream_projects_incremental_json_arguments ... ok
+test server::tests::generated_dsml_becomes_openai_tool_calls ... ok
+test server::tests::generated_tool_parser_accepts_ds4_recovery_syntaxes ... ok
+test server::tests::model_metadata_uses_the_requested_alias_and_default_token_limit ... ok
+test server::tests::protocol_tool_results_require_live_or_replayed_call_ids ... ok
+test dev_brain::tests::source_hash_drift_excludes_page_from_authoritative_queries ... ok
+test server::tests::responses_parser_rejects_dropped_state_and_preserves_hosted_calls ... ok
+test server::tests::sampled_tool_replay_survives_server_restart ... ok
+test server::tests::streaming_errors_match_ds4_server_framing ... ok
+test dev_brain::tests::verified_skills_are_indexed_and_injected_on_demand ... ok
+test server::tests::tool_prompt_matches_ds4_server_text ... ok
+test server::tests::tool_stream_projects_ds4_recovery_syntaxes ... ok
+test settings::tests::captured_ds4_cli_options_are_uniquely_classified ... ok
+test settings::tests::effective_settings_build_one_engine_and_turn_configuration ... ok
+test settings::tests::effective_settings_preserve_unset_model_defaults ... ok
+test settings::tests::execution_settings_keep_ds4_automatic_values_and_glm_rules ... ok
+test settings::tests::kv_cache_gates_follow_the_ds4_store_rules ... ok
+test settings::tests::reasoning_selection_is_exact_and_model_specific ... ok
+test settings::tests::runtime_settings_preserve_automatic_values_and_cross_field_rules ... ok
+test settings::tests::speculative_settings_match_acceleration_defaults_and_dependencies ... ok
+test dev_brain::tests::validation_keeps_broken_links_editable_and_reports_provenance_errors ... ok
+test model_eval::tests::cpu_accounting_matches_getrusage_units ... ok
+test dev_brain::tests::source_versions_follow_that_file_instead_of_the_whole_worktree ... ok
+test server::tests::streaming_prefill_keeps_the_client_connection_alive ... ok
+test engine::validation::tests::installed_ds4_fixture_opens_and_renders_a_prompt ... ok
+test model::transfer::tests::stalled_download_times_out_and_keeps_the_partial_file ... ok
+test extensions::tests::direct_hook_runner_isolated_environment_and_handles_failures ... ok
+test server::tests::request_deadline_stops_slow_clients ... ok
+test engine::sampling_tests::ds4_sampler_matches_original_reference_tokens_and_rng ... ok
+kill: 2236: No such process
+test agent::tests::dropping_session_tools_stops_live_jobs_and_removes_output ... ok
+test agent::tests::execution_failures_and_timeouts_have_distinct_codes ... ok
+test engine::qwen_vision::qwen_image_preprocessing_matches_pillow_mtplx ... ok
+test extensions::tests::lifecycle_matchers_preserve_declared_event_order ... ok
+test agent::tests::local_file_and_bash_tools_execute_in_the_project ... ok
+test engine::metal::qwen_mtplx_tests::mtplx_runtime_shaders_are_pinned ... ok
+test extensions::tests::hook_failures_are_bounded_actionable_and_non_shell ... ok
+test engine::metal::qwen_mtplx::sampling::mtplx_sampler_mixed_distributions_match_reference ... ok
+
+test result: ok. 316 passed; 0 failed; 239 ignored; 0 measured; 0 filtered out; finished in 5.60s
+
+     Running unittests tools/qwen38-artifacts.rs (target/debug/deps/qwen38_artifacts-3a34dfcd9c4f3de8)
+
+running 4 tests
+test tests::artifact_names_cannot_escape_the_output_directory ... ok
+test tests::affine_sample_digest_uses_packed_values_scales_and_biases ... ok
+test tests::safetensors_validation_rejects_duplicate_names_and_invalid_ranges ... ok
+test tests::artifact_download_resumes_an_authenticated_partial ... ok
+
+test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
+
+     Running unittests tools/test-supervisor.rs (target/debug/deps/test_supervisor-978addc9e5e8dbda)
+
+running 4 tests
+test supervisor::tests::direct_worker_arguments_are_not_rust_test_arguments ... ok
+test supervisor::tests::split_records_cannot_be_interleaved_with_watchdog_samples ... ok
+test supervisor::tests::split_probe_records_do_not_reset_progress ... ok
+ready{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+ready
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+{"event":"gpu_canary_sample","ok":true}
+progressprogressprogressprogressprogressprogressprogressprogresstest supervisor::tests::watchdog_stops_memory_and_silence_but_not_total_runtime ... ok
+
+test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.38s
+

docs/issue116-nfc/tokens.json

@@ -0,0 +1 @@
+[{"text": "", "normalized": "", "plain_ids": [], "ids": [], "decoded": ""}, {"text": "ASCII 123\n", "normalized": "ASCII 123\n", "plain_ids": [54556, 220, 16, 17, 18, 198], "ids": [54556, 220, 16, 17, 18, 198], "decoded": "ASCII 123\n"}, {"text": "́a á ́ १́b", "normalized": "́a á ́ १́b", "plain_ids": [52033, 64, 18135, 220, 52033, 220, 12256, 100, 52033, 65], "ids": [52033, 64, 18135, 220, 52033, 220, 12256, 100, 52033, 65], "decoded": "́a á ́ १́b"}, {"text": "é", "normalized": "é", "plain_ids": [933], "ids": [933], "decoded": "é"}, {"text": "é", "normalized": "é", "plain_ids": [933], "ids": [933], "decoded": "é"}, {"text": "ạ́", "normalized": "ạ́", "plain_ids": [19614, 52033], "ids": [19614, 52033], "decoded": "ạ́"}, {"text": "ạ́", "normalized": "ạ́", "plain_ids": [19614, 52033], "ids": [19614, 52033], "decoded": "ạ́"}, {"text": "̣́a", "normalized": "̣́a", "plain_ids": [136, 96, 52033, 64], "ids": [136, 96, 52033, 64], "decoded": "̣́a"}, {"text": "각", "normalized": "각", "plain_ids": [149252], "ids": [149252], "decoded": "각"}, {"text": "각", "normalized": "각", "plain_ids": [149252], "ids": [149252], "decoded": "각"}, {"text": "① Ⅷ ² 𝟘", "normalized": "① Ⅷ ² 𝟘", "plain_ids": [46701, 254, 220, 68086, 100, 220, 28495, 220, 54362, 253, 246], "ids": [46701, 254, 220, 68086, 100, 220, 28495, 220, 54362, 253, 246], "decoded": "① Ⅷ ² 𝟘"}, {"text": "Å Ω K", "normalized": "Å Ω K", "plain_ids": [169111, 208307, 710], "ids": [169111, 208307, 710], "decoded": "Å Ω K"}, {"text": "x́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́", "normalized": "x́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́", "plain_ids": [87, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033], "ids": [87, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033, 52033], "decoded": "x́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́́"}, {"text": "a͏̣́", "normalized": "a͏̣́", "plain_ids": [64, 137, 237, 136, 96, 52033], "ids": [64, 137, 237, 136, 96, 52033], "decoded": "a͏̣́"}, {"text": "̣𞓬a", "normalized": "̣𞓬a", "plain_ids": [172, 252, 241, 105, 136, 96, 64], "ids": [172, 252, 241, 105, 136, 96, 64], "decoded": "̣𞓬a"}, {"text": "À", "normalized": "À", "plain_ids": [69508], "ids": [69508], "decoded": "À"}, {"text": "Á", "normalized": "Á", "plain_ids": [42254], "ids": [42254], "decoded": "Á"}, {"text": "Â", "normalized": "Â", "plain_ids": [31709], "ids": [31709], "decoded": "Â"}, {"text": "Ã", "normalized": "Ã", "plain_ids": [18224], "ids": [18224], "decoded": "Ã"}, {"text": "Ä", "normalized": "Ä", "plain_ids": [83975], "ids": [83975], "decoded": "Ä"}, {"text": "Å", "normalized": "Å", "plain_ids": [169111], "ids": [169111], "decoded": "Å"}, {"text": "Ç", "normalized": "Ç", "plain_ids": [44382], "ids": [44382], "decoded": "Ç"}, {"text": "È", "normalized": "È", "plain_ids": [168975], "ids": [168975], "decoded": "È"}, {"text": "É", "normalized": "É", "plain_ids": [25933], "ids": [25933], "decoded": "É"}, {"text": "Ê", "normalized": "Ê", "plain_ids": [150003], "ids": [150003], "decoded": "Ê"}, {"text": "Ë", "normalized": "Ë", "plain_ids": [127, 233], "ids": [127, 233], "decoded": "Ë"}, {"text": "Ì", "normalized": "Ì", "plain_ids": [152116], "ids": [152116], "decoded": "Ì"}, {"text": "Í", "normalized": "Í", "plain_ids": [50467], "ids": [50467], "decoded": "Í"}, {"text": "Î", "normalized": "Î", "plain_ids": [69368], "ids": [69368], "decoded": "Î"}, {"text": "Ï", "normalized": "Ï", "plain_ids": [127, 237], "ids": [127, 237], "decoded": "Ï"}, {"text": "Ñ", "normalized": "Ñ", "plain_ids": [68933], "ids": [68933], "decoded": "Ñ"}, {"text": "Ò", "normalized": "Ò", "plain_ids": [154990], "ids": [154990], "decoded": "Ò"}, {"text": "Ó", "normalized": "Ó", "plain_ids": [37769], "ids": [37769], "decoded": "Ó"}, {"text": "Ô", "normalized": "Ô", "plain_ids": [150209], "ids": [1Line truncated

docs/issue116-nfc/verification.json

@@ -0,0 +1,14 @@
+{
+  "base_commit": "5129c450ee6232f6752660c42bc9bdec39bff0ff",
+  "binary_sha256": "eb1a190a1c13f5cacfffbffd4edf9d625595cf7df07092a4b522f07718a4b8ce",
+  "date": "2026-10-04",
+  "platform": "Apple M5 Max 128 GiB",
+  "reference": "9882703f3105363ddc37eca9f97aa09a1d387112",
+  "normalizer_crate": "unicode-normalization-alignments 0.1.12",
+  "files": {
+    "Cargo.toml": "717e05ec41ed88d5be75e533be4c72f619e0d8c1bc83e92277cd53f78825ebb6",
+    "Cargo.lock": "49e8503559b1dbf69366684f601d4d62774f5300910583d4618ca4ed447009e7",
+    "src/engine.rs": "1ac05a99bb7f5a0902c60119569f99c62b5ae44f3e9434d033fadaf09c5a0cd0",
+    "src/engine/tokenizer.rs": "b8d8e9a561c72c28c383824abad9b87c15cc7c4455b6ac71ecec948abc785ba5"
+  }
+}

src/engine.rs

@@ -352,6 +352,13 @@ impl LoadedModel {
 }
 
 impl ModelRef<'_> {
+    fn family(self) -> ModelFamily {
+        match self {
+            Self::Gguf(model) => model.shape.family,
+            Self::Qwen(_) => ModelFamily::Qwen,
+        }
+    }
+
     fn summary(self) -> ModelSummary {
         match self {
             Self::Gguf(model) => model.summary(),
@@ -440,6 +447,7 @@ fn render_text_prompt(
                 && !history.is_empty()
                 && checkpoint_tag
                     == conversation_tag(
+                        model.family(),
                         &settings.system_prompt,
                         settings.reasoning_mode,
                         history,
@@ -846,8 +854,10 @@ impl Generator {
     ) -> Result<GenerationOutput, String> {
         let result: Result<_, String> = (|| {
             let checkpoint_present = checkpoint.is_file();
+            let family = self.executor.model().family();
             let selected = self.select_checkpoint(checkpoint, |tag| {
                 checkpoint_matches_prefix(
+                    family,
                     tag,
                     &settings.system_prompt,
                     settings.reasoning_mode,
@@ -883,7 +893,12 @@ impl Generator {
             self.save_checkpoint(
                 checkpoint,
                 if prompt_complete {
-                    conversation_tag(&settings.system_prompt, settings.reasoning_mode, &completed)
+                    conversation_tag(
+                        family,
+                        &settings.system_prompt,
+                        settings.reasoning_mode,
+                        &completed,
+                    )
                 } else {
                     [0; 32]
                 },
@@ -898,7 +913,12 @@ impl Generator {
         if let Err(error) = &result {
             let mut paths = vec![checkpoint.to_owned()];
             if let Some(directory) = bootstrap {
-                let key = conversation_key(&settings.system_prompt, settings.reasoning_mode, &[]);
+                let key = conversation_key(
+                    self.executor.model().family(),
+                    &settings.system_prompt,
+                    settings.reasoning_mode,
+                    &[],
+                );
                 let tag = Sha256::digest(&key).into();
                 paths.push(directory.join(format!("{}.bin", kvstore::key_hash(&key))));
                 paths.push(resident_key(directory, tag));
@@ -958,8 +978,12 @@ impl Generator {
                 .split_last()
                 .map_or(messages, |(_, history)| history);
             let store = KvStore::open(directory, settings.kv_cache.budget_bytes)?;
-            let history_key =
+            let history_key = conversation_key(
-                conversation_key(&settings.system_prompt, settings.reasoning_mode, history);
+                self.executor.model().family(),
+                &settings.system_prompt,
+                settings.reasoning_mode,
+                history,
+            );
             let history_tag: [u8; 32] = Sha256::digest(&history_key).into();
             let previous_checkpoint =
                 self.restore_cached_prefix(directory, &store, &history_key, history_tag)?;
@@ -978,8 +1002,12 @@ impl Generator {
                 .and_then(|path| std::fs::metadata(path).ok())
                 .map(|item| item.len());
             if prompt_complete {
-                let completed_key =
+                let completed_key = conversation_key(
-                    conversation_key(&settings.system_prompt, settings.reasoning_mode, &completed);
+                    self.executor.model().family(),
+                    &settings.system_prompt,
+                    settings.reasoning_mode,
+                    &completed,
+                );
                 let completed_tag: [u8; 32] = Sha256::digest(&completed_key).into();
                 if !settings.kv_cache.stores(
                     self.executor.position(),
@@ -1030,6 +1058,7 @@ impl Generator {
             let mut paths = Vec::new();
             for count in 0..=messages.len() {
                 let key = conversation_key(
+                    self.executor.model().family(),
                     &settings.system_prompt,
                     settings.reasoning_mode,
                     &messages[..count],
@@ -1097,7 +1126,12 @@ impl Generator {
         progress: &mut impl FnMut(u32, u32, Option<f32>),
         phase: &mut impl FnMut(&'static str),
     ) -> Result<(), String> {
-        let key = conversation_key(&settings.system_prompt, settings.reasoning_mode, &[]);
+        let key = conversation_key(
+            self.executor.model().family(),
+            &settings.system_prompt,
+            settings.reasoning_mode,
+            &[],
+        );
         let tag: [u8; 32] = Sha256::digest(&key).into();
         let store = KvStore::open(directory, settings.kv_cache.budget_bytes)?;
         self.restore_cached_prefix(directory, &store, &key, tag)?;
@@ -1295,7 +1329,12 @@ impl Generator {
             if completed != history_tokens.len() || cancelled.load(Ordering::Relaxed) {
                 return Err("context compaction interrupted during rebuild".into());
             }
-            let tag = conversation_tag(&rebuilt_system, settings.reasoning_mode, &tail);
+            let tag = conversation_tag(
+                self.executor.model().family(),
+                &rebuilt_system,
+                settings.reasoning_mode,
+                &tail,
+            );
             phase("Saving compacted context…");
             self.save_checkpoint(checkpoint, tag)?;
             Ok(CompactionOutput {
@@ -2053,11 +2092,23 @@ fn emit_safe_text(
 }
 
 #[cfg(any(target_os = "macos", test))]
-fn conversation_key(system: &str, reasoning: ReasoningMode, messages: &[ChatTurn]) -> Vec<u8> {
+fn conversation_key(
-    fn text(output: &mut Vec<u8>, value: &str) {
+    family: ModelFamily,
+    system: &str,
+    reasoning: ReasoningMode,
+    messages: &[ChatTurn],
+) -> Vec<u8> {
+    // Keep the original byte keys for both DS4 families and unaffected Qwen
+    // text. A field changed by NFC must not authenticate a pre-NFC checkpoint
+    // through the completed-history fast path. Per-field markers preserve the
+    // prefix property when appending turns, and keep distinct raw histories.
+    let text = |output: &mut Vec<u8>, value: &str| {
+        if family == ModelFamily::Qwen && tokenizer::qwen_normalization_changes(value) {
+            output.extend_from_slice(b"\0qwen-nfc");
+        }
         output.extend_from_slice(&(value.len() as u64).to_le_bytes());
         output.extend_from_slice(value.as_bytes());
-    }
+    };
 
     let mut output = b"DS4Server chat checkpoint v5".to_vec();
     text(&mut output, system);
@@ -2088,12 +2139,18 @@ fn conversation_key(system: &str, reasoning: ReasoningMode, messages: &[ChatTurn
 }
 
 #[cfg(any(target_os = "macos", test))]
-fn conversation_tag(system: &str, reasoning: ReasoningMode, messages: &[ChatTurn]) -> [u8; 32] {
+fn conversation_tag(
-    Sha256::digest(conversation_key(system, reasoning, messages)).into()
+    family: ModelFamily,
+    system: &str,
+    reasoning: ReasoningMode,
+    messages: &[ChatTurn],
+) -> [u8; 32] {
+    Sha256::digest(conversation_key(family, system, reasoning, messages)).into()
 }
 
 #[cfg(any(target_os = "macos", test))]
 fn checkpoint_matches_prefix(
+    family: ModelFamily,
     checkpoint: [u8; 32],
     system: &str,
     reasoning: ReasoningMode,
@@ -2103,7 +2160,7 @@ fn checkpoint_matches_prefix(
     // scanning a genuinely changed, very long history becomes measurable.
     (0..messages.len())
         .rev()
-        .any(|end| checkpoint == conversation_tag(system, reasoning, &messages[..end]))
+        .any(|end| checkpoint == conversation_tag(family, system, reasoning, &messages[..end]))
 }
 
 #[cfg(target_os = "macos")]
@@ -2921,31 +2978,66 @@ mod sampling_tests {
             reasoning_complete: true,
             content: "Hello".into(),
         }];
-        let tag = conversation_tag("System", ReasoningMode::High, &messages);
+        let tag = conversation_tag(
+            ModelFamily::DeepSeek,
+            "System",
+            ReasoningMode::High,
+            &messages,
+        );
         assert_eq!(
             tag,
-            conversation_tag("System", ReasoningMode::High, &messages)
+            conversation_tag(
+                ModelFamily::DeepSeek,
+                "System",
+                ReasoningMode::High,
+                &messages
+            )
         );
         assert_ne!(
             tag,
-            conversation_tag("Changed", ReasoningMode::High, &messages)
+            conversation_tag(
+                ModelFamily::DeepSeek,
+                "Changed",
+                ReasoningMode::High,
+                &messages
+            )
         );
         assert_ne!(
             tag,
-            conversation_tag("System", ReasoningMode::Direct, &messages)
+            conversation_tag(
+                ModelFamily::DeepSeek,
+                "System",
+                ReasoningMode::Direct,
+                &messages
+            )
         );
         messages[0].system = true;
         assert_ne!(
             tag,
-            conversation_tag("System", ReasoningMode::High, &messages)
+            conversation_tag(
+                ModelFamily::DeepSeek,
+                "System",
+                ReasoningMode::High,
+                &messages
+            )
         );
         messages[0].system = false;
         messages[0].skip_previous_eos = true;
         assert_ne!(
             tag,
-            conversation_tag("System", ReasoningMode::High, &messages)
+            conversation_tag(
+                ModelFamily::DeepSeek,
+                "System",
+                ReasoningMode::High,
+                &messages
+            )
+        );
+        let prefix = conversation_key(
+            ModelFamily::DeepSeek,
+            "System",
+            ReasoningMode::High,
+            &messages,
         );
-        let prefix = conversation_key("System", ReasoningMode::High, &messages);
         messages.push(ChatTurn {
             user: false,
             tool: false,
@@ -2955,7 +3047,15 @@ mod sampling_tests {
             reasoning_complete: true,
             content: "Hi".into(),
         });
-        assert!(conversation_key("System", ReasoningMode::High, &messages).starts_with(&prefix));
+        assert!(
+            conversation_key(
+                ModelFamily::DeepSeek,
+                "System",
+                ReasoningMode::High,
+                &messages
+            )
+            .starts_with(&prefix)
+        );
     }
 
     #[test]
@@ -2969,7 +3069,12 @@ mod sampling_tests {
             reasoning_complete: true,
             content: "Question".into(),
         }];
-        let checkpoint = conversation_tag("System", ReasoningMode::High, &messages);
+        let checkpoint = conversation_tag(
+            ModelFamily::DeepSeek,
+            "System",
+            ReasoningMode::High,
+            &messages,
+        );
         messages.extend([
             ChatTurn {
                 user: false,
@@ -2992,12 +3097,14 @@ mod sampling_tests {
         ]);
 
         assert!(checkpoint_matches_prefix(
+            ModelFamily::DeepSeek,
             checkpoint,
             "System",
             ReasoningMode::High,
             &messages,
         ));
         assert!(!checkpoint_matches_prefix(
+            ModelFamily::DeepSeek,
             checkpoint,
             "Changed",
             ReasoningMode::High,
@@ -3008,7 +3115,7 @@ mod sampling_tests {
     #[test]
     fn workspace_instruction_identity_participates_in_append_only_kv_keys() {
         let system = "System";
-        let bootstrap = conversation_key(system, ReasoningMode::High, &[]);
+        let bootstrap = conversation_key(ModelFamily::DeepSeek, system, ReasoningMode::High, &[]);
         let mut messages = vec![
             ChatTurn {
                 user: false,
@@ -3030,16 +3137,39 @@ mod sampling_tests {
             },
         ];
 
-        assert!(conversation_key(system, ReasoningMode::High, &messages).starts_with(&bootstrap));
+        assert!(
+            conversation_key(
+                ModelFamily::DeepSeek,
+                system,
+                ReasoningMode::High,
+                &messages
+            )
+            .starts_with(&bootstrap)
+        );
         let mut changed_opening = messages[0].clone();
         changed_opening.content = changed_opening
             .content
             .replace("identity: abc", "identity: def");
         assert_ne!(
-            conversation_tag(system, ReasoningMode::High, &messages[..1]),
+            conversation_tag(
-            conversation_tag(system, ReasoningMode::High, &[changed_opening])
+                ModelFamily::DeepSeek,
+                system,
+                ReasoningMode::High,
+                &messages[..1]
+            ),
+            conversation_tag(
+                ModelFamily::DeepSeek,
+                system,
+                ReasoningMode::High,
+                &[changed_opening]
+            )
+        );
+        let baseline = conversation_key(
+            ModelFamily::DeepSeek,
+            system,
+            ReasoningMode::High,
+            &messages,
         );
-        let baseline = conversation_key(system, ReasoningMode::High, &messages);
         messages.push(ChatTurn {
             user: false,
             tool: false,
@@ -3049,7 +3179,12 @@ mod sampling_tests {
             reasoning_complete: true,
             content: "<system-reminder>\nWorkspace instruction identity: def\nReplacement instructions from: /project/AGENTS.md\nkeep that\n</system-reminder>".into(),
         });
-        let changed = conversation_key(system, ReasoningMode::High, &messages);
+        let changed = conversation_key(
+            ModelFamily::DeepSeek,
+            system,
+            ReasoningMode::High,
+            &messages,
+        );
         assert!(changed.starts_with(&baseline));
     }
 

src/engine/metal/qwen_mtplx/issue105_product.rs

@@ -105,8 +105,18 @@ fn product(mtp: bool) {
     let mut messages = vec![message("Count from one.")];
     let baseline = generate(&mut generator, &baseline_path, &messages, &settings);
     messages.push(baseline.message);
-    let tag = conversation_tag(&settings.system_prompt, settings.reasoning_mode, &messages);
+    let tag = conversation_tag(
-    let key = conversation_key(&settings.system_prompt, settings.reasoning_mode, &messages);
+        crate::engine::ModelFamily::Qwen,
+        &settings.system_prompt,
+        settings.reasoning_mode,
+        &messages,
+    );
+    let key = conversation_key(
+        crate::engine::ModelFamily::Qwen,
+        &settings.system_prompt,
+        settings.reasoning_mode,
+        &messages,
+    );
     messages.push(message("Continue."));
     let healthy_path = directory.join("healthy.bin");
     generate(

src/engine/metal/qwen_mtplx/issue116_product.rs

@@ -0,0 +1,146 @@
+//! Actual live and disk cache migration through the whole-turn request path.
+use super::*;
+use crate::engine::{ModelFamily, conversation_key, conversation_tag};
+use crate::settings::{
+    GenerationPreferences, ReasoningMode, RuntimePreferences, SpeculativePreferences,
+    effective_settings,
+};
+use sha2::{Digest, Sha256};
+use std::path::PathBuf;
+
+#[test]
+#[ignore = "installed model; serial external memory supervisor required"]
+fn issue116_normalized_token_cache() {
+    let root = PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
+    let oracle = PathBuf::from(std::env::var_os("DS4_ISSUE116_ORACLE").unwrap());
+    let directory = PathBuf::from(std::env::var_os("DS4_ISSUE116_DIR").unwrap());
+    std::fs::create_dir_all(&directory).unwrap();
+    let requests: serde_json::Value =
+        serde_json::from_slice(&std::fs::read(oracle.join("requests.json")).unwrap()).unwrap();
+    let case = &requests[3]; // Low, with a decomposed system prompt and history.
+    let ids: Vec<i32> = serde_json::from_value(case["ids"].clone()).unwrap();
+    let legacy: Vec<i32> = serde_json::from_value(case["legacy_history_ids"].clone()).unwrap();
+    let settings = effective_settings(
+        crate::model::ModelChoice::Qwen38FlashNext,
+        &GenerationPreferences {
+            context_tokens: 8192,
+            max_generated_tokens: 2,
+            system_prompt: case["system"].as_str().unwrap().into(),
+            temperature: Some(0.),
+            top_p: Some(0.95),
+            min_p: Some(0.),
+            seed: Some(1),
+            reasoning_mode: ReasoningMode::Low,
+        },
+        &RuntimePreferences {
+            speculative: SpeculativePreferences {
+                glm_mtp: true,
+                ..Default::default()
+            },
+            ..Default::default()
+        },
+        root.parent().unwrap(),
+    )
+    .unwrap();
+    let messages = case["messages"]
+        .as_array()
+        .unwrap()
+        .iter()
+        .map(|m| ChatTurn {
+            user: m["role"] == "user",
+            tool: false,
+            system: false,
+            skip_previous_eos: false,
+            reasoning: m["reasoning_content"].as_str().map(str::to_owned),
+            reasoning_complete: true,
+            content: m["content"].as_str().unwrap().into(),
+        })
+        .collect::<Vec<_>>();
+    let mut owner = QwenExecutor::open(&settings.engine).unwrap();
+    owner.policy.store_min_suffix = Some(0);
+    assert_eq!(
+        owner.metadata.tokenizer.encode_conversation(
+            &settings.turn.system_prompt,
+            &messages,
+            ReasoningMode::Low
+        ),
+        ids
+    );
+    let old_tag: [u8; 32] = Sha256::digest(conversation_key(
+        ModelFamily::DeepSeek,
+        &settings.turn.system_prompt,
+        ReasoningMode::Low,
+        &messages[..2],
+    ))
+    .into();
+    let new_tag = conversation_tag(
+        ModelFamily::Qwen,
+        &settings.turn.system_prompt,
+        ReasoningMode::Low,
+        &messages[..2],
+    );
+    assert_ne!(old_tag, new_tag);
+    owner.prefill(&legacy, |_| true).unwrap();
+    owner.note_checkpoint_tag(old_tag);
+    owner
+        .save_checkpoint(&directory.join("old.bin"), old_tag, &mut |_| {})
+        .unwrap();
+    let metrics = Metrics::new(&directory.join("metrics"));
+    let run = |owner: &mut QwenExecutor, messages: &[ChatTurn]| {
+        let (out, complete) = owner
+            .generate(
+                messages,
+                &settings.turn,
+                &AtomicBool::new(false),
+                &metrics,
+                |_, _, _| {},
+                |_, _| {},
+            )
+            .unwrap();
+        assert!(complete);
+        out
+    };
+    // Directly exercise the matching raw-history tag shortcut: old live/disk
+    // histories must miss, even though their source messages are unchanged.
+    let cold = run(&mut owner, &messages);
+    assert_eq!(cold.cached_tokens, 0);
+    assert_eq!(cold.prompt_tokens as usize, ids.len());
+    assert!(owner.tokens().starts_with(&ids));
+    let cold_state = super::issue107_product::state(&owner);
+    owner.reset().unwrap();
+    assert!(
+        owner
+            .load_checkpoint(&directory.join("old.bin"), &mut |_| {})
+            .unwrap()
+    );
+    assert_eq!(owner.tokens(), legacy);
+    let disk_cold = run(&mut owner, &messages);
+    assert_eq!(disk_cold.cached_tokens, 0);
+    assert_eq!(cold_state, super::issue107_product::state(&owner));
+    let tag = owner.checkpoint_tag();
+    owner
+        .save_checkpoint(&directory.join("new.bin"), tag, &mut |_| {})
+        .unwrap();
+    let mut next = messages.clone();
+    next.push(disk_cold.message);
+    next.push(ChatTurn {
+        content: "Continue e\u{301} and 가.".into(),
+        ..messages[0].clone()
+    });
+    let warm = run(&mut owner, &next);
+    assert!(warm.cached_tokens >= cold.prompt_tokens);
+    let expected = super::issue107_product::state(&owner);
+    owner.reset().unwrap();
+    assert!(
+        owner
+            .load_checkpoint(&directory.join("new.bin"), &mut |_| {})
+            .unwrap()
+    );
+    let reopened = run(&mut owner, &next);
+    assert_eq!(warm.cached_tokens, reopened.cached_tokens);
+    assert_eq!(expected, super::issue107_product::state(&owner));
+    println!(
+        "{}",
+        serde_json::json!({"case":"nfc_cache", "legacy_history_tokens":legacy.len(),"normalized_prompt_tokens":ids.len(),"live_old_miss":true,"disk_old_miss":true,"cold_state_exact":true,"warm_cached":warm.cached_tokens,"disk_cached":reopened.cached_tokens,"warm_disk_state_exact":true})
+    );
+}

src/engine/metal/qwen_mtplx/owner.rs

@@ -1031,7 +1031,12 @@ fn qwen_vision_chat_checkpoint_preserves_image_identity() {
     assert_eq!(repeat.cached_tokens, first.prompt_tokens);
     assert_eq!(repeat.message.content, first.message.content);
     messages.push(repeat.message);
-    let tag = crate::engine::conversation_tag("", ReasoningMode::Low, &messages);
+    let tag = crate::engine::conversation_tag(
+        crate::engine::ModelFamily::Qwen,
+        "",
+        ReasoningMode::Low,
+        &messages,
+    );
     let checkpoint = directory.join("vision.kv");
     executor
         .save_checkpoint(&checkpoint, tag, &mut |_| {})
@@ -1081,3 +1086,7 @@ mod issue109_product;
 #[cfg(test)]
 #[path = "issue111_product.rs"]
 mod issue111_product;
+
+#[cfg(test)]
+#[path = "issue116_product.rs"]
+mod issue116_product;

src/engine/metal/qwen_mtplx/request.rs

@@ -220,8 +220,12 @@ impl LiveSession {
         {
             let mut completed = messages.to_vec();
             completed.push(result.output.message.clone());
-            self.tag =
+            self.tag = conversation_tag(
-                conversation_tag(&settings.system_prompt, settings.reasoning_mode, &completed);
+                crate::engine::ModelFamily::Qwen,
+                &settings.system_prompt,
+                settings.reasoning_mode,
+                &completed,
+            );
             self.current = Some(snapshot);
             self.current_images = images;
         }

src/engine/metal/qwen_mtplx/session_cache.rs

@@ -537,6 +537,7 @@ fn mtplx_full_installed_chat_matches_reference() {
                     }),
                 };
                 let key = crate::engine::conversation_key(
+                    crate::engine::ModelFamily::Qwen,
                     &settings.system_prompt,
                     settings.reasoning_mode,
                     &messages,

src/engine/qwen.rs

@@ -1280,7 +1280,12 @@ mod tests {
                 let ending = metadata.tokenizer.tokenize_rendered("<|im_end|>\n");
                 assert_eq!(ending, [metadata.tokenizer.eos(), 198]);
                 assert!(cached.ends_with(&ending));
-                let tag = super::super::conversation_tag("system", mode, &history);
+                let tag = super::super::conversation_tag(
+                    super::super::ModelFamily::Qwen,
+                    "system",
+                    mode,
+                    &history,
+                );
                 // Full template, MTPLX EOS frontier and scalar pre-EOS
                 // frontier must all append the same complete turn boundary.
                 for missing in 0..=ending.len() {

src/engine/tokenizer.rs

@@ -8,6 +8,7 @@ use serde::Deserialize;
 use std::collections::HashMap;
 use std::fs;
 use std::path::Path;
+use unicode_normalization_alignments::UnicodeNormalization;
 
 const HIGH_REASONING_PREFIX: &str = "Reasoning Effort: Absolute maximum with no shortcuts permitted.\n\
 You MUST be very thorough in your thinking and comprehensively decompose the problem to resolve the root cause, rigorously stress-testing your logic against all potential paths, edge cases, and adversarial scenarios.\n\
@@ -54,10 +55,18 @@ pub(super) struct Tokenizer {
 
 #[derive(Deserialize)]
 struct QwenTokenizerFile {
+    normalizer: QwenNormalizer,
     added_tokens: Vec<QwenAddedToken>,
     model: QwenBpe,
 }
 
+#[derive(Deserialize)]
+#[serde(tag = "type")]
+enum QwenNormalizer {
+    #[serde(rename = "NFC")]
+    Nfc,
+}
+
 #[derive(Deserialize)]
 struct QwenAddedToken {
     id: usize,
@@ -190,6 +199,7 @@ impl Tokenizer {
             &fs::read(path).map_err(|error| format!("{}: {error}", path.display()))?,
         )
         .map_err(|error| format!("{}: {error}", path.display()))?;
+        let QwenNormalizer::Nfc = file.normalizer;
         if file.model.kind != "BPE" || file.model.byte_fallback {
             return Err("Qwen tokenizer must use BPE without byte fallback".into());
         }
@@ -772,7 +782,10 @@ impl Tokenizer {
     }
 
     fn tokenize_qwen(&self, text: &str, output: &mut Vec<i32>) {
-        for piece in qwen_pieces(text) {
+        // Added tokens are extracted first by tokenize_rendered. Normalize each
+        // remaining span before the trained split and BPE, as in tokenizers.
+        let normalized = qwen_normalize(text);
+        for piece in qwen_pieces(&normalized) {
             self.emit_piece(piece.as_bytes(), output);
         }
     }
@@ -867,6 +880,21 @@ impl Tokenizer {
     }
 }
 
+fn qwen_normalize(text: &str) -> std::borrow::Cow<'_, str> {
+    if text.is_ascii() {
+        text.into()
+    } else {
+        text.nfc()
+            .map(|(character, _)| character)
+            .collect::<String>()
+            .into()
+    }
+}
+
+pub(super) fn qwen_normalization_changes(text: &str) -> bool {
+    qwen_normalize(text) != text
+}
+
 /// The trained Qwen3 Split expression. Rust regex supports its Unicode
 /// categories but not the penultimate `\s+(?!\S)` alternative: before a
 /// non-whitespace character that branch backtracks by one whitespace scalar,
@@ -1214,7 +1242,7 @@ mod tests {
                 let rendered = render_text_prompt(
                     ModelRef::Gguf(&model),
                     &bootstrap,
-                    conversation_tag("", settings.reasoning_mode, &[]),
+                    conversation_tag(model.shape.family, "", settings.reasoning_mode, &[]),
                     &messages,
                     &settings,
                 );
@@ -1386,3 +1414,7 @@ mod tests {
 #[cfg(test)]
 #[path = "tokenizer_issue111.rs"]
 mod issue111;
+
+#[cfg(test)]
+#[path = "tokenizer_issue116.rs"]
+mod issue116;

src/engine/tokenizer_issue111.rs

@@ -100,13 +100,22 @@ fn issue111_qwen_tokens_and_requests() {
             case["pieces"],
             "split {text:?}"
         );
+        let mut component = Vec::new();
+        for piece in qwen_pieces(text) {
+            tokenizer.emit_piece(piece.as_bytes(), &mut component);
+        }
+        assert_eq!(
+            serde_json::json!(component),
+            case["ids"],
+            "component {text:?}"
+        );
         for ids in [tokenizer.tokenize(text), tokenizer.tokenize_rendered(text)] {
-            assert_eq!(serde_json::json!(ids), case["ids"], "{text:?}");
+            assert_eq!(serde_json::json!(ids), case["full_ids"], "{text:?}");
             let decoded = ids
                 .iter()
                 .flat_map(|&id| tokenizer.token_bytes(id).unwrap())
                 .collect::<Vec<_>>();
-            assert_eq!(decoded, text.as_bytes());
+            assert_eq!(decoded, qwen_normalize(text).as_bytes());
         }
     }
     let requests: serde_json::Value =

src/engine/tokenizer_issue116.rs

@@ -0,0 +1,187 @@
+use super::*;
+use sha2::{Digest, Sha256};
+
+#[test]
+fn nfc_contract_and_composition() {
+    assert!(serde_json::from_str::<QwenNormalizer>(r#"{"type":"NFC"}"#).is_ok());
+    for unsupported in [
+        "null",
+        r#"{"type":"NFD"}"#,
+        r#"{"type":"NFKC"}"#,
+        r#"{"type":"Sequence","normalizers":[]}"#,
+    ] {
+        assert!(serde_json::from_str::<QwenNormalizer>(unsupported).is_err());
+    }
+    for (text, expected) in [
+        ("e\u{301}", "é"),
+        ("a\u{301}\u{323}", "ạ\u{301}"),
+        ("각", "각"),
+        ("① Ⅷ ² 𝟘", "① Ⅷ ² 𝟘"),
+        ("ASCII", "ASCII"),
+    ] {
+        assert_eq!(qwen_normalize(text), expected);
+        assert_eq!(qwen_normalization_changes(text), text != expected);
+    }
+}
+
+#[test]
+fn only_affected_qwen_history_changes_checkpoint_tags() {
+    use crate::engine::{checkpoint_matches_prefix, conversation_key, conversation_tag};
+    let message = ChatTurn {
+        user: true,
+        tool: false,
+        system: false,
+        skip_previous_eos: false,
+        reasoning: None,
+        reasoning_complete: true,
+        content: "é".into(),
+    };
+    for (system, content, reasoning) in [
+        ("", "é", None),
+        ("e\u{301}", "é", None),
+        ("", "e\u{301}", None),
+        ("", "é", Some("e\u{301}")),
+    ] {
+        let history = [ChatTurn {
+            content: content.into(),
+            reasoning: reasoning.map(str::to_owned),
+            ..message.clone()
+        }];
+        let old: [u8; 32] = Sha256::digest(conversation_key(
+            ModelFamily::DeepSeek,
+            system,
+            ReasoningMode::Low,
+            &history,
+        ))
+        .into();
+        let new = conversation_tag(ModelFamily::Qwen, system, ReasoningMode::Low, &history);
+        let changed = !system.is_empty() || content != "é" || reasoning.is_some();
+        assert_eq!(old != new, changed);
+        for family in [ModelFamily::DeepSeek, ModelFamily::Glm] {
+            assert_eq!(
+                conversation_tag(family, system, ReasoningMode::Low, &history),
+                old
+            );
+        }
+        let next = [history[0].clone(), message.clone()];
+        let next_key = conversation_key(ModelFamily::Qwen, system, ReasoningMode::Low, &next);
+        assert!(next_key.starts_with(&conversation_key(
+            ModelFamily::Qwen,
+            system,
+            ReasoningMode::Low,
+            &history
+        )));
+        assert_eq!(
+            next_key.starts_with(&conversation_key(
+                ModelFamily::DeepSeek,
+                system,
+                ReasoningMode::Low,
+                &history
+            )),
+            !changed
+        );
+        assert!(checkpoint_matches_prefix(
+            ModelFamily::Qwen,
+            new,
+            system,
+            ReasoningMode::Low,
+            &next
+        ));
+        assert_eq!(
+            checkpoint_matches_prefix(ModelFamily::Qwen, old, system, ReasoningMode::Low, &next),
+            !changed
+        );
+    }
+}
+
+#[test]
+#[ignore = "CPU-only; pinned NFC receipts and installed Qwen tokenizer"]
+fn issue116_nfc_tokens_and_requests() {
+    let root = std::path::PathBuf::from(std::env::var_os("DS4_QWEN38_ARTIFACTS").unwrap());
+    let oracle = std::path::PathBuf::from(std::env::var_os("DS4_ISSUE116_ORACLE").unwrap());
+    let read = |name: &str| -> serde_json::Value {
+        serde_json::from_slice(&fs::read(oracle.join(name)).unwrap()).unwrap()
+    };
+    let tokenizer = Tokenizer::load_qwen(&root.join("tokenizer.json")).unwrap();
+    let scalar = read("scalar-nfc.json");
+    let text: String = (0..=0x10ffff)
+        .filter_map(char::from_u32)
+        .flat_map(|c| ['a', c, '\u{301}', '\u{323}', '|'])
+        .collect();
+    let normalized = qwen_normalize(&text);
+    assert_eq!(normalized.len() as u64, scalar["bytes"].as_u64().unwrap());
+    assert_eq!(
+        Sha256::digest(normalized.as_bytes())
+            .iter()
+            .map(|b| format!("{b:02x}"))
+            .collect::<String>(),
+        scalar["sha256"].as_str().unwrap()
+    );
+    let cases = read("tokens.json");
+    for case in cases.as_array().unwrap() {
+        let text = case["text"].as_str().unwrap();
+        assert_eq!(
+            qwen_normalize(text),
+            case["normalized"].as_str().unwrap(),
+            "{text:?}"
+        );
+        assert_eq!(
+            serde_json::json!(tokenizer.tokenize(text)),
+            case["plain_ids"],
+            "plain {text:?}"
+        );
+        let ids = tokenizer.tokenize_rendered(text);
+        assert_eq!(serde_json::json!(ids), case["ids"], "rendered {text:?}");
+        let decoded = ids
+            .iter()
+            .flat_map(|&id| tokenizer.token_bytes(id).unwrap())
+            .collect::<Vec<_>>();
+        assert_eq!(
+            decoded,
+            case["decoded"].as_str().unwrap().as_bytes(),
+            "decode {text:?}"
+        );
+    }
+    for case in read("requests.json").as_array().unwrap() {
+        let effort = match case["effort"].as_str().unwrap() {
+            "direct" => ReasoningMode::Direct,
+            "low" => ReasoningMode::Low,
+            "medium" => ReasoningMode::Medium,
+            "xhigh" => ReasoningMode::XHigh,
+            _ => unreachable!(),
+        };
+        let system = case["system"].as_str().unwrap();
+        let messages = case["messages"]
+            .as_array()
+            .unwrap()
+            .iter()
+            .map(|m| ChatTurn {
+                user: m["role"] == "user",
+                tool: false,
+                system: false,
+                skip_previous_eos: false,
+                reasoning: m["reasoning_content"].as_str().map(str::to_owned),
+                reasoning_complete: true,
+                content: m["content"].as_str().unwrap().into(),
+            })
+            .collect::<Vec<_>>();
+        let ids = tokenizer.encode_conversation(system, &messages, effort);
+        assert_eq!(
+            serde_json::json!(ids),
+            case["ids"],
+            "{} {system:?}",
+            case["effort"]
+        );
+        let mut resumed = tokenizer.encode_history(system, &messages[..2], effort);
+        resumed.extend(tokenizer.encode_continuation(&messages[2].content, effort, false));
+        assert_eq!(resumed, ids);
+        println!(
+            "{}",
+            serde_json::json!({"effort":case["effort"],"system":system,"context_tokens":ids.len(),"exact":true})
+        );
+    }
+    println!(
+        "{}",
+        serde_json::json!({"case":"nfc", "scalar_sha256":scalar["sha256"],"token_cases":cases.as_array().unwrap().len(),"requests":8,"exact":true})
+    );
+}

tools/issue116-reference.py

@@ -0,0 +1,44 @@
+"""Independent NFC, full tokenizer and request receipts from pinned MTPLX."""
+import hashlib, json, os, subprocess, sys, unicodedata
+from pathlib import Path
+checkout, root, out = map(Path, sys.argv[1:])
+assert subprocess.check_output(['git','-C',str(checkout),'rev-parse','HEAD'],text=True).strip() == '9882703f3105363ddc37eca9f97aa09a1d387112'
+os.environ.update(HF_HUB_OFFLINE='1', TRANSFORMERS_OFFLINE='1')
+sys.path.insert(0, str(checkout))
+from mtplx.runtime import _load_tokenizer_resilient
+from mtplx.server import openai as server
+from tokenizers import Tokenizer
+import tokenizers
+out.mkdir(parents=True, exist_ok=True)
+tok = _load_tokenizer_resilient(root, json.loads((root/'config.json').read_text()))
+normalizer = tok.backend_tokenizer.normalizer
+assert json.loads((root/'tokenizer.json').read_text())['normalizer'] == {'type':'NFC'}
+backend = json.loads(tok.backend_tokenizer.to_str())
+backend['added_tokens'] = []
+plain = Tokenizer.from_str(json.dumps(backend))
+legacy = Tokenizer.from_str(tok.backend_tokenizer.to_str()); legacy.normalizer = None
+text = ''.join('a'+chr(i)+'\u0301\u0323|' for i in range(0x110000) if not 0xd800 <= i <= 0xdfff)
+normalized = normalizer.normalize_str(text)
+(out/'scalar-nfc.json').write_text(json.dumps(dict(scalars=1112064,bytes=len(normalized.encode()),sha256=hashlib.sha256(normalized.encode()).hexdigest(),tokenizers=tokenizers.__version__)))
+texts = ['', 'ASCII 123\n', '́a á ́ १́b', 'e\u0301', 'é', 'a\u0301\u0323', 'a\u0323\u0301', '\u0301\u0323a', '각', '각', '① Ⅷ ² 𝟘', 'Å Ω K', 'x'+ '\u0301'*64, 'a\u034f\u0323\u0301', '\U0001e4ec\u0323a']
+texts += [unicodedata.normalize('NFD', chr(i)) for i in range(0x110000) if not 0xd800 <= i <= 0xdfff and unicodedata.decomposition(chr(i)) and not unicodedata.decomposition(chr(i)).startswith('<')]
+texts += [unicodedata.normalize('NFD',chr(i)) for i in range(0xac00,0xd7a4)]
+texts += ['e\u0301'+t['content']+'\u0301a\u0323\u0301' for t in json.loads((root/'tokenizer.json').read_text())['added_tokens']]
+records=[]
+for text in texts:
+ ids=tok.encode(text,add_special_tokens=False)
+ records.append(dict(text=text, normalized=normalizer.normalize_str(text), plain_ids=plain.encode(text,add_special_tokens=False).ids, ids=ids, decoded=tok.decode(ids,skip_special_tokens=False,clean_up_tokenization_spaces=False)))
+(out/'tokens.json').write_text(json.dumps(records,ensure_ascii=False))
+requests=[]
+for effort in ['direct','low','medium','xhigh']:
+ for system in ['', 'Cafe\u0301 각.']:
+  messages=[dict(role='user',content=' ́a á ́ १́b 각\n '),dict(role='assistant',content='Cafe\u0301 ①.',reasoning_content='' if effort=='direct' else 'a\u0301\u0323 thought.'),dict(role='user',content='Continue e\u0301 and 가.\n   x')]
+  full=([dict(role='system',content=system)] if system else [])+messages
+  ids=server._encode_messages_uncached(tok,[server.ChatMessage(**m) for m in full],enable_thinking=effort!='direct',reasoning_effort=None if effort=='direct' else effort,preserve_reasoning_history=True,tools=None)
+  rendered=tok.apply_chat_template(full,tokenize=False,add_generation_prompt=True,enable_thinking=effort!='direct',reasoning_effort=None if effort=='direct' else effort,preserve_reasoning_history=True)
+  # The server uses this template; keep pre-normalization text to build old IDs.
+  assert tok.encode(rendered,add_special_tokens=False)==ids
+  history_rendered=tok.apply_chat_template(full[:-1],tokenize=False,add_generation_prompt=False,enable_thinking=effort!='direct',reasoning_effort=None if effort=='direct' else effort,preserve_reasoning_history=True)
+  requests.append(dict(legacy_history_ids=legacy.encode(history_rendered,add_special_tokens=False).ids,effort=effort,system=system,messages=messages,ids=ids,legacy_ids=legacy.encode(rendered,add_special_tokens=False).ids,decoded=tok.decode(ids,skip_special_tokens=False,clean_up_tokenization_spaces=False)))
+(out/'requests.json').write_text(json.dumps(requests,ensure_ascii=False))
+print(json.dumps(dict(cases=len(records),requests=len(requests),scalar_sha256=hashlib.sha256(normalized.encode()).hexdigest(),tokenizer_sha256=hashlib.sha256((root/'tokenizer.json').read_bytes()).hexdigest(),tokenizers=tokenizers.__version__)),flush=True)