Reject invalid Qwen logits and discard poisoned sessions
This commit is contained in:
Georg Bauer committed
1 parent
1de78fa3ef
commit
5ecf8ab4f3
31 files changed
+1812
-746
No files matched your search
@@ -0,0 +1,95 @@
|
||||
# Issue 105: reject invalid Qwen logits and invalidate the failed session
|
||||
|
||||
Qwen now reports numerical failure instead of emitting fabricated IDs for NaN,
|
||||
positive infinity or all-masked logits. Negative-infinity masks with a finite
|
||||
winner remain legal. Failed sessions lose their live, resident and persisted
|
||||
state, while unrelated healthy conversations remain reusable.
|
||||
|
||||
## Oracle and approved exception
|
||||
|
||||
The oracle is MTPLX `9882703f3105363ddc37eca9f97aa09a1d387112`, MLX 0.32.2.
|
||||
The unchanged installed Bare-Speed model and environment were verified for
|
||||
`../issue104-ar/environment.json` immediately before this issue.
|
||||
|
||||
Ordinary greedy uses the upstream `any(isnan(row)) | isinf(max(row))` graph,
|
||||
evaluated with argmax. Only failures read the full row for the NaN/infinity
|
||||
census. Sampled support retains its existing GPU route and host-softmax fallback;
|
||||
invalid mass now raises the same numerical error instead of selecting token 0.
|
||||
SparseDistribution no longer invents an ID when positive mass is absent.
|
||||
|
||||
The pinned oracle remains inconsistent in specialized greedy MTP paths:
|
||||
`generation.py:7577,13295,15371` still use unchecked argmax. The user explicitly
|
||||
approved extending rejection throughout MTP on 2026-10-04. This deliberate
|
||||
invalid-row exception is not claimed as upstream parity. `oracle.jsonl` records
|
||||
the ordinary error, lazy sentinel and specialized-MTP token 0 side by side.
|
||||
Batched verification validates every row, so a healthy row cannot conceal an
|
||||
all-masked neighbor. Valid choices and RNG draws are unchanged.
|
||||
|
||||
The old lazy sampler is only a historical #104/RNG test diagnostic. Its finite
|
||||
law remains historical and its invalid result is now the checked -1 sentinel.
|
||||
No product path calls it. The unrelated diagnostic prefix sampler retains its
|
||||
upstream contract; it is not a production target/draft sampling route.
|
||||
|
||||
## Ownership and family boundary review
|
||||
|
||||
| Requirement | Implementation and evidence |
|
||||
| --- | --- |
|
||||
| Reject before consuming a fabricated ID | Ordinary AR, MTP primary/target/draft and context-copy target all reach checked sampling; invalid prefill emits no text. |
|
||||
| Clear a prefix captured before failure | LiveSession discards current and prefix; QwenExecutor resets trunk/head/boundary caches. Installed faults occur after real prefill and prefix capture. |
|
||||
| Clear selected, resident and disk state | Generator invalidates the failed conversation's checkpoints, resident aliases, indexes and staging files, and clears selection/store frontier. |
|
||||
| No stale bootstrap or later persistence | First local fault creates then removes a real system bootstrap. A later explicit save fails without a committed snapshot. All failed requests retry cold. |
|
||||
| Preserve other sessions | A real healthy checkpoint remains byte-identical and resident throughout both product matrices, then restores successfully. |
|
||||
| Final capture ownership | A test-only fault consumes an invalid row through the real numerical guard during final capture; the resulting error crosses the actual snapshot/request/Generator owners and prevents persistence. Normal capture adds no sampling pass. |
|
||||
| Streaming and non-streaming | Actual Generator events pass through the production HTTP response adapters; explicit SSE errors/HTTP 500 are checked, followed by valid requests using the same Generator. |
|
||||
| DS4/GLM unchanged | Shared cleanup requires both QwenMtplx and the numerical-error discriminator. DS4 `ds4.c:39410-39460` retains its greedy/no-finite-mass fallback; shared RNG, samplers and checkpoint formats are untouched. Default tests include exact DS4 token/RNG controls. |
|
||||
|
||||
DS4Server uses content-addressed API conversations rather than MTPLX session IDs.
|
||||
Its cleanup therefore invalidates every ancestor of the failed API request and
|
||||
its selected source. Local chats invalidate their file and system bootstrap.
|
||||
Shared bootstrap prefixes belonging to the failed ancestry are invalidated,
|
||||
while other conversations' complete checkpoints remain intact. This is the
|
||||
DS4Server-specific persistence integration of MTPLX's session-bank eviction.
|
||||
|
||||
Checkpoint writes are synchronous on the serialized model worker. Errors return
|
||||
before final persistence; the cleared owner cannot later save or be banked by an
|
||||
outgoing resident swap. Filesystem cleanup failures are reported explicitly.
|
||||
Production remains Rust/Metal; Python is used solely for isolated oracle receipts.
|
||||
|
||||
## Validation
|
||||
|
||||
- Fresh ordinary-entry fixtures: 1,080 cases, three consecutive draws each,
|
||||
including evaluation boundaries, valid tokens/probabilities and next RNG.
|
||||
- Sparse fallback: 150 cases; batched fallback: 653; mixed distributions: 100;
|
||||
historical keyed/lazy diagnostics: 387. Mixed and distribution/RNG tests run
|
||||
in the ordinary cargo suite, not with `--ignored`.
|
||||
- Four synthetic rows at temperatures 0/.6/1, with exact oracle error census,
|
||||
legal tokens and invalid lazy sentinel: `native-invalid-rows.jsonl`.
|
||||
- Batched greedy: both row orders, widths 64/4097, all four rows, including a
|
||||
healthy neighbor beside an all-masked row.
|
||||
- Installed AR: 24 cases (prefill, later forward, final capture; four rows;
|
||||
both HTTP modes), 18 invalid/cold-retry cases and six legal-mask successes.
|
||||
- Installed MTP: 32 cases, additionally exercising draft-head faults; 24
|
||||
invalid/cold-retry cases and eight legal-mask successes. A real supported
|
||||
penalty disables context-copy for draft tests and half the target tests,
|
||||
ensuring both target routes and the actual draft head are exercised.
|
||||
- All four commit gates: formatting, Clippy with warnings denied, signed app
|
||||
bundle, and all-feature cargo tests. Exact outcomes are in `checks.json`.
|
||||
|
||||
These are correctness and cleanup tests, not throughput measurements. Test-only
|
||||
injection fields, output substitution and HTTP harness are absent from release
|
||||
code. The initial test runs deliberately failed if the requested fault boundary
|
||||
was bypassed; the final matrices reach every specified boundary.
|
||||
|
||||
## Reproduction
|
||||
|
||||
Build with `cargo test --all-features --bin ds4-server --no-run` and
|
||||
`cargo build --bin test-supervisor`. Resolve the executable printed by Cargo.
|
||||
Run model-free ignored tests serially under `test-supervisor 4096 60 120
|
||||
--command <test-executable> <test-name> --ignored --nocapture --test-threads=1`.
|
||||
|
||||
For `issue105_product_ar_cleanup_and_http_retry` and
|
||||
`issue105_product_mtp_cleanup_and_http_retry`, use memory limit 114688 MiB,
|
||||
`DS4_QWEN38_ARTIFACTS` pointing to the installed unchanged model, and a fresh
|
||||
`DS4_ISSUE105_DIR` per run. Keep model processes serial. For the oracle, run
|
||||
`tools/issue105-reference.py` in the isolated MLX 0.32.2 environment with
|
||||
`PYTHONPATH` pointing at the pinned MTPLX checkout.
|
||||
@@ -0,0 +1,25 @@
|
||||
{"case":0,"mtp":false,"stage":"Prefill","row":"Nan","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":1,"mtp":false,"stage":"Prefill","row":"Nan","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":2,"mtp":false,"stage":"Prefill","row":"PositiveInf","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":3,"mtp":false,"stage":"Prefill","row":"PositiveInf","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":4,"mtp":false,"stage":"Prefill","row":"Masked","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":5,"mtp":false,"stage":"Prefill","row":"Masked","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":6,"mtp":false,"stage":"Prefill","row":"LegalMask","streaming":false,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":7,"mtp":false,"stage":"Prefill","row":"LegalMask","streaming":true,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":8,"mtp":false,"stage":"Target","row":"Nan","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":9,"mtp":false,"stage":"Target","row":"Nan","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":10,"mtp":false,"stage":"Target","row":"PositiveInf","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":11,"mtp":false,"stage":"Target","row":"PositiveInf","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":12,"mtp":false,"stage":"Target","row":"Masked","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":13,"mtp":false,"stage":"Target","row":"Masked","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":14,"mtp":false,"stage":"Target","row":"LegalMask","streaming":false,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":15,"mtp":false,"stage":"Target","row":"LegalMask","streaming":true,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":16,"mtp":false,"stage":"Capture","row":"Nan","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":17,"mtp":false,"stage":"Capture","row":"Nan","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":18,"mtp":false,"stage":"Capture","row":"PositiveInf","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":19,"mtp":false,"stage":"Capture","row":"PositiveInf","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":20,"mtp":false,"stage":"Capture","row":"Masked","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":21,"mtp":false,"stage":"Capture","row":"Masked","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":22,"mtp":false,"stage":"Capture","row":"LegalMask","streaming":false,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":23,"mtp":false,"stage":"Capture","row":"LegalMask","streaming":true,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"event":"test_resource_summary","elapsed_ms":58704,"peak":{"cpu_time_ns":62056161375,"resident_bytes":25310855168,"physical_bytes":77867608424,"peak_physical_bytes":77867624808,"disk_read_bytes":68731633664,"disk_write_bytes":9359335424},"error":null}
|
||||
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"mtplx_sampler_entry_matches_reference": [
|
||||
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 521 filtered out; finished in 59.33s"
|
||||
],
|
||||
"mtplx_sampler_fallback_matches_reference": [
|
||||
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 520 filtered out; finished in 3.76s"
|
||||
],
|
||||
"mtplx_sampler_resident_matches_reference": [
|
||||
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 520 filtered out; finished in 11.51s"
|
||||
],
|
||||
"mtplx_sampler_batched": [
|
||||
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 520 filtered out; finished in 115.98s"
|
||||
],
|
||||
"product_ar": {
|
||||
"cases": 24,
|
||||
"invalid": 18,
|
||||
"legal_mask": 6,
|
||||
"status": "passed"
|
||||
},
|
||||
"product_mtp": {
|
||||
"cases": 32,
|
||||
"invalid": 24,
|
||||
"legal_mask": 8,
|
||||
"status": "passed"
|
||||
},
|
||||
"completion": "Acceptance, source review and all four commit gates passed.",
|
||||
"commit_gates": {
|
||||
"fmt": "passed",
|
||||
"clippy_all_targets_all_features": "passed",
|
||||
"make_bundle": "passed; ad-hoc codesigned",
|
||||
"cargo_test_all_features": "308 application tests passed, 214 ignored; two tools each 4 passed"
|
||||
},
|
||||
"invalid_rows_vs_pinned_oracle": "passed; exact census/error text and valid tokens; invalid lazy sentinel -1",
|
||||
"mtplx_sampler_mixed_distributions_match_reference": "100 cases passed in default cargo test",
|
||||
"approved_exception": "User approved rejection throughout Qwen MTP on 2026-10-04, including specialized paths where the pinned oracle still fabricates token 0.",
|
||||
"batched_greedy": "passed: each of 4 rows in both positions beside a healthy neighbor, widths 64 and 4097"
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
{"case":0,"mtp":true,"stage":"Prefill","row":"Nan","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":1,"mtp":true,"stage":"Prefill","row":"Nan","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":2,"mtp":true,"stage":"Prefill","row":"PositiveInf","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":3,"mtp":true,"stage":"Prefill","row":"PositiveInf","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":4,"mtp":true,"stage":"Prefill","row":"Masked","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":5,"mtp":true,"stage":"Prefill","row":"Masked","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":6,"mtp":true,"stage":"Prefill","row":"LegalMask","streaming":false,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":7,"mtp":true,"stage":"Prefill","row":"LegalMask","streaming":true,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":8,"mtp":true,"stage":"Target","row":"Nan","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":9,"mtp":true,"stage":"Target","row":"Nan","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":10,"mtp":true,"stage":"Target","row":"PositiveInf","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":11,"mtp":true,"stage":"Target","row":"PositiveInf","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":12,"mtp":true,"stage":"Target","row":"Masked","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":13,"mtp":true,"stage":"Target","row":"Masked","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":14,"mtp":true,"stage":"Target","row":"LegalMask","streaming":false,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":15,"mtp":true,"stage":"Target","row":"LegalMask","streaming":true,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":16,"mtp":true,"stage":"Capture","row":"Nan","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":17,"mtp":true,"stage":"Capture","row":"Nan","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":18,"mtp":true,"stage":"Capture","row":"PositiveInf","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":19,"mtp":true,"stage":"Capture","row":"PositiveInf","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":20,"mtp":true,"stage":"Capture","row":"Masked","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":21,"mtp":true,"stage":"Capture","row":"Masked","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":22,"mtp":true,"stage":"Capture","row":"LegalMask","streaming":false,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":23,"mtp":true,"stage":"Capture","row":"LegalMask","streaming":true,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":24,"mtp":true,"stage":"Draft","row":"Nan","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":25,"mtp":true,"stage":"Draft","row":"Nan","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":26,"mtp":true,"stage":"Draft","row":"PositiveInf","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":27,"mtp":true,"stage":"Draft","row":"PositiveInf","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":28,"mtp":true,"stage":"Draft","row":"Masked","streaming":false,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":29,"mtp":true,"stage":"Draft","row":"Masked","streaming":true,"legal":false,"healthy_preserved":true,"cold_retry":true}
|
||||
{"case":30,"mtp":true,"stage":"Draft","row":"LegalMask","streaming":false,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"case":31,"mtp":true,"stage":"Draft","row":"LegalMask","streaming":true,"legal":true,"healthy_preserved":true,"cold_retry":false}
|
||||
{"event":"test_resource_summary","elapsed_ms":81120,"peak":{"cpu_time_ns":83647679458,"resident_bytes":27535687680,"physical_bytes":78398221632,"peak_physical_bytes":78399221056,"disk_read_bytes":70370205696,"disk_write_bytes":12404834304},"error":null}
|
||||
@@ -0,0 +1,16 @@
|
||||
{"case":"invalid_logits","row":"nan","temperature":0.0,"ordinary":{"Err":"Qwen non-finite logits in greedy argmax (nan=64, inf=0, vocab=64)"}}
|
||||
{"case":"invalid_logits","row":"nan","temperature":0.6,"ordinary":{"Err":"Qwen non-finite logits in softmax (nan=64, inf=0, vocab=64)"}}
|
||||
{"case":"invalid_logits","row":"nan","temperature":1.0,"ordinary":{"Err":"Qwen non-finite logits in softmax (nan=64, inf=0, vocab=64)"}}
|
||||
{"case":"invalid_logits_lazy","row":"nan","result":{"Ok":[-1]}}
|
||||
{"case":"invalid_logits","row":"positive_inf","temperature":0.0,"ordinary":{"Err":"Qwen non-finite logits in greedy argmax (nan=0, inf=1, vocab=64)"}}
|
||||
{"case":"invalid_logits","row":"positive_inf","temperature":0.6,"ordinary":{"Err":"Qwen non-finite logits in softmax (nan=0, inf=1, vocab=64)"}}
|
||||
{"case":"invalid_logits","row":"positive_inf","temperature":1.0,"ordinary":{"Err":"Qwen non-finite logits in softmax (nan=0, inf=1, vocab=64)"}}
|
||||
{"case":"invalid_logits_lazy","row":"positive_inf","result":{"Ok":[-1]}}
|
||||
{"case":"invalid_logits","row":"all_negative_inf","temperature":0.0,"ordinary":{"Err":"Qwen non-finite logits in greedy argmax (nan=0, inf=64, vocab=64)"}}
|
||||
{"case":"invalid_logits","row":"all_negative_inf","temperature":0.6,"ordinary":{"Err":"Qwen non-finite logits in softmax (nan=0, inf=64, vocab=64)"}}
|
||||
{"case":"invalid_logits","row":"all_negative_inf","temperature":1.0,"ordinary":{"Err":"Qwen non-finite logits in softmax (nan=0, inf=64, vocab=64)"}}
|
||||
{"case":"invalid_logits_lazy","row":"all_negative_inf","result":{"Ok":[-1]}}
|
||||
{"case":"invalid_logits","row":"valid_mask","temperature":0.0,"ordinary":{"Ok":1}}
|
||||
{"case":"invalid_logits","row":"valid_mask","temperature":0.6,"ordinary":{"Ok":1}}
|
||||
{"case":"invalid_logits","row":"valid_mask","temperature":1.0,"ordinary":{"Ok":1}}
|
||||
{"case":"invalid_logits_lazy","row":"valid_mask","result":{"Ok":[3]}}
|
||||
@@ -0,0 +1,20 @@
|
||||
{"case": "ordinary", "row": "nan", "temperature": 0.0, "error": "non-finite logits in greedy argmax (nan=64, inf=0, vocab=64)", "nan": 64, "inf": 0, "vocab": 64}
|
||||
{"case": "ordinary", "row": "nan", "temperature": 0.6, "error": "non-finite logits in softmax (nan=64, inf=0, vocab=64)", "nan": 64, "inf": 0, "vocab": 64}
|
||||
{"case": "ordinary", "row": "nan", "temperature": 1.0, "error": "non-finite logits in softmax (nan=64, inf=0, vocab=64)", "nan": 64, "inf": 0, "vocab": 64}
|
||||
{"case": "lazy", "row": "nan", "token": -1}
|
||||
{"case": "mtp_greedy_specialized", "row": "nan", "token": 0}
|
||||
{"case": "ordinary", "row": "positive_inf", "temperature": 0.0, "error": "non-finite logits in greedy argmax (nan=0, inf=1, vocab=64)", "nan": 0, "inf": 1, "vocab": 64}
|
||||
{"case": "ordinary", "row": "positive_inf", "temperature": 0.6, "error": "non-finite logits in softmax (nan=0, inf=1, vocab=64)", "nan": 0, "inf": 1, "vocab": 64}
|
||||
{"case": "ordinary", "row": "positive_inf", "temperature": 1.0, "error": "non-finite logits in softmax (nan=0, inf=1, vocab=64)", "nan": 0, "inf": 1, "vocab": 64}
|
||||
{"case": "lazy", "row": "positive_inf", "token": -1}
|
||||
{"case": "mtp_greedy_specialized", "row": "positive_inf", "token": 0}
|
||||
{"case": "ordinary", "row": "all_negative_inf", "temperature": 0.0, "error": "non-finite logits in greedy argmax (nan=0, inf=64, vocab=64)", "nan": 0, "inf": 64, "vocab": 64}
|
||||
{"case": "ordinary", "row": "all_negative_inf", "temperature": 0.6, "error": "non-finite logits in softmax (nan=0, inf=64, vocab=64)", "nan": 0, "inf": 64, "vocab": 64}
|
||||
{"case": "ordinary", "row": "all_negative_inf", "temperature": 1.0, "error": "non-finite logits in softmax (nan=0, inf=64, vocab=64)", "nan": 0, "inf": 64, "vocab": 64}
|
||||
{"case": "lazy", "row": "all_negative_inf", "token": -1}
|
||||
{"case": "mtp_greedy_specialized", "row": "all_negative_inf", "token": 0}
|
||||
{"case": "ordinary", "row": "valid_mask", "temperature": 0.0, "token": 1}
|
||||
{"case": "ordinary", "row": "valid_mask", "temperature": 0.6, "token": 1}
|
||||
{"case": "ordinary", "row": "valid_mask", "temperature": 1.0, "token": 1}
|
||||
{"case": "lazy", "row": "valid_mask", "token": 3}
|
||||
{"case": "mtp_greedy_specialized", "row": "valid_mask", "token": 1}
|
||||
Reference in new issue
Block a user