105 lines
5.1 KiB
Markdown
105 lines
5.1 KiB
Markdown
# Qwen3.8 Flash Next Bare Speed artifacts
|
|
|
|
DS4Server uses the text artifacts published at
|
|
`Youssofal/Qwen3.8-Flash-Next-MTPLX-Bare-Speed` revision
|
|
`74559cdf34fbfc0b593de72d17e93f37fd4f9ea7`. The upstream base is
|
|
`Qwen/Qwen3.8-Flash-Next` revision
|
|
`de4b8e4d43b917e7706784d8bb445c9af86a3540`, under the Qwen Community
|
|
License 1.0 included in the artifact set.
|
|
|
|
The source already separates the 71,742,682,599-byte resident text core,
|
|
32,000,154,008-byte PLE table, and 1,672,575,532-byte MTP head. DS4Server's
|
|
artifact conversion is therefore deliberately byte-preserving: the Rust tool
|
|
validates and materializes the selected files without changing their tensor
|
|
containers or numerical interpretation. The complete selected download is
|
|
105,438,842,430 bytes.
|
|
|
|
The manifest at `assets/models/qwen38-flash-next-bare-speed.json` pins every
|
|
selected file by byte size and SHA-256. Its companion TSV records all 2,527
|
|
text, PLE, and MTP tensors with their source file, name, dtype, shape,
|
|
quantization parameters, and byte range. Validation also compares the 2,466
|
|
core tensors with the source index, rejects duplicate names, invalid byte
|
|
ranges, and precision-map drift, and checks the fixed `qwen4_exp` architecture
|
|
and MTPLX serving contract before any payload is accepted.
|
|
|
|
The precision contract follows the pinned files and their per-tensor
|
|
`quantization_config`: affine 4-bit matrices use 64-weight groups; router
|
|
matrices and QSA `index_qk_proj` use the source's declared 8-bit groups; GDN
|
|
convolution and recurrent parameters, norms, hyper-connection parameters, and
|
|
the MTP mixing/norm parameters remain BF16. The MTP layer's attention and MoE
|
|
matrices remain in their published 2-, 4-, or 8-bit affine representation;
|
|
they are not up-converted to satisfy a broader prose description. The PLE
|
|
sidecar's own header pins its 4-bit rows to 32-weight groups. No Python, MLX,
|
|
llama.cpp runtime, or foreign converter is part of the tool or product.
|
|
|
|
`model-vision.safetensors` and its image/video processor configs are explicitly
|
|
excluded. The full config is retained because its `text_config` is the loader
|
|
authority. llama.cpp commit `b356fa2624643b6d5753162ae43efff8cdd4d8cb`
|
|
is not used as a converter because its reviewed path did not preserve this
|
|
separate MTP artifact.
|
|
|
|
Download and authenticate every selected file without loading tensors:
|
|
|
|
```sh
|
|
cargo run --release --bin qwen38-artifacts -- \
|
|
fetch assets/models/qwen38-flash-next-bare-speed.json OUTPUT_DIRECTORY
|
|
```
|
|
|
|
Interrupted downloads remain as resumable `.part` files. The fetcher uses the
|
|
manifest's immutable revision URLs, validates resumed ranges, authenticates
|
|
each completed file, and then runs the complete inventory and precision
|
|
verifier.
|
|
|
|
Verify a pinned checkout:
|
|
|
|
```sh
|
|
cargo run --release --bin qwen38-artifacts -- \
|
|
verify assets/models/qwen38-flash-next-bare-speed.json SOURCE_DIRECTORY
|
|
```
|
|
|
|
Materialize a clean, authenticated artifact directory:
|
|
|
|
```sh
|
|
cargo run --release --bin qwen38-artifacts -- \
|
|
materialize assets/models/qwen38-flash-next-bare-speed.json \
|
|
SOURCE_DIRECTORY OUTPUT_DIRECTORY
|
|
```
|
|
|
|
The output directory must not already exist. Materialization uses hard links
|
|
when possible and otherwise copies bytes, then re-runs the same full verifier
|
|
against the output.
|
|
|
|
## Application intake
|
|
|
|
The native Model Manager exposes the complete 29-file set as one resumable
|
|
artifact. It downloads from the immutable source revision above, verifies each
|
|
file before promotion, retains interrupted `.part` files, and removes the core,
|
|
PLE, MTP, tokenizer, metadata, checksum markers, and partials together.
|
|
|
|
Model selection is persisted as `qwen3.8-flash-next`; intake recognizes the
|
|
artifact from the pinned `qwen4_exp` metadata rather than its directory name.
|
|
Before Metal allocation, the Rust loader checks every configuration and runtime
|
|
contract in the manifest and all 2,527 safetensors records against the frozen
|
|
inventory. The tokenizer uses the pinned byte-level BPE, ChatML special tokens,
|
|
official low/medium/xhigh/direct formatting, grouped tool responses, and the
|
|
official temperature 1.0, top-p 0.95, and top-k 20 defaults.
|
|
|
|
At the native 262,144-token ceiling, admission reports the 71,742,682,599-byte
|
|
resident core, the separately mapped 32,000,154,008-byte PLE table, the
|
|
1,672,575,532-byte optional MTP artifact, 6,557,908,992 bytes of KV/recurrent
|
|
state, and 27,262,976 bytes of prefill transient storage. PLE mapping is not
|
|
counted as resident allocation, and MTP is counted only when enabled. Execution
|
|
still fails with an explicit backend-unavailable error until the Rust Metal
|
|
backend is delivered by issue #95; it never falls through to a GGUF executor.
|
|
|
|
## Verification receipt
|
|
|
|
On 2026-09-03 the pinned source passed full authentication and inventory,
|
|
precision, and representative-row verification. Two separately empty output
|
|
directories were then materialized and independently verified. Each contained
|
|
the same 29 selected files and 105,438,842,430 artifact bytes; normalized
|
|
SHA-256 listings for all 31 output files (artifacts, manifest, and inventory)
|
|
were byte-identical and had list digest
|
|
`3e0233c2356b26859de43b25b2a3df08169dd8263071432bdf2fc497da74007c`.
|
|
The six dequantized-row receipts are pinned directly in the manifest.
|