hugo/DS4Server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement DS4 model intake

Browse Source
This commit is contained in:
Georg Bauer
2026-07-24 16:53:03 +02:00
parent 87dfc01e74
commit 616b550849
13 changed files with 2418 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
AGENTS.md
View File

@@ -3,6 +3,7 @@
- Prefer simple, idiomatic Rust; reuse existing code and dependencies before adding abstractions or crates.
- Keep changes focused, handle errors explicitly, and add the smallest useful test for non-trivial behavior.
- Preserve `rustfmt` output and keep Clippy warning-free.
- This is not a GitHub project. Use direct `git` commands for version control and the `tea` CLI for forge operations; do not use GitHub tools or workflows.
## Visual language

1
Cargo.lock generated
View File

@@ -1136,6 +1136,7 @@ dependencies = [
"diesel",
"diesel_migrations",
"iced",
"memmap2",
"muda",
"png",
"rfd",

1
Cargo.toml
View File

@@ -11,6 +11,7 @@ publish = false
diesel = { version = "2.3.11", features = ["sqlite", "returning_clauses_for_sqlite_3_35", "64-column-tables"] }
diesel_migrations = "2.3.2"
iced = { version = "0.13.1", features = ["svg", "tokio"] }
memmap2 = "0.9.11"
png = "0.17.16"
rfd = "0.15.4"
sha2 = "0.11.0"

44
PLAN.md
View File

@@ -15,7 +15,7 @@ sessions, tools, and turn orchestration.
| App shell | Implemented | Native multi-window Iced app, Codex-inspired layout, native Application/Edit/Window menus, Preferences panel, and Model Manager window | Functional chat content and native app packaging |
| Projects and sessions | Implemented for metadata | Native folder picker, project-name dialog, create/select/delete projects and sessions | Transcripts, KV payloads, rename/archive, and model binding |
| Persistence | Implemented for metadata and preferences | SQLite in Application Support, Diesel ORM, embedded migrations, constrained defaults and CRUD tests | Messages, downloaded-artifact metadata, and session-runtime migrations |
| Model runtime | Preferences and downloads implemented | Complete typed DS4 preference contract plus a Model Manager for background download, validation, deletion, progress, ETA, stop, and restart-resume | Loading, inference, idle unloading, and Metal integration |
| Model runtime | Model intake implemented | Typed DS4 preferences, managed downloads, mmap-backed GGUF v3 loading, complete DS4 shape/quantization validation, DeepSeek/GLM tokenization, and prompt rendering | Metal inference, sessions/KV state, lifecycle coordination, and idle unloading |
| Local endpoint | Not started | Nothing listening | OpenAI-compatible HTTP and streaming surfaces |
| Agent | UI shell only | Empty conversation pane and composer placeholder | Messages, generation, tools, approvals, compaction, and cancellation |
| A2UI | Future extension | bDS2 provides the reference structured render-tool contract | Native inline surfaces for local chat after core chat and tools are stable |
@@ -23,10 +23,12 @@ sessions, tools, and turn orchestration.
| Release | Not started | Development binary builds and tests | `.app` packaging, signing, entitlements, and notarization |
The application currently manages durable project/session metadata and model
preferences. Its separate Model Manager can download, validate, and delete
main and DSpark GGUF artifacts in managed Application Support storage without
blocking the UI. It does not yet load a model, serve HTTP, save chat messages,
or run an agent.
preferences. Its separate Model Manager can download, fully validate, and
delete main and DSpark GGUF artifacts in managed Application Support storage
without blocking the UI. The engine can mmap a configured model, validate its
metadata and complete tensor layout, load its tokenizer, and render a prompt;
the app does not yet acquire that engine for inference, serve HTTP, save chat
messages, or run an agent.
## Phase 0 — project and session shell
@@ -45,17 +47,21 @@ Exit criterion: restart the app and see the same project/session tree.
## Phase 1 — model library and on-demand loading
Status: **partially implemented**. The typed DS4 preference contract and the
complete background download workflow are implemented; loading, inference,
and idle unloading remain open.
Status: **partially implemented**. The typed DS4 preference contract, shared
effective-settings builder, background download workflow, and model-intake
boundary are implemented; Metal inference, session state, and idle lifecycle
coordination remain open.
1. **Implemented:** the typed model/runtime preference contract described below
is complete before chat, the HTTP endpoint, or the engine lifecycle.
2. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session
API behind a small Rust `Engine` surface modeled on `ds4.h`:
`open`, `close`, `create_session`, `sync`, `sample`, and KV save/load.
3. Retain the tested model restrictions from DwarfStar; reject unsupported
shapes and quantizations before allocating model state.
2. **Partially implemented:** the mmap-backed GGUF loader, DeepSeek/GLM
tokenizer, prompt renderer, and model `open`/`close` ownership boundary are
complete. The Metal session API modeled on `ds4.h` remains:
`create_session`, `sync`, `sample`, and KV save/load.
3. **Implemented:** retain the tested model restrictions from DwarfStar. Main
and DSpark artifacts reject invalid GGUF versions, catalog mismatches, and
incompatible metadata, tensor shapes, offsets, or quantization before
promotion and again before model use.
4. Reuse the existing `.metal` kernels and preserve mmap-backed resident
loading plus the explicit SSD-streaming path. Keep Objective-C only at the
Metal interop boundary.
@@ -475,15 +481,13 @@ tests, and release packaging remain open.
## Recommended next milestones
1. Complete DS4 CLI model/runtime/generation preference parity and its shared
typed effective-settings builder.
2. Implement the single-engine lifecycle and local generation path: lazy load,
1. Implement the single-engine lifecycle and local generation path: lazy load,
shared concurrent acquisition, activity tracking, and idle unload.
3. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
2. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
chats and HTTP requests cannot load duplicate engines.
4. Add transcript/KV persistence, then connect the existing conversation UI and
3. Add transcript/KV persistence, then connect the existing conversation UI and
finally port the agent tools.
5. Add the opt-in Dev Brain tool after local file/search boundaries and agent
4. Add the opt-in Dev Brain tool after local file/search boundaries and agent
approvals are stable.
6. Add bDS2-style A2UI render tools and native inline surfaces after the core
5. Add bDS2-style A2UI render tools and native inline surfaces after the core
local chat/tool loop is complete.

4
src/app.rs
View File

@@ -63,7 +63,7 @@ impl PreferenceDraft {
.ok_or_else(|| format!("Unsupported model: {}", preferences.selected_model))?;
let generation = preferences.generation()?;
let runtime = preferences.runtime()?;
runtime.engine_settings(model)?;
runtime.validate(model)?;
let execution = &runtime.execution;
let speculative = &runtime.speculative;
Ok(Self {
@@ -1030,7 +1030,7 @@ impl App {
return;
}
};
if let Err(error) = runtime.engine_settings(model) {
if let Err(error) = runtime.validate(model) {
self.preference_error = Some(error);
return;
}

37
src/app/view.rs
View File

@@ -314,18 +314,19 @@ impl App {
let effective = self
.preference_draft
.generation()
.ok()
.map(|generation| generation.turn_settings(self.preference_draft.model));
let engine = self
.preference_draft
.runtime()
.ok()
.and_then(|runtime| runtime.engine_settings(self.preference_draft.model).ok());
let artifacts = model::engine_artifacts(
self.preference_draft.model,
self.preference_draft.dspark_enabled,
&models_path(),
);
.and_then(|generation| {
self.preference_draft.runtime().and_then(|runtime| {
crate::settings::effective_settings(
self.preference_draft.model,
&generation,
&runtime,
&models_path(),
)
})
})
.ok();
let engine = effective.as_ref().map(|settings| &settings.engine);
let turn = effective.as_ref().map(|settings| &settings.turn);
let mut power = text_input("100", &self.preference_draft.power_percent);
let mut prefill = text_input("Automatic", &self.preference_draft.prefill_chunk);
let mut ssd_full_layers = text_input("Automatic", &self.preference_draft.ssd_full_layers);
@@ -364,10 +365,12 @@ impl App {
.width(Length::Fill),
text(format!(
"Main: {}{}",
artifacts.model.display(),
artifacts
.mtp
.as_ref()
engine.map_or_else(
|| "Invalid settings".to_owned(),
|engine| engine.artifacts.model.display().to_string(),
),
engine
.and_then(|engine| engine.artifacts.mtp.as_ref())
.map_or_else(String::new, |path| format!(" • support: {}", path.display())),
))
.size(12),
@@ -577,7 +580,7 @@ impl App {
.align_y(Alignment::Center),
text("Blank sampling values retain DS4's model-family defaults. Think Max needs at least 393216 context tokens.")
.size(12),
text(effective.map_or_else(
text(turn.map_or_else(
|| "Effective settings will appear after valid values are entered.".to_owned(),
|settings| format!(
"Effective: {} context • {} max • temp {} • top-p {} • min-p {} • seed {} • {} • system prompt {}",

2
src/database.rs
View File

@@ -341,7 +341,7 @@ impl Database {
generation.validate()?;
let model = crate::model::ModelChoice::from_id(selected_model)
.ok_or_else(|| format!("Unsupported model: {selected_model}"))?;
runtime.engine_settings(model)?;
runtime.validate(model)?;
let execution = &runtime.execution;
let speculative = &runtime.speculative;
let (ssd_cache_experts, ssd_cache_gib) = match runtime.ssd.cache {

1156
src/engine.rs Normal file
View File

File diff suppressed because it is too large Load Diff

509
src/engine/gguf.rs Normal file
View File

@@ -0,0 +1,509 @@
use memmap2::{Mmap, MmapOptions};
use std::collections::HashMap;
use std::fs::File;
use std::path::{Path, PathBuf};
const MAGIC: u32 = 0x4655_4747;
const MAX_COUNT: u64 = 1_000_000;
const MAX_DIMS: usize = 8;
const MAX_DEPTH: u8 = 8;
pub(super) const F32: u32 = 0;
pub(super) const F16: u32 = 1;
pub(super) const Q4_0: u32 = 2;
pub(super) const Q8_0: u32 = 8;
pub(super) const Q2_K: u32 = 10;
pub(super) const Q4_K: u32 = 12;
pub(super) const Q5_K: u32 = 13;
pub(super) const Q6_K: u32 = 14;
pub(super) const IQ2_XXS: u32 = 16;
pub(super) const I32: u32 = 26;
#[derive(Clone, Debug)]
pub(super) enum Value {
U32(u32),
I32(i32),
U64(u64),
I64(i64),
F32(f32),
F64(f64),
Bool(bool),
Bytes(Vec<u8>),
U32s(Vec<u32>),
F32s(Vec<f32>),
Strings(Vec<Vec<u8>>),
Other,
}
#[derive(Clone, Debug)]
pub(super) struct Tensor {
pub(super) kind: u32,
pub(super) dims: Vec<u64>,
pub(super) offset: u64,
pub(super) bytes: u64,
}
pub(super) struct Gguf {
path: PathBuf,
map: Mmap,
pub(super) metadata: HashMap<String, Value>,
pub(super) tensors: HashMap<String, Tensor>,
}
impl Gguf {
pub(super) fn open(path: &Path) -> Result<Self, String> {
let file =
File::open(path).map_err(|error| format!("Cannot open {}: {error}", path.display()))?;
if file.metadata().map_err(|error| error.to_string())?.len() < 32 {
return Err(format!("{} is too small to be GGUF", path.display()));
}
// SAFETY: managed model artifacts are opened read-only and are never mutated
// while a Model owns this mapping.
let map = unsafe { MmapOptions::new().map(&file) }
.map_err(|error| format!("Cannot map {}: {error}", path.display()))?;
let mut cursor = Cursor::new(&map);
if cursor.u32()? != MAGIC {
return Err(format!("{} is not a GGUF file", path.display()));
}
if cursor.u32()? != 3 {
return Err(format!("{} is not GGUF v3", path.display()));
}
let tensor_count = cursor.u64()?;
let metadata_count = cursor.u64()?;
if tensor_count > MAX_COUNT || metadata_count > MAX_COUNT {
return Err("GGUF directory count is too large".into());
}
let mut metadata = HashMap::with_capacity(metadata_count as usize);
let mut alignment = 32_u64;
for _ in 0..metadata_count {
let key = String::from_utf8(cursor.string()?.to_vec())
.map_err(|_| "GGUF metadata key is not UTF-8".to_owned())?;
let kind = cursor.u32()?;
let value = cursor.value(kind, &key, 0)?;
if key == "general.alignment"
&& let Value::U32(value) = value
{
alignment = u64::from(value);
}
if metadata.insert(key.clone(), value).is_some() {
return Err(format!("duplicate GGUF metadata key: {key}"));
}
}
if alignment == 0 {
return Err("GGUF alignment is zero".into());
}
let mut tensors = HashMap::with_capacity(tensor_count as usize);
for _ in 0..tensor_count {
let name = String::from_utf8(cursor.string()?.to_vec())
.map_err(|_| "GGUF tensor name is not UTF-8".to_owned())?;
let ndim = cursor.u32()? as usize;
if ndim == 0 || ndim > MAX_DIMS {
return Err(format!(
"tensor {name} has unsupported dimension count {ndim}"
));
}
let mut dims = Vec::with_capacity(ndim);
let mut elements = 1_u64;
for _ in 0..ndim {
let dim = cursor.u64()?;
elements = elements
.checked_mul(dim)
.ok_or_else(|| format!("tensor {name} element count overflows"))?;
dims.push(dim);
}
let kind = cursor.u32()?;
let relative_offset = cursor.u64()?;
let (block_elements, block_bytes) = tensor_type(kind)
.ok_or_else(|| format!("tensor {name} uses unsupported GGUF type {kind}"))?;
let blocks = elements
.checked_add(block_elements - 1)
.ok_or_else(|| format!("tensor {name} size overflows"))?
/ block_elements;
let bytes = blocks
.checked_mul(block_bytes)
.ok_or_else(|| format!("tensor {name} size overflows"))?;
if tensors
.insert(
name.clone(),
Tensor {
kind,
dims,
offset: relative_offset,
bytes,
},
)
.is_some()
{
return Err(format!("duplicate GGUF tensor: {name}"));
}
}
let data_start = align(cursor.position() as u64, alignment)?;
for (name, tensor) in &mut tensors {
tensor.offset = data_start
.checked_add(tensor.offset)
.ok_or_else(|| format!("tensor {name} offset overflows"))?;
let end = tensor
.offset
.checked_add(tensor.bytes)
.ok_or_else(|| format!("tensor {name} end overflows"))?;
if end > map.len() as u64 {
return Err(format!("tensor {name} points outside the GGUF file"));
}
}
Ok(Self {
path: path.to_owned(),
map,
metadata,
tensors,
})
}
pub(super) fn path(&self) -> &Path {
&self.path
}
pub(super) fn len(&self) -> u64 {
self.map.len() as u64
}
pub(super) fn tensor(&self, name: &str) -> Result<&Tensor, String> {
self.tensors
.get(name)
.ok_or_else(|| format!("required tensor is missing: {name}"))
}
pub(super) fn tensor_data(&self, name: &str) -> Result<&[u8], String> {
let tensor = self.tensor(name)?;
let start = tensor.offset as usize;
let end = start + tensor.bytes as usize;
Ok(&self.map[start..end])
}
pub(super) fn u32(&self, key: &str) -> Result<u32, String> {
match self.metadata.get(key) {
Some(Value::U32(value)) => Ok(*value),
_ => Err(format!("required uint32 metadata key is missing: {key}")),
}
}
pub(super) fn u64(&self, key: &str) -> Result<u64, String> {
match self.metadata.get(key) {
Some(Value::U64(value)) => Ok(*value),
Some(Value::U32(value)) => Ok(u64::from(*value)),
_ => Err(format!("required integer metadata key is missing: {key}")),
}
}
pub(super) fn f32(&self, key: &str) -> Result<f32, String> {
match self.metadata.get(key) {
Some(Value::F32(value)) => Ok(*value),
Some(Value::F64(value)) => Ok(*value as f32),
Some(Value::U32(value)) => Ok(*value as f32),
Some(Value::I32(value)) => Ok(*value as f32),
_ => Err(format!("required numeric metadata key is missing: {key}")),
}
}
pub(super) fn token_id(&self, key: &str) -> Result<i32, String> {
let value = match self.metadata.get(key) {
Some(Value::U32(value)) => i64::from(*value),
Some(Value::I32(value)) => i64::from(*value),
Some(Value::U64(value)) => i64::try_from(*value).unwrap_or(-1),
Some(Value::I64(value)) => *value,
_ => -1,
};
i32::try_from(value)
.ok()
.filter(|value| *value >= 0)
.ok_or_else(|| format!("required tokenizer token ID is missing: {key}"))
}
pub(super) fn boolean(&self, key: &str) -> Result<bool, String> {
match self.metadata.get(key) {
Some(Value::Bool(value)) => Ok(*value),
_ => Err(format!("required boolean metadata key is missing: {key}")),
}
}
pub(super) fn bytes(&self, key: &str) -> Result<&[u8], String> {
match self.metadata.get(key) {
Some(Value::Bytes(value)) => Ok(value),
_ => Err(format!("required string metadata key is missing: {key}")),
}
}
pub(super) fn u32s(&self, key: &str) -> Result<&[u32], String> {
match self.metadata.get(key) {
Some(Value::U32s(value)) => Ok(value),
_ => Err(format!(
"required integer-array metadata key is missing: {key}"
)),
}
}
pub(super) fn f32s(&self, key: &str) -> Result<&[f32], String> {
match self.metadata.get(key) {
Some(Value::F32s(value)) => Ok(value),
_ => Err(format!(
"required float-array metadata key is missing: {key}"
)),
}
}
pub(super) fn strings(&self, key: &str) -> Result<&[Vec<u8>], String> {
match self.metadata.get(key) {
Some(Value::Strings(value)) => Ok(value),
_ => Err(format!(
"required string-array metadata key is missing: {key}"
)),
}
}
}
fn tensor_type(kind: u32) -> Option<(u64, u64)> {
Some(match kind {
0 => (1, 4),
1 => (1, 2),
2 => (32, 18),
3 => (32, 20),
6 => (32, 22),
7 => (32, 24),
8 => (32, 34),
9 => (32, 40),
10 => (256, 84),
11 => (256, 110),
12 => (256, 144),
13 => (256, 176),
14 => (256, 210),
15 => (256, 292),
16 => (256, 66),
17 => (256, 74),
18 => (256, 98),
19 => (256, 110),
20 => (256, 50),
21 => (256, 110),
22 => (256, 82),
23 => (256, 136),
24 => (1, 1),
25 => (1, 2),
26 => (1, 4),
27 => (1, 8),
28 => (1, 8),
29 => (256, 56),
30 => (1, 2),
_ => return None,
})
}
fn align(value: u64, alignment: u64) -> Result<u64, String> {
let remainder = value % alignment;
if remainder == 0 {
Ok(value)
} else {
value
.checked_add(alignment - remainder)
.ok_or_else(|| "GGUF alignment overflows".into())
}
}
struct Cursor<'a> {
bytes: &'a [u8],
position: usize,
}
impl<'a> Cursor<'a> {
fn new(bytes: &'a [u8]) -> Self {
Self { bytes, position: 0 }
}
fn position(&self) -> usize {
self.position
}
fn take(&mut self, count: u64) -> Result<&'a [u8], String> {
let count = usize::try_from(count).map_err(|_| "GGUF value is too large")?;
let end = self
.position
.checked_add(count)
.filter(|end| *end <= self.bytes.len())
.ok_or_else(|| format!("truncated GGUF at byte {}", self.position))?;
let value = &self.bytes[self.position..end];
self.position = end;
Ok(value)
}
fn u8(&mut self) -> Result<u8, String> {
Ok(self.take(1)?[0])
}
fn u16(&mut self) -> Result<u16, String> {
Ok(u16::from_le_bytes(self.take(2)?.try_into().unwrap()))
}
fn u32(&mut self) -> Result<u32, String> {
Ok(u32::from_le_bytes(self.take(4)?.try_into().unwrap()))
}
fn i32(&mut self) -> Result<i32, String> {
Ok(i32::from_le_bytes(self.take(4)?.try_into().unwrap()))
}
fn u64(&mut self) -> Result<u64, String> {
Ok(u64::from_le_bytes(self.take(8)?.try_into().unwrap()))
}
fn i64(&mut self) -> Result<i64, String> {
Ok(i64::from_le_bytes(self.take(8)?.try_into().unwrap()))
}
fn f32(&mut self) -> Result<f32, String> {
Ok(f32::from_le_bytes(self.take(4)?.try_into().unwrap()))
}
fn f64(&mut self) -> Result<f64, String> {
Ok(f64::from_le_bytes(self.take(8)?.try_into().unwrap()))
}
fn string(&mut self) -> Result<&'a [u8], String> {
let len = self.u64()?;
self.take(len)
}
fn value(&mut self, kind: u32, key: &str, depth: u8) -> Result<Value, String> {
if depth > MAX_DEPTH {
return Err("GGUF metadata arrays are nested too deeply".into());
}
Ok(match kind {
0 => {
self.u8()?;
Value::Other
}
1 => {
self.u8()?;
Value::Other
}
2 | 3 => {
self.u16()?;
Value::Other
}
4 => Value::U32(self.u32()?),
5 => Value::I32(self.i32()?),
6 => Value::F32(self.f32()?),
7 => Value::Bool(self.u8()? != 0),
8 => Value::Bytes(self.string()?.to_vec()),
9 => self.array(key, depth + 1)?,
10 => Value::U64(self.u64()?),
11 => Value::I64(self.i64()?),
12 => Value::F64(self.f64()?),
_ => return Err(format!("unknown GGUF metadata type {kind}")),
})
}
fn array(&mut self, key: &str, depth: u8) -> Result<Value, String> {
let item = self.u32()?;
let len = self.u64()?;
if len > MAX_COUNT {
return Err(format!("GGUF metadata array is too large: {key}"));
}
let keep = matches!(
key,
"tokenizer.ggml.tokens"
| "tokenizer.ggml.merges"
| "deepseek4.attention.compress_ratios"
| "deepseek4.swiglu_clamp_exp"
| "deepseek4.dspark.target_layer_ids"
| "deepseek4.dspark_target_layer_ids"
| "dspark.target_layer_ids"
);
if !keep {
for _ in 0..len {
self.value(item, key, depth)?;
}
return Ok(Value::Other);
}
match item {
4 | 5 => {
let mut values = Vec::with_capacity(len as usize);
for _ in 0..len {
let value = if item == 4 {
self.u32()?
} else {
self.i32()?
.try_into()
.map_err(|_| format!("negative value in {key}"))?
};
values.push(value);
}
Ok(Value::U32s(values))
}
6 | 12 => {
let mut values = Vec::with_capacity(len as usize);
for _ in 0..len {
values.push(if item == 6 {
self.f32()?
} else {
self.f64()? as f32
});
}
Ok(Value::F32s(values))
}
8 => {
let mut values = Vec::with_capacity(len as usize);
for _ in 0..len {
values.push(self.string()?.to_vec());
}
Ok(Value::Strings(values))
}
_ => Err(format!("unsupported array type {item} for {key}")),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::fs;
use std::time::{SystemTime, UNIX_EPOCH};
#[test]
fn maps_metadata_and_tensor_payload_without_copying_weights() {
let path = std::env::temp_dir().join(format!(
"ds4-server-gguf-{}",
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_nanos()
));
let mut bytes = Vec::new();
bytes.extend(MAGIC.to_le_bytes());
bytes.extend(3_u32.to_le_bytes());
bytes.extend(1_u64.to_le_bytes());
bytes.extend(1_u64.to_le_bytes());
push_string(&mut bytes, b"general.architecture");
bytes.extend(8_u32.to_le_bytes());
push_string(&mut bytes, b"deepseek4");
push_string(&mut bytes, b"weight");
bytes.extend(1_u32.to_le_bytes());
bytes.extend(1_u64.to_le_bytes());
bytes.extend(F32.to_le_bytes());
bytes.extend(0_u64.to_le_bytes());
bytes.resize(bytes.len().div_ceil(32) * 32, 0);
bytes.extend(1_f32.to_le_bytes());
fs::write(&path, bytes).unwrap();
let model = Gguf::open(&path).unwrap();
assert_eq!(model.bytes("general.architecture").unwrap(), b"deepseek4");
assert_eq!(model.tensor("weight").unwrap().dims, [1]);
assert_eq!(model.tensor_data("weight").unwrap(), 1_f32.to_le_bytes());
fs::remove_file(path).unwrap();
}
fn push_string(bytes: &mut Vec<u8>, value: &[u8]) {
bytes.extend((value.len() as u64).to_le_bytes());
bytes.extend(value);
}
}

600
src/engine/tokenizer.rs Normal file
View File

@@ -0,0 +1,600 @@
use super::ModelFamily;
use super::gguf::Gguf;
use crate::settings::ReasoningMode;
use std::collections::HashMap;
const MAX_REASONING_PREFIX: &str = "Reasoning Effort: Absolute maximum with no shortcuts permitted.\n\
You MUST be very thorough in your thinking and comprehensively decompose the problem to resolve the root cause, rigorously stress-testing your logic against all potential paths, edge cases, and adversarial scenarios.\n\
Explicitly write out your entire deliberation process, documenting every intermediate step, considered alternative, and rejected hypothesis to ensure absolutely no assumption is left unchecked.\n\n";
pub(super) struct Tokenizer {
family: ModelFamily,
tokens: Vec<Vec<u8>>,
token_to_id: HashMap<Vec<u8>, i32>,
merge_rank: HashMap<Vec<u8>, usize>,
bos: i32,
eos: i32,
system: i32,
user: i32,
assistant: i32,
observation: i32,
sop: i32,
think_start: i32,
think_end: i32,
}
impl Tokenizer {
pub(super) fn load(model: &Gguf, family: ModelFamily) -> Result<Self, String> {
let tokens = model.strings("tokenizer.ggml.tokens")?.to_vec();
let merges = model.strings("tokenizer.ggml.merges")?;
let token_to_id = tokens
.iter()
.enumerate()
.map(|(id, token)| (token.clone(), id as i32))
.collect::<HashMap<_, _>>();
let merge_rank = merges
.iter()
.enumerate()
.map(|(rank, merge)| (merge.clone(), rank))
.collect::<HashMap<_, _>>();
let lookup = |token: &[u8]| token_to_id.get(token).copied().unwrap_or(-1);
let required = |token: &[u8]| {
token_to_id.get(token).copied().ok_or_else(|| {
format!(
"required tokenizer token is missing: {}",
String::from_utf8_lossy(token)
)
})
};
let (bos, eos, system, user, assistant, observation, sop, think_start, think_end) =
match family {
ModelFamily::DeepSeek => (
required("<｜begin▁of▁sentence｜>".as_bytes())?,
required("<｜end▁of▁sentence｜>".as_bytes())?,
-1,
required("<｜User｜>".as_bytes())?,
required("<｜Assistant｜>".as_bytes())?,
-1,
-1,
required(b"<think>")?,
required(b"</think>")?,
),
ModelFamily::Glm => (
model
.token_id("tokenizer.ggml.bos_token_id")
.ok()
.unwrap_or_else(|| lookup(b"<sop>")),
model
.token_id("tokenizer.ggml.eos_token_id")
.ok()
.unwrap_or_else(|| lookup(b"<|endoftext|>")),
lookup(b"<|system|>"),
lookup(b"<|user|>"),
lookup(b"<|assistant|>"),
lookup(b"<|observation|>"),
lookup(b"<sop>"),
lookup(b"<think>"),
lookup(b"</think>"),
),
};
if [bos, user, assistant, think_start, think_end]
.into_iter()
.any(|token| token < 0)
|| (family == ModelFamily::Glm && system < 0)
{
return Err("tokenizer does not provide the required DS4 chat markers".into());
}
Ok(Self {
family,
tokens,
token_to_id,
merge_rank,
bos,
eos,
system,
user,
assistant,
observation,
sop,
think_start,
think_end,
})
}
pub(super) fn vocab_size(&self) -> usize {
self.tokens.len()
}
pub(super) fn token_bytes(&self, token: i32) -> Option<Vec<u8>> {
let token = self.tokens.get(usize::try_from(token).ok()?)?;
if token.windows(3).any(|window| window == [0xef, 0xbd, 0x9c]) {
return Some(token.clone());
}
let text = std::str::from_utf8(token).ok()?;
Some(text.chars().filter_map(gpt2_codepoint_to_byte).collect())
}
pub(super) fn tokenize(&self, text: &str) -> Vec<i32> {
let mut output = Vec::new();
if self.family == ModelFamily::Glm {
self.tokenize_glm(text, &mut output);
} else {
self.tokenize_joyai(text, &mut output);
}
output
}
pub(super) fn encode_chat(
&self,
system_prompt: &str,
prompt: &str,
reasoning: ReasoningMode,
) -> Vec<i32> {
let mut output = vec![self.bos];
if self.family == ModelFamily::Glm && self.sop >= 0 {
output.push(self.sop);
}
match (self.family, reasoning) {
(ModelFamily::Glm, ReasoningMode::High | ReasoningMode::Max) => {
output.push(self.system);
output.extend(self.tokenize(if reasoning == ReasoningMode::Max {
"Reasoning Effort: Max"
} else {
"Reasoning Effort: High"
}));
}
(ModelFamily::DeepSeek, ReasoningMode::Max) => {
output.extend(self.tokenize(MAX_REASONING_PREFIX));
}
_ => {}
}
if !system_prompt.is_empty() {
if self.family == ModelFamily::Glm {
output.push(self.system);
}
output.extend(self.tokenize(system_prompt));
}
output.push(self.user);
output.extend(self.tokenize(prompt));
output.push(self.assistant);
if reasoning != ReasoningMode::Direct {
output.push(self.think_start);
} else if self.family == ModelFamily::Glm {
output.extend([self.think_start, self.think_end]);
} else {
output.push(self.think_end);
}
output
}
pub(super) fn eos(&self) -> i32 {
self.eos
}
pub(super) fn is_stop(&self, token: i32) -> bool {
token == self.eos
|| (self.family == ModelFamily::Glm
&& [self.system, self.user, self.assistant, self.observation].contains(&token))
}
fn emit_piece(&self, raw: &[u8], output: &mut Vec<i32>) {
if raw.is_empty() {
return;
}
let encoded = byte_encode(raw);
let mut symbols = encoded
.char_indices()
.map(|(start, character)| {
encoded.as_bytes()[start..start + character.len_utf8()].to_vec()
})
.collect::<Vec<_>>();
loop {
let best = symbols
.windows(2)
.enumerate()
.filter_map(|(index, pair)| {
let mut key = Vec::with_capacity(pair[0].len() + pair[1].len() + 1);
key.extend(&pair[0]);
key.push(b' ');
key.extend(&pair[1]);
self.merge_rank.get(&key).map(|rank| (index, *rank))
})
.min_by_key(|(_, rank)| *rank);
let Some((index, _)) = best else { break };
let right = symbols.remove(index + 1);
symbols[index].extend(right);
}
for symbol in symbols {
if let Some(token) = self.token_to_id.get(symbol.as_slice()) {
output.push(*token);
} else {
output.extend(
symbol
.iter()
.filter_map(|byte| self.token_to_id.get(std::slice::from_ref(byte)))
.copied(),
);
}
}
}
fn tokenize_joyai(&self, text: &str, output: &mut Vec<i32>) {
let bytes = text.as_bytes();
let mut position = 0;
while position < bytes.len() {
let start = position;
let byte = bytes[position];
if byte.is_ascii_digit() {
let mut digits = 0;
while position < bytes.len() && bytes[position].is_ascii_digit() && digits < 3 {
position += 1;
digits += 1;
}
} else if cjk_at(text, position) {
while position < bytes.len() && cjk_at(text, position) {
position = next_char(text, position);
}
} else if ascii_punct(byte)
&& position + 1 < bytes.len()
&& bytes[position + 1].is_ascii_alphabetic()
{
position += 1;
while position < bytes.len() && bytes[position].is_ascii_alphabetic() {
position += 1;
}
} else if letter_like(text, position) {
position = consume_letters(text, position);
} else if !ascii_newline(byte)
&& !ascii_punct(byte)
&& position + 1 < bytes.len()
&& letter_like(text, position + 1)
{
position = consume_letters(text, position + 1);
} else if byte == b' ' && position + 1 < bytes.len() && ascii_punct(bytes[position + 1])
{
position += 1;
while position < bytes.len() && ascii_punct(bytes[position]) {
position += 1;
}
while position < bytes.len() && ascii_newline(bytes[position]) {
position += 1;
}
} else if ascii_punct(byte) {
while position < bytes.len() && ascii_punct(bytes[position]) {
position += 1;
}
while position < bytes.len() && ascii_newline(bytes[position]) {
position += 1;
}
} else if byte.is_ascii_whitespace() {
let mut scan = position;
let mut last_newline = None;
while scan < bytes.len() && bytes[scan].is_ascii_whitespace() {
scan += 1;
if ascii_newline(bytes[scan - 1]) {
last_newline = Some(scan);
}
}
position = last_newline.unwrap_or_else(|| {
if scan < bytes.len()
&& scan > position + 1
&& (letter_like(text, scan) || ascii_punct(bytes[scan]))
{
scan - 1
} else {
scan
}
});
} else {
position = next_char(text, position);
}
if position == start {
position = next_char(text, position);
}
self.emit_piece(&bytes[start..position], output);
}
}
fn tokenize_glm(&self, text: &str, output: &mut Vec<i32>) {
let mut position = 0;
while position < text.len() {
let start = position;
let current = char_info(text, position);
if current.ch == '\'' {
let next = char_info(text, current.next);
let lower = next.ch.to_ascii_lowercase();
if matches!(lower, 's' | 't' | 'm' | 'd') {
position = next.next;
self.emit_piece(&text.as_bytes()[start..position], output);
continue;
}
if next.next < text.len() {
let next2 = char_info(text, next.next);
if matches!(
(lower, next2.ch.to_ascii_lowercase()),
('r', 'e') | ('v', 'e') | ('l', 'l')
) {
position = next2.next;
self.emit_piece(&text.as_bytes()[start..position], output);
continue;
}
}
}
let following = char_info(text, current.next);
if !matches!(current.ch, '\r' | '\n')
&& !current.number
&& (current.letter || following.letter)
{
position = current.next;
while position < text.len() && char_info(text, position).letter {
position = char_info(text, position).next;
}
} else if current.number {
let mut digits = 0;
while position < text.len() && char_info(text, position).number && digits < 3 {
position = char_info(text, position).next;
digits += 1;
}
} else {
let (punctuation, punct_start) = if current.ch == ' ' {
(char_info(text, current.next), current.next)
} else {
(current, position)
};
if punctuation.punctuation {
position = punct_start;
while position < text.len() && char_info(text, position).punctuation {
position = char_info(text, position).next;
}
while position < text.len()
&& matches!(char_info(text, position).ch, '\r' | '\n')
{
position = char_info(text, position).next;
}
} else if current.whitespace {
let mut scan = position;
let mut last_newline = None;
let mut last_start = position;
let mut count = 0;
while scan < text.len() && char_info(text, scan).whitespace {
let item = char_info(text, scan);
last_start = scan;
if matches!(item.ch, '\r' | '\n') {
last_newline = Some(item.next);
}
scan = item.next;
count += 1;
}
position = last_newline.unwrap_or(if count > 1 && scan < text.len() {
last_start
} else {
scan
});
} else {
position = current.next;
}
}
if position == start {
position = next_char(text, position);
}
self.emit_piece(&text.as_bytes()[start..position], output);
}
}
}
fn byte_encode(bytes: &[u8]) -> String {
bytes
.iter()
.map(|byte| char::from_u32(gpt2_byte_to_codepoint(*byte)).unwrap())
.collect()
}
fn gpt2_byte_to_codepoint(byte: u8) -> u32 {
if (33..=126).contains(&byte) || (161..=172).contains(&byte) || byte >= 174 {
return u32::from(byte);
}
let mut index = 0;
for candidate in 0..=255_u16 {
let candidate = candidate as u8;
if (33..=126).contains(&candidate) || (161..=172).contains(&candidate) || candidate >= 174 {
continue;
}
if candidate == byte {
return 256 + index;
}
index += 1;
}
u32::from(byte)
}
fn gpt2_codepoint_to_byte(character: char) -> Option<u8> {
let codepoint = character as u32;
if (33..=126).contains(&codepoint)
|| (161..=172).contains(&codepoint)
|| (174..=255).contains(&codepoint)
{
return Some(codepoint as u8);
}
let mut index = 0;
for byte in 0..=255_u16 {
let byte = byte as u8;
if (33..=126).contains(&byte) || (161..=172).contains(&byte) || byte >= 174 {
continue;
}
if codepoint == 256 + index {
return Some(byte);
}
index += 1;
}
None
}
fn next_char(text: &str, position: usize) -> usize {
if position >= text.len() {
return text.len();
}
position + text[position..].chars().next().unwrap().len_utf8()
}
fn cjk_at(text: &str, position: usize) -> bool {
let character = text[position..].chars().next().unwrap() as u32;
(0x4e00..=0x9fa5).contains(&character)
|| (0x3040..=0x309f).contains(&character)
|| (0x30a0..=0x30ff).contains(&character)
}
fn letter_like(text: &str, position: usize) -> bool {
let byte = text.as_bytes()[position];
byte >= 128 || byte.is_ascii_alphabetic()
}
fn consume_letters(text: &str, mut position: usize) -> usize {
while position < text.len() && letter_like(text, position) {
position = next_char(text, position);
}
position
}
fn ascii_punct(byte: u8) -> bool {
byte.is_ascii_punctuation()
}
fn ascii_newline(byte: u8) -> bool {
matches!(byte, b'\n' | b'\r')
}
#[derive(Clone, Copy)]
struct CharInfo {
ch: char,
next: usize,
letter: bool,
number: bool,
whitespace: bool,
punctuation: bool,
}
fn char_info(text: &str, position: usize) -> CharInfo {
if position >= text.len() {
return CharInfo {
ch: '\0',
next: text.len(),
letter: false,
number: false,
whitespace: false,
punctuation: false,
};
}
let ch = text[position..].chars().next().unwrap();
let codepoint = ch as u32;
let whitespace = unicode_whitespace(codepoint);
let number = unicode_number(codepoint);
let punctuation = unicode_punctuation(codepoint);
CharInfo {
ch,
next: position + ch.len_utf8(),
letter: if codepoint < 128 {
ch.is_ascii_alphabetic()
} else {
!whitespace && !number && !punctuation
},
number,
whitespace,
punctuation,
}
}
fn unicode_whitespace(cp: u32) -> bool {
(cp < 128 && (cp as u8).is_ascii_whitespace())
|| matches!(
cp,
0x0085 | 0x00a0 | 0x1680 | 0x2028 | 0x2029 | 0x202f | 0x205f | 0x3000
)
|| (0x2000..=0x200a).contains(&cp)
}
fn unicode_number(cp: u32) -> bool {
(cp < 128 && (cp as u8).is_ascii_digit())
|| [
0x0660..=0x0669,
0x06f0..=0x06f9,
0x07c0..=0x07c9,
0x0966..=0x096f,
0x09e6..=0x09ef,
0x0a66..=0x0a6f,
0x0ae6..=0x0aef,
0x0b66..=0x0b6f,
0x0be6..=0x0bef,
0x0c66..=0x0c6f,
0x0ce6..=0x0cef,
0x0d66..=0x0d6f,
0x0de6..=0x0def,
0x0e50..=0x0e59,
0x0ed0..=0x0ed9,
0x0f20..=0x0f29,
0x1040..=0x1049,
0x1090..=0x1099,
0x17e0..=0x17e9,
0x1810..=0x1819,
0xff10..=0xff19,
]
.iter()
.any(|range| range.contains(&cp))
}
fn unicode_punctuation(cp: u32) -> bool {
if cp < 128 {
return (cp as u8).is_ascii_punctuation();
}
matches!(
cp,
0x00b4
| 0x00bb
| 0x00bf
| 0x00d7
| 0x00f7
| 0x0387
| 0x05c3
| 0x061b
| 0x061e
| 0x061f
| 0x066a
| 0x066d
| 0x06d4
) || [
0x00a1..=0x00a9,
0x00ab..=0x00ac,
0x00ae..=0x00b1,
0x00b6..=0x00b8,
0x02c2..=0x02df,
0x02e5..=0x02eb,
0x02ed..=0x02ff,
0x0375..=0x037e,
0x0384..=0x0385,
0x055a..=0x055f,
0x0589..=0x058a,
0x05be..=0x05c0,
0x05c6..=0x05c7,
0x0609..=0x060a,
0x060c..=0x060d,
0x2000..=0x206f,
0x20a0..=0x20cf,
0x2100..=0x214f,
0x2190..=0x23ff,
0x2460..=0x24ff,
0x2500..=0x2775,
0x2794..=0x2bff,
0x2e00..=0x2e7f,
0x3000..=0x303f,
0xfd3e..=0xfd3f,
0xfe10..=0xfe6f,
0xff01..=0xff0f,
0xff1a..=0xff20,
0xff3b..=0xff40,
0xff5b..=0xff65,
0x1f000..=0x1faff,
]
.iter()
.any(|range| range.contains(&cp))
}

1
src/main.rs
View File

@@ -1,5 +1,6 @@
mod app;
mod database;
mod engine;
mod model;
#[cfg(target_os = "macos")]
mod native_menu;

29
src/model.rs
View File

@@ -27,6 +27,7 @@ const FLASH: Artifact = Artifact {
repository: DEEPSEEK_REPOSITORY,
size: 86_720_111_488,
sha256: "efc7ed607ff27076e3e501fc3fefefa33c0ed8cf1eff483a2b7fdc0c2e616668",
support: Some(false),
};
const FLASH_DSPARK: Artifact = Artifact {
label: "DSpark support",
@@ -34,6 +35,7 @@ const FLASH_DSPARK: Artifact = Artifact {
repository: DEEPSEEK_REPOSITORY,
size: 5_989_114_272,
sha256: "8b3adf5942bec22ae2ea867cd7079cf13530ba83ffcffaf00f5de48664a1a34e",
support: Some(true),
};
const PRO: Artifact = Artifact {
label: "DeepSeek V4 Pro model",
@@ -41,6 +43,7 @@ const PRO: Artifact = Artifact {
repository: DEEPSEEK_REPOSITORY,
size: 464_627_334_560,
sha256: "a0314d9c0e16122cd60071079124a2d17185d317c55a8f95ecb3ed3506278a96",
support: Some(false),
};
const GLM: Artifact = Artifact {
label: "GLM 5.2 model",
@@ -48,6 +51,7 @@ const GLM: Artifact = Artifact {
repository: GLM_REPOSITORY,
size: 211_075_856_448,
sha256: "a49de64c5020432bdae23de36a423a9660a5621bc0db8d12b66bd8814b07fea0",
support: Some(false),
};
#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
@@ -239,6 +243,7 @@ struct Artifact {
repository: &'static str,
size: u64,
sha256: &'static str,
support: Option<bool>,
}
impl Artifact {
@@ -401,7 +406,7 @@ pub(crate) fn validate_managed_artifact(
return Err(format!("{} is not downloaded", artifact.label));
};
match verify(&path, artifact, cancel, verified_bytes) {
match verify(&path, artifact, model, cancel, verified_bytes) {
Ok(DownloadOutcome::Stopped) => return Ok(DownloadOutcome::Stopped),
Ok(DownloadOutcome::Complete) => {}
Err(error) => {
@@ -473,7 +478,9 @@ fn download_artifact_with_cancel(
return Ok(DownloadOutcome::Complete);
}
if destination.exists() {
if verify(&destination, artifact, cancel, verified_bytes)? == DownloadOutcome::Stopped {
if verify(&destination, artifact, model, cancel, verified_bytes)?
== DownloadOutcome::Stopped
{
return Ok(DownloadOutcome::Stopped);
}
mark_verified(model, artifact, models_path)?;
@@ -498,7 +505,7 @@ fn download_artifact_with_cancel(
if cancel.load(Ordering::Relaxed) {
return Ok(DownloadOutcome::Stopped);
}
match verify(&partial, artifact, cancel, verified_bytes) {
match verify(&partial, artifact, model, cancel, verified_bytes) {
Ok(DownloadOutcome::Stopped) => return Ok(DownloadOutcome::Stopped),
Ok(DownloadOutcome::Complete) => {}
Err(error) => {
@@ -528,6 +535,7 @@ fn mark_verified(
fn verify(
path: &Path,
artifact: &Artifact,
model: ModelChoice,
cancel: &AtomicBool,
verified_bytes: &AtomicU64,
) -> Result<DownloadOutcome, String> {
@@ -562,6 +570,9 @@ fn verify(
path.display()
));
}
if let Some(support) = artifact.support {
crate::engine::validate_model_artifact(path, model, support)?;
}
Ok(DownloadOutcome::Complete)
}
@@ -695,6 +706,7 @@ mod tests {
repository: "",
size: 0,
sha256: "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
support: None,
};
let partial = empty.partial_path(ModelChoice::DeepSeekV4Flash, &models_path);
fs::create_dir_all(partial.parent().unwrap()).unwrap();
@@ -726,11 +738,19 @@ mod tests {
repository: "unused",
size: 3,
sha256: "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
support: None,
};
let verified_bytes = AtomicU64::new(999);
assert_eq!(
verify(&path, &artifact, &AtomicBool::new(false), &verified_bytes,).unwrap(),
verify(
&path,
&artifact,
ModelChoice::DeepSeekV4Flash,
&AtomicBool::new(false),
&verified_bytes,
)
.unwrap(),
DownloadOutcome::Complete
);
assert_eq!(verified_bytes.load(Ordering::Relaxed), 3);
@@ -832,6 +852,7 @@ mod tests {
repository: "unused",
size: 10,
sha256: "unused",
support: None,
};
let partial = artifact.partial_path(ModelChoice::DeepSeekV4Flash, &directory);
fs::create_dir_all(partial.parent().unwrap()).unwrap();

86
src/settings.rs
View File

@@ -1,5 +1,6 @@
use crate::model::ModelChoice;
use crate::model::{self, EngineArtifacts, ModelChoice};
use std::fmt;
use std::path::Path;
pub(crate) const REASONING_MODES: [ReasoningMode; 3] = [
ReasoningMode::High,
@@ -247,7 +248,7 @@ pub(crate) struct RuntimePreferences {
}
impl RuntimePreferences {
pub(crate) fn engine_settings(&self, model: ModelChoice) -> Result<EngineSettings, String> {
pub(crate) fn validate(&self, model: ModelChoice) -> Result<(), String> {
self.execution.validate(model)?;
self.speculative.validate(model)?;
self.ssd.validate(model)?;
@@ -256,8 +257,20 @@ impl RuntimePreferences {
if self.ssd.enabled && self.speculative.dspark_enabled {
return Err("SSD streaming is not compatible with DSpark support.".into());
}
Ok(())
}
fn engine_settings(
&self,
model: ModelChoice,
context_tokens: i32,
models_path: &Path,
) -> Result<EngineSettings, String> {
self.validate(model)?;
Ok(EngineSettings {
model,
artifacts: model::engine_artifacts(model, self.speculative.dspark_enabled, models_path),
context_tokens,
execution: self.execution.engine_settings(),
speculative: self.speculative.engine_settings(),
ssd: self.ssd.engine_settings(),
@@ -270,6 +283,8 @@ impl RuntimePreferences {
#[derive(Clone, Debug, PartialEq)]
pub(crate) struct EngineSettings {
pub(crate) model: ModelChoice,
pub(crate) artifacts: EngineArtifacts,
pub(crate) context_tokens: i32,
pub(crate) execution: EngineExecutionSettings,
pub(crate) speculative: EngineSpeculativeSettings,
pub(crate) ssd: EngineSsdSettings,
@@ -478,6 +493,25 @@ pub(crate) struct TurnSettings {
pub(crate) reasoning_mode: ReasoningMode,
}
#[derive(Clone, Debug, PartialEq)]
pub(crate) struct EffectiveSettings {
pub(crate) engine: EngineSettings,
pub(crate) turn: TurnSettings,
}
pub(crate) fn effective_settings(
model: ModelChoice,
generation: &GenerationPreferences,
runtime: &RuntimePreferences,
models_path: &Path,
) -> Result<EffectiveSettings, String> {
generation.validate()?;
Ok(EffectiveSettings {
engine: runtime.engine_settings(model, generation.context_tokens, models_path)?,
turn: generation.turn_settings(model),
})
}
#[cfg(test)]
mod tests {
use super::*;
@@ -576,9 +610,16 @@ mod tests {
},
..RuntimePreferences::default()
};
let engine = runtime
.engine_settings(ModelChoice::DeepSeekV4Flash)
.unwrap();
let effective = effective_settings(
ModelChoice::DeepSeekV4Flash,
&GenerationPreferences::default(),
&runtime,
Path::new("/models"),
)
.unwrap();
let engine = effective.engine;
assert_eq!(engine.context_tokens, 32_768);
assert!(engine.artifacts.mtp.is_none());
assert_eq!(engine.ssd.cache_bytes, 64 * GIB);
assert!(engine.ssd.full_layers_set);
assert_eq!(engine.ssd.full_layers, 0);
@@ -592,11 +633,38 @@ mod tests {
},
..runtime
};
assert!(
incompatible
.engine_settings(ModelChoice::DeepSeekV4Flash)
.is_err()
assert!(incompatible.validate(ModelChoice::DeepSeekV4Flash).is_err());
}
#[test]
fn effective_settings_build_one_engine_and_turn_configuration() {
let generation = GenerationPreferences {
context_tokens: 65_536,
temperature: Some(0.25),
..GenerationPreferences::default()
};
let runtime = RuntimePreferences {
speculative: SpeculativePreferences {
dspark_enabled: true,
..SpeculativePreferences::default()
},
..RuntimePreferences::default()
};
let effective = effective_settings(
ModelChoice::DeepSeekV4Flash,
&generation,
&runtime,
Path::new("/models"),
)
.unwrap();
assert_eq!(effective.engine.context_tokens, 65_536);
assert_eq!(
effective.engine.artifacts.model.parent(),
Some(Path::new("/models/deepseek-v4-flash"))
);
assert!(effective.engine.artifacts.mtp.is_some());
assert_eq!(effective.turn.temperature, 0.25);
}
#[test]