Document DS4 preference parity

This commit is contained in:
Georg Bauer
2026-07-24 13:19:12 +02:00
parent 952fb79edc
commit 2e4b811f48

97
PLAN.md
View File

@@ -43,21 +43,23 @@ Exit criterion: restart the app and see the same project/session tree.
## Phase 1 — model library and on-demand loading
Status: **partially implemented**. Preferences and the complete background
download workflow are implemented; loading, inference, and idle unloading
remain open.
Status: **partially implemented**. Basic preferences and the complete
background download workflow are implemented; DS4 CLI preference parity,
loading, inference, and idle unloading remain open.
1. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session
1. Complete the typed model/runtime preference contract described below before
connecting chat, the HTTP endpoint, or the engine lifecycle.
2. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session
API behind a small Rust `Engine` surface modeled on `ds4.h`:
`open`, `close`, `create_session`, `sync`, `sample`, and KV save/load.
2. Retain the tested model restrictions from DwarfStar; reject unsupported
3. Retain the tested model restrictions from DwarfStar; reject unsupported
shapes and quantizations before allocating model state.
3. Reuse the existing `.metal` kernels and preserve mmap-backed resident
4. Reuse the existing `.metal` kernels and preserve mmap-backed resident
loading plus the explicit SSD-streaming path. Keep Objective-C only at the
Metal interop boundary.
4. Keep one engine resident at most, matching DwarfStar's instance-lock and
5. Keep one engine resident at most, matching DwarfStar's instance-lock and
memory assumptions.
5. Move engine work off the UI thread and support cooperative cancellation at
6. Move engine work off the UI thread and support cooperative cancellation at
the safe session boundaries already defined by `ds4_session_sync`.
### DS4 KV-cache port and shared session core
@@ -121,7 +123,7 @@ to Rust while preserving its semantics and on-disk compatibility:
- Changing the selected model prevents new work from using the old model and
unloads it as soon as current work reaches a safe boundary.
### Preferences (implemented)
### Preferences foundation (implemented)
- Provide a Preferences panel from the sidebar and the standard
macOS `Command-,` shortcut.
@@ -132,6 +134,73 @@ to Rust while preserving its semantics and on-disk compatibility:
- Persist preferences in the application SQLite database. Changing preferences
never loads a model by itself.
### DS4 model and generation preferences (required before chat)
The Preferences panel must expose every persistent model, runtime, and
generation parameter accepted by the `ds4` command line. `../ds4/ds4_cli.c`,
`ds4.h`, and the shared GPU/distributed parsers are the source of truth; use the
same defaults, ranges, units, dependencies, and model-family overrides rather
than inventing application-specific behavior.
- **Model and capacity:** selected main GGUF (`--model`), optional MTP/DSpark
support GGUF (`--mtp`), context tokens (`--ctx`), maximum generated tokens
(`--tokens`), and system prompt (`--system`). Managed catalog selections map
to their installed files instead of requiring users to type paths.
- **Sampling and reasoning:** temperature (`--temp`), top-p, min-p, optional
seed, and one reasoning mode representing `--think`, `--think-max`, or
`--nothink`. Preserve DS4's explicit-vs-default distinction because GLM
applies model-family sampling defaults only when the user has not overridden
them.
- **Backend and execution:** backend (`--backend` and its aliases), CPU helper
threads, GPU power percentage, prefill chunk, exact/quality kernels, and warm
weights. Only offer backend/device choices supported by the current build.
- **Speculative decoding:** MTP draft-token count and margin, GLM MTP, GLM MTP
timing, DSpark enablement, DSpark confidence threshold, and DSpark strict
target-only decode. Enabling a dependent control must enable or require its
support artifact exactly as the CLI does.
- **SSD streaming:** enablement, cold start, expert count or GiB cache budget,
fully resident GLM layers, and explicit expert preload count. Retain the
distinction between unset/automatic and an explicit zero where DS4 does.
- **GPU placement:** per-device VRAM budgets, device indices, and CUDA tensor
parallelism when available. Hide or disable these controls in a Metal-only or
CPU-only build rather than accepting values the engine cannot honor.
- **Directional steering:** direction-vector file, FFN scale, and attention
scale, including the CLI rule that a file with no explicit scale defaults the
FFN scale to 1.
- **Advanced diagnostics that change model execution:** simulated used memory
and routed expert profile output. Keep these in an Advanced section with
clear diagnostic labels, but persist and pass them through like their CLI
equivalents.
Prompt sources, raw one-shot mode, inspect/dump/perplexity/imatrix/self-test
actions, and distributed coordinator/worker network topology are CLI workflows,
not persistent model parameters; they belong to their corresponding chat,
diagnostic, or deployment surfaces rather than Preferences. Any future DS4 CLI
option must be classified explicitly as either a preference or one of these
non-preference actions.
Implement this before chat as one typed settings path shared by the local GUI,
the lazy engine lifecycle, and HTTP defaults:
- Persist unset/automatic values separately from explicit values and provide a
Reset to DS4 defaults action. Validate before saving with the same accepted
ranges and cross-field rules as the CLI.
- Mark settings as load-time or turn-time. A load-time change unloads/reloads at
the next safe boundary; sampling, seed, token budget, and reasoning changes
apply to the next turn without duplicating the engine.
- Local chat uses these values directly. HTTP requests use them as defaults and
may override only request-scoped generation fields supported by the API; both
paths still resolve through the same effective-settings builder.
- Add a focused parity test that inventories every option branch in the DS4 CLI
model/runtime, sampling, and steering groups and fails until each option is
mapped to a typed preference or explicitly classified as a non-preference
action.
Exit criterion: save, restart, and recover every supported DS4 model/runtime
and generation value; show the same effective defaults and validation as the
CLI; and construct one engine configuration plus one turn configuration without
chat or HTTP code re-parsing preference fields.
### Targeted downloads and storage
- The Preferences panel can download the selected model's main GGUF and the
@@ -319,11 +388,13 @@ tests, and release packaging remain open.
## Recommended next milestones
1. Implement the single-engine lifecycle and local generation path: lazy load,
1. Complete DS4 CLI model/runtime/generation preference parity and its shared
typed effective-settings builder.
2. Implement the single-engine lifecycle and local generation path: lazy load,
shared concurrent acquisition, activity tracking, and idle unload.
2. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
3. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
chats and HTTP requests cannot load duplicate engines.
3. Add transcript/KV persistence, then connect the existing conversation UI and
4. Add transcript/KV persistence, then connect the existing conversation UI and
finally port the agent tools.
4. Add the opt-in Dev Brain tool after local file/search boundaries and agent
5. Add the opt-in Dev Brain tool after local file/search boundaries and agent
approvals are stable.