Document DS4 preference parity
This commit is contained in:
97
PLAN.md
97
PLAN.md
@@ -43,21 +43,23 @@ Exit criterion: restart the app and see the same project/session tree.
|
||||
|
||||
## Phase 1 — model library and on-demand loading
|
||||
|
||||
Status: **partially implemented**. Preferences and the complete background
|
||||
download workflow are implemented; loading, inference, and idle unloading
|
||||
remain open.
|
||||
Status: **partially implemented**. Basic preferences and the complete
|
||||
background download workflow are implemented; DS4 CLI preference parity,
|
||||
loading, inference, and idle unloading remain open.
|
||||
|
||||
1. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session
|
||||
1. Complete the typed model/runtime preference contract described below before
|
||||
connecting chat, the HTTP endpoint, or the engine lifecycle.
|
||||
2. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session
|
||||
API behind a small Rust `Engine` surface modeled on `ds4.h`:
|
||||
`open`, `close`, `create_session`, `sync`, `sample`, and KV save/load.
|
||||
2. Retain the tested model restrictions from DwarfStar; reject unsupported
|
||||
3. Retain the tested model restrictions from DwarfStar; reject unsupported
|
||||
shapes and quantizations before allocating model state.
|
||||
3. Reuse the existing `.metal` kernels and preserve mmap-backed resident
|
||||
4. Reuse the existing `.metal` kernels and preserve mmap-backed resident
|
||||
loading plus the explicit SSD-streaming path. Keep Objective-C only at the
|
||||
Metal interop boundary.
|
||||
4. Keep one engine resident at most, matching DwarfStar's instance-lock and
|
||||
5. Keep one engine resident at most, matching DwarfStar's instance-lock and
|
||||
memory assumptions.
|
||||
5. Move engine work off the UI thread and support cooperative cancellation at
|
||||
6. Move engine work off the UI thread and support cooperative cancellation at
|
||||
the safe session boundaries already defined by `ds4_session_sync`.
|
||||
|
||||
### DS4 KV-cache port and shared session core
|
||||
@@ -121,7 +123,7 @@ to Rust while preserving its semantics and on-disk compatibility:
|
||||
- Changing the selected model prevents new work from using the old model and
|
||||
unloads it as soon as current work reaches a safe boundary.
|
||||
|
||||
### Preferences (implemented)
|
||||
### Preferences foundation (implemented)
|
||||
|
||||
- Provide a Preferences panel from the sidebar and the standard
|
||||
macOS `Command-,` shortcut.
|
||||
@@ -132,6 +134,73 @@ to Rust while preserving its semantics and on-disk compatibility:
|
||||
- Persist preferences in the application SQLite database. Changing preferences
|
||||
never loads a model by itself.
|
||||
|
||||
### DS4 model and generation preferences (required before chat)
|
||||
|
||||
The Preferences panel must expose every persistent model, runtime, and
|
||||
generation parameter accepted by the `ds4` command line. `../ds4/ds4_cli.c`,
|
||||
`ds4.h`, and the shared GPU/distributed parsers are the source of truth; use the
|
||||
same defaults, ranges, units, dependencies, and model-family overrides rather
|
||||
than inventing application-specific behavior.
|
||||
|
||||
- **Model and capacity:** selected main GGUF (`--model`), optional MTP/DSpark
|
||||
support GGUF (`--mtp`), context tokens (`--ctx`), maximum generated tokens
|
||||
(`--tokens`), and system prompt (`--system`). Managed catalog selections map
|
||||
to their installed files instead of requiring users to type paths.
|
||||
- **Sampling and reasoning:** temperature (`--temp`), top-p, min-p, optional
|
||||
seed, and one reasoning mode representing `--think`, `--think-max`, or
|
||||
`--nothink`. Preserve DS4's explicit-vs-default distinction because GLM
|
||||
applies model-family sampling defaults only when the user has not overridden
|
||||
them.
|
||||
- **Backend and execution:** backend (`--backend` and its aliases), CPU helper
|
||||
threads, GPU power percentage, prefill chunk, exact/quality kernels, and warm
|
||||
weights. Only offer backend/device choices supported by the current build.
|
||||
- **Speculative decoding:** MTP draft-token count and margin, GLM MTP, GLM MTP
|
||||
timing, DSpark enablement, DSpark confidence threshold, and DSpark strict
|
||||
target-only decode. Enabling a dependent control must enable or require its
|
||||
support artifact exactly as the CLI does.
|
||||
- **SSD streaming:** enablement, cold start, expert count or GiB cache budget,
|
||||
fully resident GLM layers, and explicit expert preload count. Retain the
|
||||
distinction between unset/automatic and an explicit zero where DS4 does.
|
||||
- **GPU placement:** per-device VRAM budgets, device indices, and CUDA tensor
|
||||
parallelism when available. Hide or disable these controls in a Metal-only or
|
||||
CPU-only build rather than accepting values the engine cannot honor.
|
||||
- **Directional steering:** direction-vector file, FFN scale, and attention
|
||||
scale, including the CLI rule that a file with no explicit scale defaults the
|
||||
FFN scale to 1.
|
||||
- **Advanced diagnostics that change model execution:** simulated used memory
|
||||
and routed expert profile output. Keep these in an Advanced section with
|
||||
clear diagnostic labels, but persist and pass them through like their CLI
|
||||
equivalents.
|
||||
|
||||
Prompt sources, raw one-shot mode, inspect/dump/perplexity/imatrix/self-test
|
||||
actions, and distributed coordinator/worker network topology are CLI workflows,
|
||||
not persistent model parameters; they belong to their corresponding chat,
|
||||
diagnostic, or deployment surfaces rather than Preferences. Any future DS4 CLI
|
||||
option must be classified explicitly as either a preference or one of these
|
||||
non-preference actions.
|
||||
|
||||
Implement this before chat as one typed settings path shared by the local GUI,
|
||||
the lazy engine lifecycle, and HTTP defaults:
|
||||
|
||||
- Persist unset/automatic values separately from explicit values and provide a
|
||||
Reset to DS4 defaults action. Validate before saving with the same accepted
|
||||
ranges and cross-field rules as the CLI.
|
||||
- Mark settings as load-time or turn-time. A load-time change unloads/reloads at
|
||||
the next safe boundary; sampling, seed, token budget, and reasoning changes
|
||||
apply to the next turn without duplicating the engine.
|
||||
- Local chat uses these values directly. HTTP requests use them as defaults and
|
||||
may override only request-scoped generation fields supported by the API; both
|
||||
paths still resolve through the same effective-settings builder.
|
||||
- Add a focused parity test that inventories every option branch in the DS4 CLI
|
||||
model/runtime, sampling, and steering groups and fails until each option is
|
||||
mapped to a typed preference or explicitly classified as a non-preference
|
||||
action.
|
||||
|
||||
Exit criterion: save, restart, and recover every supported DS4 model/runtime
|
||||
and generation value; show the same effective defaults and validation as the
|
||||
CLI; and construct one engine configuration plus one turn configuration without
|
||||
chat or HTTP code re-parsing preference fields.
|
||||
|
||||
### Targeted downloads and storage
|
||||
|
||||
- The Preferences panel can download the selected model's main GGUF and the
|
||||
@@ -319,11 +388,13 @@ tests, and release packaging remain open.
|
||||
|
||||
## Recommended next milestones
|
||||
|
||||
1. Implement the single-engine lifecycle and local generation path: lazy load,
|
||||
1. Complete DS4 CLI model/runtime/generation preference parity and its shared
|
||||
typed effective-settings builder.
|
||||
2. Implement the single-engine lifecycle and local generation path: lazy load,
|
||||
shared concurrent acquisition, activity tracking, and idle unload.
|
||||
2. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
|
||||
3. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
|
||||
chats and HTTP requests cannot load duplicate engines.
|
||||
3. Add transcript/KV persistence, then connect the existing conversation UI and
|
||||
4. Add transcript/KV persistence, then connect the existing conversation UI and
|
||||
finally port the agent tools.
|
||||
4. Add the opt-in Dev Brain tool after local file/search boundaries and agent
|
||||
5. Add the opt-in Dev Brain tool after local file/search boundaries and agent
|
||||
approvals are stable.
|
||||
|
||||
Reference in New Issue
Block a user