Document DS4 preference parity
This commit is contained in:
97
PLAN.md
97
PLAN.md
@@ -43,21 +43,23 @@ Exit criterion: restart the app and see the same project/session tree.
|
|||||||
|
|
||||||
## Phase 1 — model library and on-demand loading
|
## Phase 1 — model library and on-demand loading
|
||||||
|
|
||||||
Status: **partially implemented**. Preferences and the complete background
|
Status: **partially implemented**. Basic preferences and the complete
|
||||||
download workflow are implemented; loading, inference, and idle unloading
|
background download workflow are implemented; DS4 CLI preference parity,
|
||||||
remain open.
|
loading, inference, and idle unloading remain open.
|
||||||
|
|
||||||
1. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session
|
1. Complete the typed model/runtime preference contract described below before
|
||||||
|
connecting chat, the HTTP endpoint, or the engine lifecycle.
|
||||||
|
2. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session
|
||||||
API behind a small Rust `Engine` surface modeled on `ds4.h`:
|
API behind a small Rust `Engine` surface modeled on `ds4.h`:
|
||||||
`open`, `close`, `create_session`, `sync`, `sample`, and KV save/load.
|
`open`, `close`, `create_session`, `sync`, `sample`, and KV save/load.
|
||||||
2. Retain the tested model restrictions from DwarfStar; reject unsupported
|
3. Retain the tested model restrictions from DwarfStar; reject unsupported
|
||||||
shapes and quantizations before allocating model state.
|
shapes and quantizations before allocating model state.
|
||||||
3. Reuse the existing `.metal` kernels and preserve mmap-backed resident
|
4. Reuse the existing `.metal` kernels and preserve mmap-backed resident
|
||||||
loading plus the explicit SSD-streaming path. Keep Objective-C only at the
|
loading plus the explicit SSD-streaming path. Keep Objective-C only at the
|
||||||
Metal interop boundary.
|
Metal interop boundary.
|
||||||
4. Keep one engine resident at most, matching DwarfStar's instance-lock and
|
5. Keep one engine resident at most, matching DwarfStar's instance-lock and
|
||||||
memory assumptions.
|
memory assumptions.
|
||||||
5. Move engine work off the UI thread and support cooperative cancellation at
|
6. Move engine work off the UI thread and support cooperative cancellation at
|
||||||
the safe session boundaries already defined by `ds4_session_sync`.
|
the safe session boundaries already defined by `ds4_session_sync`.
|
||||||
|
|
||||||
### DS4 KV-cache port and shared session core
|
### DS4 KV-cache port and shared session core
|
||||||
@@ -121,7 +123,7 @@ to Rust while preserving its semantics and on-disk compatibility:
|
|||||||
- Changing the selected model prevents new work from using the old model and
|
- Changing the selected model prevents new work from using the old model and
|
||||||
unloads it as soon as current work reaches a safe boundary.
|
unloads it as soon as current work reaches a safe boundary.
|
||||||
|
|
||||||
### Preferences (implemented)
|
### Preferences foundation (implemented)
|
||||||
|
|
||||||
- Provide a Preferences panel from the sidebar and the standard
|
- Provide a Preferences panel from the sidebar and the standard
|
||||||
macOS `Command-,` shortcut.
|
macOS `Command-,` shortcut.
|
||||||
@@ -132,6 +134,73 @@ to Rust while preserving its semantics and on-disk compatibility:
|
|||||||
- Persist preferences in the application SQLite database. Changing preferences
|
- Persist preferences in the application SQLite database. Changing preferences
|
||||||
never loads a model by itself.
|
never loads a model by itself.
|
||||||
|
|
||||||
|
### DS4 model and generation preferences (required before chat)
|
||||||
|
|
||||||
|
The Preferences panel must expose every persistent model, runtime, and
|
||||||
|
generation parameter accepted by the `ds4` command line. `../ds4/ds4_cli.c`,
|
||||||
|
`ds4.h`, and the shared GPU/distributed parsers are the source of truth; use the
|
||||||
|
same defaults, ranges, units, dependencies, and model-family overrides rather
|
||||||
|
than inventing application-specific behavior.
|
||||||
|
|
||||||
|
- **Model and capacity:** selected main GGUF (`--model`), optional MTP/DSpark
|
||||||
|
support GGUF (`--mtp`), context tokens (`--ctx`), maximum generated tokens
|
||||||
|
(`--tokens`), and system prompt (`--system`). Managed catalog selections map
|
||||||
|
to their installed files instead of requiring users to type paths.
|
||||||
|
- **Sampling and reasoning:** temperature (`--temp`), top-p, min-p, optional
|
||||||
|
seed, and one reasoning mode representing `--think`, `--think-max`, or
|
||||||
|
`--nothink`. Preserve DS4's explicit-vs-default distinction because GLM
|
||||||
|
applies model-family sampling defaults only when the user has not overridden
|
||||||
|
them.
|
||||||
|
- **Backend and execution:** backend (`--backend` and its aliases), CPU helper
|
||||||
|
threads, GPU power percentage, prefill chunk, exact/quality kernels, and warm
|
||||||
|
weights. Only offer backend/device choices supported by the current build.
|
||||||
|
- **Speculative decoding:** MTP draft-token count and margin, GLM MTP, GLM MTP
|
||||||
|
timing, DSpark enablement, DSpark confidence threshold, and DSpark strict
|
||||||
|
target-only decode. Enabling a dependent control must enable or require its
|
||||||
|
support artifact exactly as the CLI does.
|
||||||
|
- **SSD streaming:** enablement, cold start, expert count or GiB cache budget,
|
||||||
|
fully resident GLM layers, and explicit expert preload count. Retain the
|
||||||
|
distinction between unset/automatic and an explicit zero where DS4 does.
|
||||||
|
- **GPU placement:** per-device VRAM budgets, device indices, and CUDA tensor
|
||||||
|
parallelism when available. Hide or disable these controls in a Metal-only or
|
||||||
|
CPU-only build rather than accepting values the engine cannot honor.
|
||||||
|
- **Directional steering:** direction-vector file, FFN scale, and attention
|
||||||
|
scale, including the CLI rule that a file with no explicit scale defaults the
|
||||||
|
FFN scale to 1.
|
||||||
|
- **Advanced diagnostics that change model execution:** simulated used memory
|
||||||
|
and routed expert profile output. Keep these in an Advanced section with
|
||||||
|
clear diagnostic labels, but persist and pass them through like their CLI
|
||||||
|
equivalents.
|
||||||
|
|
||||||
|
Prompt sources, raw one-shot mode, inspect/dump/perplexity/imatrix/self-test
|
||||||
|
actions, and distributed coordinator/worker network topology are CLI workflows,
|
||||||
|
not persistent model parameters; they belong to their corresponding chat,
|
||||||
|
diagnostic, or deployment surfaces rather than Preferences. Any future DS4 CLI
|
||||||
|
option must be classified explicitly as either a preference or one of these
|
||||||
|
non-preference actions.
|
||||||
|
|
||||||
|
Implement this before chat as one typed settings path shared by the local GUI,
|
||||||
|
the lazy engine lifecycle, and HTTP defaults:
|
||||||
|
|
||||||
|
- Persist unset/automatic values separately from explicit values and provide a
|
||||||
|
Reset to DS4 defaults action. Validate before saving with the same accepted
|
||||||
|
ranges and cross-field rules as the CLI.
|
||||||
|
- Mark settings as load-time or turn-time. A load-time change unloads/reloads at
|
||||||
|
the next safe boundary; sampling, seed, token budget, and reasoning changes
|
||||||
|
apply to the next turn without duplicating the engine.
|
||||||
|
- Local chat uses these values directly. HTTP requests use them as defaults and
|
||||||
|
may override only request-scoped generation fields supported by the API; both
|
||||||
|
paths still resolve through the same effective-settings builder.
|
||||||
|
- Add a focused parity test that inventories every option branch in the DS4 CLI
|
||||||
|
model/runtime, sampling, and steering groups and fails until each option is
|
||||||
|
mapped to a typed preference or explicitly classified as a non-preference
|
||||||
|
action.
|
||||||
|
|
||||||
|
Exit criterion: save, restart, and recover every supported DS4 model/runtime
|
||||||
|
and generation value; show the same effective defaults and validation as the
|
||||||
|
CLI; and construct one engine configuration plus one turn configuration without
|
||||||
|
chat or HTTP code re-parsing preference fields.
|
||||||
|
|
||||||
### Targeted downloads and storage
|
### Targeted downloads and storage
|
||||||
|
|
||||||
- The Preferences panel can download the selected model's main GGUF and the
|
- The Preferences panel can download the selected model's main GGUF and the
|
||||||
@@ -319,11 +388,13 @@ tests, and release packaging remain open.
|
|||||||
|
|
||||||
## Recommended next milestones
|
## Recommended next milestones
|
||||||
|
|
||||||
1. Implement the single-engine lifecycle and local generation path: lazy load,
|
1. Complete DS4 CLI model/runtime/generation preference parity and its shared
|
||||||
|
typed effective-settings builder.
|
||||||
|
2. Implement the single-engine lifecycle and local generation path: lazy load,
|
||||||
shared concurrent acquisition, activity tracking, and idle unload.
|
shared concurrent acquisition, activity tracking, and idle unload.
|
||||||
2. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
|
3. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
|
||||||
chats and HTTP requests cannot load duplicate engines.
|
chats and HTTP requests cannot load duplicate engines.
|
||||||
3. Add transcript/KV persistence, then connect the existing conversation UI and
|
4. Add transcript/KV persistence, then connect the existing conversation UI and
|
||||||
finally port the agent tools.
|
finally port the agent tools.
|
||||||
4. Add the opt-in Dev Brain tool after local file/search boundaries and agent
|
5. Add the opt-in Dev Brain tool after local file/search boundaries and agent
|
||||||
approvals are stable.
|
approvals are stable.
|
||||||
|
|||||||
Reference in New Issue
Block a user