diff --git a/PLAN.md b/PLAN.md index 9338357..c2c1ee9 100644 --- a/PLAN.md +++ b/PLAN.md @@ -43,21 +43,23 @@ Exit criterion: restart the app and see the same project/session tree. ## Phase 1 — model library and on-demand loading -Status: **partially implemented**. Preferences and the complete background -download workflow are implemented; loading, inference, and idle unloading -remain open. +Status: **partially implemented**. Basic preferences and the complete +background download workflow are implemented; DS4 CLI preference parity, +loading, inference, and idle unloading remain open. -1. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session +1. Complete the typed model/runtime preference contract described below before + connecting chat, the HTTP endpoint, or the engine lifecycle. +2. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session API behind a small Rust `Engine` surface modeled on `ds4.h`: `open`, `close`, `create_session`, `sync`, `sample`, and KV save/load. -2. Retain the tested model restrictions from DwarfStar; reject unsupported +3. Retain the tested model restrictions from DwarfStar; reject unsupported shapes and quantizations before allocating model state. -3. Reuse the existing `.metal` kernels and preserve mmap-backed resident +4. Reuse the existing `.metal` kernels and preserve mmap-backed resident loading plus the explicit SSD-streaming path. Keep Objective-C only at the Metal interop boundary. -4. Keep one engine resident at most, matching DwarfStar's instance-lock and +5. Keep one engine resident at most, matching DwarfStar's instance-lock and memory assumptions. -5. Move engine work off the UI thread and support cooperative cancellation at +6. Move engine work off the UI thread and support cooperative cancellation at the safe session boundaries already defined by `ds4_session_sync`. ### DS4 KV-cache port and shared session core @@ -121,7 +123,7 @@ to Rust while preserving its semantics and on-disk compatibility: - Changing the selected model prevents new work from using the old model and unloads it as soon as current work reaches a safe boundary. -### Preferences (implemented) +### Preferences foundation (implemented) - Provide a Preferences panel from the sidebar and the standard macOS `Command-,` shortcut. @@ -132,6 +134,73 @@ to Rust while preserving its semantics and on-disk compatibility: - Persist preferences in the application SQLite database. Changing preferences never loads a model by itself. +### DS4 model and generation preferences (required before chat) + +The Preferences panel must expose every persistent model, runtime, and +generation parameter accepted by the `ds4` command line. `../ds4/ds4_cli.c`, +`ds4.h`, and the shared GPU/distributed parsers are the source of truth; use the +same defaults, ranges, units, dependencies, and model-family overrides rather +than inventing application-specific behavior. + +- **Model and capacity:** selected main GGUF (`--model`), optional MTP/DSpark + support GGUF (`--mtp`), context tokens (`--ctx`), maximum generated tokens + (`--tokens`), and system prompt (`--system`). Managed catalog selections map + to their installed files instead of requiring users to type paths. +- **Sampling and reasoning:** temperature (`--temp`), top-p, min-p, optional + seed, and one reasoning mode representing `--think`, `--think-max`, or + `--nothink`. Preserve DS4's explicit-vs-default distinction because GLM + applies model-family sampling defaults only when the user has not overridden + them. +- **Backend and execution:** backend (`--backend` and its aliases), CPU helper + threads, GPU power percentage, prefill chunk, exact/quality kernels, and warm + weights. Only offer backend/device choices supported by the current build. +- **Speculative decoding:** MTP draft-token count and margin, GLM MTP, GLM MTP + timing, DSpark enablement, DSpark confidence threshold, and DSpark strict + target-only decode. Enabling a dependent control must enable or require its + support artifact exactly as the CLI does. +- **SSD streaming:** enablement, cold start, expert count or GiB cache budget, + fully resident GLM layers, and explicit expert preload count. Retain the + distinction between unset/automatic and an explicit zero where DS4 does. +- **GPU placement:** per-device VRAM budgets, device indices, and CUDA tensor + parallelism when available. Hide or disable these controls in a Metal-only or + CPU-only build rather than accepting values the engine cannot honor. +- **Directional steering:** direction-vector file, FFN scale, and attention + scale, including the CLI rule that a file with no explicit scale defaults the + FFN scale to 1. +- **Advanced diagnostics that change model execution:** simulated used memory + and routed expert profile output. Keep these in an Advanced section with + clear diagnostic labels, but persist and pass them through like their CLI + equivalents. + +Prompt sources, raw one-shot mode, inspect/dump/perplexity/imatrix/self-test +actions, and distributed coordinator/worker network topology are CLI workflows, +not persistent model parameters; they belong to their corresponding chat, +diagnostic, or deployment surfaces rather than Preferences. Any future DS4 CLI +option must be classified explicitly as either a preference or one of these +non-preference actions. + +Implement this before chat as one typed settings path shared by the local GUI, +the lazy engine lifecycle, and HTTP defaults: + +- Persist unset/automatic values separately from explicit values and provide a + Reset to DS4 defaults action. Validate before saving with the same accepted + ranges and cross-field rules as the CLI. +- Mark settings as load-time or turn-time. A load-time change unloads/reloads at + the next safe boundary; sampling, seed, token budget, and reasoning changes + apply to the next turn without duplicating the engine. +- Local chat uses these values directly. HTTP requests use them as defaults and + may override only request-scoped generation fields supported by the API; both + paths still resolve through the same effective-settings builder. +- Add a focused parity test that inventories every option branch in the DS4 CLI + model/runtime, sampling, and steering groups and fails until each option is + mapped to a typed preference or explicitly classified as a non-preference + action. + +Exit criterion: save, restart, and recover every supported DS4 model/runtime +and generation value; show the same effective defaults and validation as the +CLI; and construct one engine configuration plus one turn configuration without +chat or HTTP code re-parsing preference fields. + ### Targeted downloads and storage - The Preferences panel can download the selected model's main GGUF and the @@ -319,11 +388,13 @@ tests, and release packaging remain open. ## Recommended next milestones -1. Implement the single-engine lifecycle and local generation path: lazy load, +1. Complete DS4 CLI model/runtime/generation preference parity and its shared + typed effective-settings builder. +2. Implement the single-engine lifecycle and local generation path: lazy load, shared concurrent acquisition, activity tracking, and idle unload. -2. Put the OpenAI-compatible endpoint on that same engine lifecycle so local +3. Put the OpenAI-compatible endpoint on that same engine lifecycle so local chats and HTTP requests cannot load duplicate engines. -3. Add transcript/KV persistence, then connect the existing conversation UI and +4. Add transcript/KV persistence, then connect the existing conversation UI and finally port the agent tools. -4. Add the opt-in Dev Brain tool after local file/search boundaries and agent +5. Add the opt-in Dev Brain tool after local file/search boundaries and agent approvals are stable.