From 21b92237061d0c5eea72baca4cddf049c27548d7 Mon Sep 17 00:00:00 2001 From: Georg Bauer Date: Sun, 26 Jul 2026 22:28:59 +0200 Subject: [PATCH] chore: updated PLAN --- PLAN.md | 359 +++++++++++------------------------------------------- README.md | 98 +++++++-------- 2 files changed, 118 insertions(+), 339 deletions(-) diff --git a/PLAN.md b/PLAN.md index 11f873d..cec8b62 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1,302 +1,87 @@ # DS4Server implementation plan -Bundle/application identifier: `DS4Server.rfc1437.de` +Only unfinished implementation work belongs here. DwarfStar remains the +behavioral oracle for model execution, token processing, context accounting, +KV-cache behavior, the HTTP API, and the built-in agent loop. -DS4Server is the native macOS Rust/Iced application counterpart to `../ds4`. -The reference implementation remains normative: `ds4.c`/`ds4.h` define model -and session behavior, `ds4_server.c` defines the HTTP API, and `ds4_agent.c` -defines local-agent behavior and tools. Parity means matching observable -behavior and fixtures, not merely exposing similar controls. All planned -execution targets one self-contained Mac. +Numbers in #xx format are issues that can be reaad with tea and have details +about the respective topic in it. -## Current state +## 1. Fix local-session behavior -- The macOS application shell, projects, session rename/pin/archive/delete, - SQLite transcripts, model preferences and downloads, Stats dashboard, native - Application/Edit/Window menu integration, development bundle, and shared lazy - model lifecycle are implemented. -- DeepSeek V4 Flash runs through the Rust/Metal executor with full configured - context, ratio-4 sparse attention, cold and resumed batched prefill, prefix - continuation, durable KV checkpoints, cancellation, and idle unload. -- The configurable localhost endpoint implements all `ds4_server.c` routes, - streaming and non-streaming envelopes, model aliases, CORS, tool replay and - continuation, malformed DSML recovery, transient prefix-aware KV reuse, - disconnect/queue cancellation, and shared model scheduling. The full - automated C/Rust differential corpus remains open. -- Local sessions now match the long-running `ds4_agent.c` baseline: the full - starting tool set, unlimited tool rounds, queued user guidance between tool - rounds, session date/time context, periodic tool-contract reminders, - cooperative Stop, and explicit activity/failure states are implemented. -- Local tools are hardened for daily use: canonical project boundaries reject - parent and symlink escapes, shell commands receive a deliberate environment, - risky shell and visible-browser actions share one cancellable Allow once/Deny - approval path, and compact tool cards expose bounded parameters, results, and - parsing/approval/queue/run/completion lifecycle state without showing DSML. - Background jobs and bounded output files stop and clean up with Stop, session - switches, and application shutdown. -- Context compaction uses the reference soft and exact token-counted hard - triggers, private live-model summaries, bounded summary and tool-result - retries, a recent verbatim tail, running-job observations, and compatible KV - checkpoints. Every summary is a visible durable history marker carrying its - tail boundary; the full chat remains scrollable, while missing or - incompatible KV state rebuilds from the last marker, its tail, and later - messages. Manual compact is available after new chat following the latest - marker, alongside a checkpoint-discard/rebuild action. -- Focused coverage exercises triggers, summary bounds and sanitizing, tail - selection, queued guidance, checkpoint identity, running jobs, durable - compaction markers, relaunch, and continued tool work after rebuild. -- DeepSeek V4 Flash now supports DS4-compatible SSD expert streaming, legacy - MTP, DSpark, directional steering, detailed native cache telemetry, expert - locality profiling, and a bounded resident multi-session pool in the Rust - executor. Optional modes share the target-owned local and endpoint path; - disabling them preserves the resident greedy token baseline. -- Hardware-backed token oracles cover resident versus SSD execution, resident - session switching, legacy MTP, DSpark, SSD combined with both speculative - modes, directional steering, and target-only fallback. Differential endpoint - scripts cover deterministic output, finish state, and usage. -- GLM 5.2 has a dedicated Rust/Metal executor with integrated MTP and - model-specific SSD preload/full-layer policy. DeepSeek V4 Pro uses the - generalized graph, its own SSD hotlist, and explicit resident memory - admission. Fixture-aware GLM/Pro hardware tests complete the model matrix. -- The native UI is on Iced 0.14. Chat transcripts use its table-aware Markdown - content and viewer path, with a regression for code-styled line-count tables - produced by coding models. +- Store model-dependent generation and runtime preferences per model and + restore them when switching models. (#35) +- Replace the default system prompt with a concise coding-agent prompt and edit + it through a multiline Preferences field. (#30) +- Add the selected project's `AGENTS.md` to the initial prompt when the first + user message starts a session. Keep title generation based on the user + conversation, and reuse a shared prompt KV prefix if measurement justifies + the extra cache lifecycle. (#29) +- Order ordinary sessions by most recent use while keeping pinned and archived + sections intact. (#31) +- Confirm every session deletion path before deleting metadata and its KV + checkpoint. (#32) +- Keep chat pinned to the end only while the user is already there; append new + streamed content without fighting manual scrolling and remove the current + jitter. (#34) -## Delivery order +## 2. Add A2UI local-chat surfaces -1. **Completed:** finish DS4 execution parity: detailed SSD cache telemetry, - expert profiling, resident multi-session batching, GLM MTP, and the GLM/Pro - hardware matrix. -2. **Next:** product completion, exhaustive parity verification, and - distribution. -3. Optional extensions: Dev Brain and A2UI. +- Use the public [A2UI protocol](https://a2ui.org/) as the normative reference. + Implement its versioned, streaming JSON messages and keep `../bDS2` only as + a reference for the research-oriented components and presentation we want. +- Define a DS4Server catalog and native Iced renderer for the A2UI Basic Catalog + plus useful research surfaces such as charts, tables, metrics, timelines, + maps, mind maps, tabs, cards, lists, and forms. +- Support `createSurface`, incremental `updateComponents` and `updateDataModel` + messages, and `deleteSurface`. The model must be able to add, replace, and + update components and data over multiple responses without recreating the + whole surface. +- Persist A2UI messages with the chat and restore the resulting surfaces after + relaunch. Validate messages against the selected catalog, report validation + failures back to the model, and retain a readable fallback in the transcript. +- Implement catalog-declared local functions and agent events. UI-local edits + update the surface immediately; agent events return the action and current + data to the conversation so the model can respond with further A2UI updates. + The existing approval flow applies only if the resulting agent turn invokes + an ordinary side-effecting tool such as shell or file mutation. -## 1. Completed — tool hardening and safety +## 3. Run local sessions independently -Goal: make the existing tool set safe and clear enough for productive daily -use without weakening its ability to inspect, edit, build, and test a project. +- Give each active local session its own agent task so users can switch + projects, start another session, and continue working while earlier turns are + still active. The shared model owner remains responsible for safe inference + scheduling and resident KV state. (#33) +- Show active state in the session list, route Stop and queued guidance to the + correct session, and isolate failures, tool jobs, approvals, compaction, and + checkpoints between sessions. (#33) +- Confirm application exit while any session still has generation or tool work + in progress. (#33) -- Keep file tools strictly inside the selected project after canonicalization, - including symlinks and nonexistent write targets. Boundary escapes are - rejected, not approved as exceptions. -- Centralize tool approval. Reuse the existing visible-browser approval path - for actions that require consent instead of adding tool-specific modal - systems. -- Require confirmation before destructive shell commands or commands that - affect state outside the project, elevate privileges, launch external apps, - or create unrequested network side effects. Show the exact command and - working directory; support Allow once and Deny. Do not persist broad trust - silently. -- Define the shell environment deliberately so application credentials and - unrelated secrets are not exposed accidentally. Preserve the minimum normal - development environment needed for project commands. -- Render streamed tool calls as compact cards with their real lifecycle: - parsing, awaiting approval, queued, running, completed, failed, or stopped. - Show bounded parameters and results, provide copy/open actions where useful, - and never display raw DSML as ordinary assistant prose. -- Keep background shell jobs cancellable and observable through app stop, - session switches, compaction, and application shutdown. Retain bounded output - files only as long as needed and report cleanup failures. -- Continue enforcing context-aware read/search/bash result limits. Oversized - output must remain available through continuation or a file reference rather - than being inserted wholesale into the prompt. -- Test project-boundary and symlink escapes, destructive-command approval, - denial and cancellation, browser approval, output bounding, and session - switching with a live job. +## 4. Finish native project and chat controls -Exit criterion: use the agent for a real inspect/edit/test cycle while every -side effect is visible, risky actions require consent, Stop works at every -stage, and no file tool can escape the selected project. +- Show the active project and Git branch in the chat status area. Add branch + selection and safe branch switching in the project directory. (#23) +- Add functional File, View, and Help menus; update native menu enabled state + from the focused control; complete undo/redo behavior; and make transcript + text selectable and copyable. +- Explain model/checkpoint mismatches where a session must rebuild context + instead of resuming its existing checkpoint. -## 2. In progress — DS4 execution technology parity +## 5. Add Dev Brain support -Goal: finish the model-independent Metal/runtime capabilities in `ds4.c` -before adding larger model families. Every capability must be shared by local -chat and the HTTP endpoint through the single process-wide model owner. +- Let the user select one Obsidian vault and give the local agent bounded + Markdown search, read, create, and append operations outside project folders. +- Load vault-backed skills on demand and advertise their Markdown entry points + in the initial local-session prompt. (#6) +- Maintain `memory/SOUL.md` and `memory/USER.md` in the vault and include them + in new local-session prompts. (#18) -### 2.1 Implemented — SSD streaming +## 6. Expand agent tools -Flash routed experts stream through the unchanged DS4 Metal kernels with -automatic or explicit cache budgets, cold start, model-specific DS4 hotlists, -preload controls, asynchronous I/O, and bounded cache eviction. Resident and -SSD generation share token oracles. Engine atomics expose resident/cache bytes, -occupancy, preload progress, hits, misses, eviction, buffer reuse, VM advice, -direct-read bytes and latency; Stats samples them without inference-thread UI -work. - -SSD streaming is the capacity prerequisite for larger models and therefore -comes before GLM 5.2 and DeepSeek V4 Pro execution. - -- Keep non-routed weights resident while loading routed MoE experts from the - GGUF into a bounded in-memory cache on demand. -- Port DS4's automatic cache sizing from the recommended Metal working set, - non-routed weights, KV/context requirements, graph scratch, and routed-prefill - headroom. Preserve explicit expert-count and GiB-budget modes. -- Port cold start, hot-expert preload, explicit preload count, full resident - layer policy, cache eviction, and the overlapped prefill/decode loading path. -- Keep inference correct under cache misses, short reads, cancellation, memory - pressure, unload, and model replacement. A streaming failure must produce a - bounded request error, not partial model state. -- Report resident bytes, cache capacity, hit/miss and load rates, SSD bytes and - latency, preload progress, and memory-pressure decisions in Stats. -- First prove resident-versus-streaming token parity on DeepSeek V4 Flash. Keep - the cache and I/O layer model-aware so later GLM/Pro milestones add policy and - graph support rather than a second streaming subsystem. - -### 2.2 Completed for Flash — legacy MTP and DSpark - -Both support GGUF formats are validated and executed by Rust. Draft generation, -confidence/margin gates, target verification, accepted-prefix commit, rollback, -strict/quality fallback, cancellation, checkpoint identity, streaming output, -usage accounting, and Stats counters use the existing generation lifecycle. -Sampled decoding remains target-only as in DS4. SSD streaming works with both -modes, and long-chat DSpark prefill has a dedicated hardware regression. - -- Load and validate the optional Flash legacy-MTP or DSpark support GGUF without - treating either as a standalone model. Preserve exact support-kind and target - compatibility checks. -- Port draft generation, confidence/margin pruning, target-model verification, - accepted-prefix commit, rejected-suffix rollback/replay, cache counters, and - cancellation boundaries. The target model always owns the output stream. -- Honor MTP draft depth/margin and DSpark confidence/strict settings. Match the - reference rules for greedy, sampled, and quality modes rather than applying - speculation where DS4 falls back to target-only decode. -- Match current DS4 compatibility between SSD streaming and DSpark; remove - temporary Rust-only exclusions where the reference supports the combination. -- Expose accepted/drafted token counts, acceptance rate, verifier cost, and - effective speedup in Stats. Do not call the feature complete merely because - it produces correct tokens; it must also preserve checkpoints, tools, - streaming responses, usage accounting, and Stop behavior. -- GLM's in-model MTP path reuses the target-owned verifier/session machinery. - -### 2.3 Completed — remaining Metal execution controls - -Directional steering, power throttling, prefill chunking, quality mode, weight -warming, simulated memory pressure, and expert profiling now affect execution. -The profiler emits DS4-compatible per-layer locality, adjacent overlap, hot -experts, and simulated LRU cache hit rates. - -- Port directional steering files and exact FFN/attention application, - including DS4 defaults, validation, zero-scale behavior, and checkpoint/model - identity effects. -- Audit every accepted execution preference against the Rust executor. Power, - prefill chunking, quality/exact kernels, warm weights, simulated memory, and - expert profiling must either affect execution as DS4 defines or be disabled - with a clear explanation; persisted no-op settings are not parity. -- Add hardware-backed token/activation fixtures for each mode and keep the - ordinary resident Flash path unchanged when optional features are off. - -### 2.4 Completed — single-machine resident sessions - -- A bounded resident pool swaps complete KV, logits, speculative state, and - checkpoint ownership between queued local and endpoint sessions. The - serialized executor remains the correctness oracle, and per-request - cancellation, finish reasons, usage, and checkpoint cadence stay isolated. -- Keep all scheduling, model state, KV state, and request handling within the - local process. Networked execution and non-Metal backends are outside the - product scope. - -Exit criterion: DeepSeek V4 Flash matches DS4 token output and lifecycle in -resident, SSD-streamed, MTP, DSpark, steering, and batched-server -configurations, with optional modes off producing the same baseline behavior -as today. - -## 3. Completed executors — additional-model validation matrix - -GLM 5.2 has a dedicated DSA/MLA executor and DeepSeek V4 Pro uses the generalized -DeepSeek graph. Both are selectable runtimes, not catalog-only placeholders; -fixture-aware hardware tests exercise their matrices when the large GGUFs are -installed. - -### GLM 5.2 - -- The GLM DSA/MLA graph covers dense-cache behavior, model-specific tensor and - quantization paths, sampling defaults, reasoning controls, prompt rendering, - and stop tokens. -- The defined GLM tool syntax uses the same durable local-agent loop and exposes - identical behavior through every HTTP route. -- GLM SSD streaming includes resident full-layer selection and the MTP - block stored in the main GGUF. Respect GLM restrictions on power, prefill - chunking, steering, and external support models. -- Resident, streamed, and MTP token paths share a hardware parity fixture. - -### DeepSeek V4 Pro - -- The generalized Flash graph isolates Pro's differing dimensions, layers, - routed experts, quantization layouts, and output path. -- Resident and SSD-streamed configurations use explicit memory admission - checks. Never begin a load that cannot leave room for the - configured KV/context and graph working set. -- Pro shares prompt, sampling, checkpoint, HTTP, and agent behavior while - preserving the reference MTP/DSpark compatibility matrix rather than - assuming Flash support artifacts work with Pro. -- Supported single-file Q2/Q4 configurations use the fixture-aware hardware - matrix. - -Exit criterion: each advertised model passes the same local-agent, checkpoint, -HTTP, SSD-capacity, cancellation, and deterministic token-output matrix as -DeepSeek V4 Flash. - -## 4. Product completion and verification - -### Reference parity and regression coverage - -- Complete the automated black-box C/Rust corpus for Models, Messages, Chat - Completions, Responses, Completions, and OPTIONS in streaming and - non-streaming modes. Cover aliases/defaults, malformed input, context limits, - tools, reasoning, cache reuse, disconnects, queueing, CORS, errors, usage, and - finish reasons. -- Add every new agent/runtime/model capability to both local and endpoint - regression coverage. HTTP conversations remain client-owned and may persist - only opaque content-addressed KV entries, never application transcripts. -- Reuse DS4 prompt, sampling, KV, server, agent, and hardware fixtures. Keep - formatting, Clippy, bundle, and unit tests as commit gates; run hardware - matrices only where the required GGUF and machine are available. - -### macOS product and distribution - -- Finish the remaining native menu and text behavior: functional File/View/Help - actions, dynamic enabled state, undo/redo parity, and selectable/copyable - transcript text. -- Finish session/model binding and recovery surfaces needed to explain why a - checkpoint is reused or rebuilt. Keep archive as metadata-only and never - delete project contents. -- Add production signing, hardened runtime, required entitlements, - notarization, update delivery, and release automation. Preserve all upstream - DwarfStar/llama.cpp notices for adapted code and kernels. - -Exit criterion: a notarized build can be installed on a clean supported Mac, -run the full local-agent and endpoint smoke matrix, restart into its previous -sessions, and update without losing projects, transcripts, models, or KV data. - -## 5. Optional future extensions - -These are not DS4 baseline parity and must not delay the milestones above. - -### Dev Brain - -- Let the user opt into one existing Obsidian vault through a native folder - picker and persist a security-scoped bookmark for sandboxed restart access. -- Add one project-independent tool with bounded search, read, and create/append - operations for Markdown notes. Return vault-relative citations and exclude - `.obsidian`, hidden files, deletion, and arbitrary overwrite. -- Reuse the existing file/search implementation with a separate explicit vault - boundary. No vault means no tool and no effect on ordinary sessions. - -Exit criterion: store a memory in one project, restart, and retrieve it with a -vault-relative citation from another project without granting broader disk -access. - -### A2UI local-chat surfaces - -- After the ordinary tool UI is stable, use `../bDS2` as the reference for - typed cards, charts, forms, lists, metrics, mind maps, tables, and tabs. -- Validate and persist versioned structured payloads with their messages. - Render with native Iced/SVG primitives; reject model-authored HTML, - JavaScript, styles, file URLs, and arbitrary commands. -- Keep A2UI local-chat-only until a real endpoint client requires a compatible - protocol. All mutating actions continue through the normal approval path. - -Exit criterion: supported surfaces render accessibly, malformed payloads fall -back safely, interactions are validated, and the same surfaces rehydrate after -restart. +- Add configurable structured-search providers while retaining the current + browser-backed Google search as a provider. (#20) +- Add local stdio and HTTP MCP server configuration and expose connected MCP + tools only to local project chat. (#5) +- Add a collapsed right sidebar with a project tree, Git status and per-file + diffs, plus an inventory of built-in tools, MCP servers, and skills. (#8) diff --git a/README.md b/README.md index a27bf65..67b7c6b 100644 --- a/README.md +++ b/README.md @@ -1,58 +1,48 @@ # DS4Server -DS4Server is a native macOS coding-agent application that rewrites the -DwarfStar (`ds4`) inference engine in Rust. It uses Rust and Iced and will combine local model loading, an -OpenAI-compatible localhost endpoint, and project-scoped agent chat in one app. +DS4Server is a native macOS application for running DwarfStar models as a local +coding agent and as a localhost inference server. The inference engine, model +lifecycle, agent loop, and application UI are implemented in Rust with Iced; +the Metal kernels and small native integration layers are adapted from +[DwarfStar (`ds4`)](https://github.com/antirez/ds4). -DS4Server vendors and adapts the Metal kernels, Objective-C Metal glue, and -visible-Chrome web tool runtime from -[DwarfStar (`ds4`)](https://github.com/antirez/ds4). Their copyright and license -notices are retained in [`native/metal/LICENSE`](native/metal/LICENSE) and -[`native/web/LICENSE`](native/web/LICENSE). +## Capabilities -The current milestone provides a Codex-inspired project/session layout. A native -macOS folder picker selects each workspace, then the app asks for its display -name. Projects and sessions are persisted through Diesel in SQLite, while -settings live in a YAML file beside it that lists only the values you changed -and can be edited by hand. Open Preferences with `Command-,` to configure model, -generation, runtime, local endpoint, and idle-unload settings. The separate Model Manager -(`Shift-Command-M`) -lists local main and DSpark artifacts, their on-disk sizes and state, and lets -you download, resume, validate, or delete them. Rust-native background work -shows live byte progress, speed, and ETA in Model Manager and the app status -bar. Stopping or quitting keeps the partial file; the next Download/Resume -action continues from that exact byte after relaunch. Exact size and SHA-256 -verification happen before an artifact becomes usable. +- **Project-based coding chat.** Add local project folders, keep multiple + persistent sessions per project, and rename, retitle, pin, archive, compact, + rebuild, or delete sessions. Chats stream Markdown, reasoning, tool activity, + context use, and generation speed. +- **Local coding tools.** The agent can read, search, list, write, and edit + project files; run and monitor asynchronous shell commands; search Google; + and visit rendered web pages. File access is confined to the project, output + is bounded, and risky shell or visible-browser actions require approval. +- **Long-running sessions.** Transcripts and summaries are stored in SQLite. + Automatic and manual context compaction preserve the complete visible chat, + while durable KV checkpoints make follow-up turns and relaunches resumable. +- **Native Metal inference.** DeepSeek V4 Flash, DeepSeek V4 Pro, and GLM 5.2 + run through model-specific Rust/Metal executors with long-context prefill, + sampling, cancellation, idle unload, and a bounded resident session pool. + Supported acceleration includes SSD expert streaming, DeepSeek legacy MTP, + DSpark, GLM MTP, directional steering, and model-specific memory admission. +- **Local HTTP APIs.** A configurable listener on `127.0.0.1:4000` exposes + Models, Chat Completions, Completions, Anthropic Messages, and Responses APIs + with streaming, tool calls, usage accounting, cancellation, and prefix-aware + KV reuse. Endpoint conversations remain client-owned and are not added to + project history. +- **Model and runtime management.** Model Manager downloads, resumes, verifies, + and deletes supported GGUF artifacts with byte progress, speed, and ETA. + Preferences cover generation, runtime, SSD, speculative decoding, steering, + KV storage, the local endpoint, and idle unload. The Stats view reports model, + cache, SSD, speculative, session, and HTTP activity. -The selected DeepSeek V4 Flash model can run directly from a project session. -The model, KV/compressor state, 43-layer graph, sampling, and lifecycle are -owned by Rust; a fixed snapshot of the Objective-C Metal boundary and unchanged -Metal kernels is vendored and built inside this repository. Tokens stream into the chat UI, Stop cancels generation, -follow-up turns reuse durable transcript and KV state, and the model unloads -after the idle timeout. The graph uses the full configured context with the -ratio-4 sparse indexer. +Projects, sessions, and transcripts use SQLite. Settings use a compact YAML file +that contains only values changed from their defaults and can also be edited by +hand. -Project chat includes the native `ds4_agent.c` starting tool set: bounded file -read/continuation, write, anchored edit, search, directory listing, asynchronous -shell jobs, Google search, and rendered page visits. Tool calls and results are -persisted as transcript roles and automatically continue the same model turn. -File tools stay inside the selected project. Web tools ask before starting a -visible Chrome profile. +## Build and run -Long sessions compact automatically while retaining the complete scrollable -chat. Each compaction appears in history with its durable summary, and a missing -or incompatible KV checkpoint rebuilds from the latest summary and subsequent -chat. An idle session's `…` menu can force compaction after new chat has been -added since the latest marker, or discard its checkpoint for a clean rebuild. - -The app also listens on `127.0.0.1:4000` by default for Models, Chat -Completions, Completions, Anthropic Messages, and Responses APIs. The listener, -port, and opt-in CORS are configurable in Preferences. The -endpoint and local chat share the single model owner. External conversations -are client-managed and never enter the project, session, message, or transcript -database; only opaque content-addressed KV cache files are retained. Model -discovery advertises only supported main artifacts that are fully downloaded -and verified on disk. +DS4Server targets macOS 13 or newer and requires Rust 1.97 plus +`cargo-packager` 0.11.8. ```sh cargo install cargo-packager --locked --version 0.11.8 @@ -60,7 +50,7 @@ make bundle open target/release/DS4Server.app ``` -State is stored at: +## Application data ```text ~/Library/Application Support/DS4Server.rfc1437.de/data.sqlite3 @@ -68,6 +58,10 @@ State is stored at: ~/Library/Application Support/DS4Server.rfc1437.de/kv-cache/ ``` -Deleting a project or session removes only DS4Server metadata. It never deletes -the referenced project directory. See [PLAN.md](PLAN.md) for the implementation -roadmap. +Deleting a project or session removes DS4Server metadata and its associated +checkpoint; it never deletes the referenced project directory. See +[PLAN.md](PLAN.md) for the remaining implementation work. + +The adapted Metal and browser sources retain their upstream notices in +[`native/metal/LICENSE`](native/metal/LICENSE) and +[`native/web/LICENSE`](native/web/LICENSE).