hugo/DS4Server
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

chore: updated PLAN

Browse Source
This commit is contained in:
Georg Bauer
2026-07-26 22:28:59 +02:00
parent 47b405e72b
commit 21b9223706
2 changed files with 118 additions and 339 deletions
Download Patch File Download Diff File Expand all files Collapse all files

359
PLAN.md
View File

@@ -1,302 +1,87 @@
# DS4Server implementation plan # DS4Server implementation plan
Bundle/application identifier: `DS4Server.rfc1437.de` Only unfinished implementation work belongs here. DwarfStar remains the
behavioral oracle for model execution, token processing, context accounting,
KV-cache behavior, the HTTP API, and the built-in agent loop.
DS4Server is the native macOS Rust/Iced application counterpart to `../ds4`. Numbers in #xx format are issues that can be reaad with tea and have details
The reference implementation remains normative: `ds4.c`/`ds4.h` define model about the respective topic in it.
and session behavior, `ds4_server.c` defines the HTTP API, and `ds4_agent.c`
defines local-agent behavior and tools. Parity means matching observable
behavior and fixtures, not merely exposing similar controls. All planned
execution targets one self-contained Mac.
## Current state ## 1. Fix local-session behavior
- The macOS application shell, projects, session rename/pin/archive/delete, - Store model-dependent generation and runtime preferences per model and
SQLite transcripts, model preferences and downloads, Stats dashboard, native restore them when switching models. (#35)
Application/Edit/Window menu integration, development bundle, and shared lazy - Replace the default system prompt with a concise coding-agent prompt and edit
model lifecycle are implemented. it through a multiline Preferences field. (#30)
- DeepSeek V4 Flash runs through the Rust/Metal executor with full configured - Add the selected project's `AGENTS.md` to the initial prompt when the first
context, ratio-4 sparse attention, cold and resumed batched prefill, prefix user message starts a session. Keep title generation based on the user
continuation, durable KV checkpoints, cancellation, and idle unload. conversation, and reuse a shared prompt KV prefix if measurement justifies
- The configurable localhost endpoint implements all `ds4_server.c` routes, the extra cache lifecycle. (#29)
streaming and non-streaming envelopes, model aliases, CORS, tool replay and - Order ordinary sessions by most recent use while keeping pinned and archived
continuation, malformed DSML recovery, transient prefix-aware KV reuse, sections intact. (#31)
disconnect/queue cancellation, and shared model scheduling. The full - Confirm every session deletion path before deleting metadata and its KV
automated C/Rust differential corpus remains open. checkpoint. (#32)
- Local sessions now match the long-running `ds4_agent.c` baseline: the full - Keep chat pinned to the end only while the user is already there; append new
starting tool set, unlimited tool rounds, queued user guidance between tool streamed content without fighting manual scrolling and remove the current
rounds, session date/time context, periodic tool-contract reminders, jitter. (#34)
cooperative Stop, and explicit activity/failure states are implemented.
- Local tools are hardened for daily use: canonical project boundaries reject
parent and symlink escapes, shell commands receive a deliberate environment,
risky shell and visible-browser actions share one cancellable Allow once/Deny
approval path, and compact tool cards expose bounded parameters, results, and
parsing/approval/queue/run/completion lifecycle state without showing DSML.
Background jobs and bounded output files stop and clean up with Stop, session
switches, and application shutdown.
- Context compaction uses the reference soft and exact token-counted hard
triggers, private live-model summaries, bounded summary and tool-result
retries, a recent verbatim tail, running-job observations, and compatible KV
checkpoints. Every summary is a visible durable history marker carrying its
tail boundary; the full chat remains scrollable, while missing or
incompatible KV state rebuilds from the last marker, its tail, and later
messages. Manual compact is available after new chat following the latest
marker, alongside a checkpoint-discard/rebuild action.
- Focused coverage exercises triggers, summary bounds and sanitizing, tail
selection, queued guidance, checkpoint identity, running jobs, durable
compaction markers, relaunch, and continued tool work after rebuild.
- DeepSeek V4 Flash now supports DS4-compatible SSD expert streaming, legacy
MTP, DSpark, directional steering, detailed native cache telemetry, expert
locality profiling, and a bounded resident multi-session pool in the Rust
executor. Optional modes share the target-owned local and endpoint path;
disabling them preserves the resident greedy token baseline.
- Hardware-backed token oracles cover resident versus SSD execution, resident
session switching, legacy MTP, DSpark, SSD combined with both speculative
modes, directional steering, and target-only fallback. Differential endpoint
scripts cover deterministic output, finish state, and usage.
- GLM 5.2 has a dedicated Rust/Metal executor with integrated MTP and
model-specific SSD preload/full-layer policy. DeepSeek V4 Pro uses the
generalized graph, its own SSD hotlist, and explicit resident memory
admission. Fixture-aware GLM/Pro hardware tests complete the model matrix.
- The native UI is on Iced 0.14. Chat transcripts use its table-aware Markdown
content and viewer path, with a regression for code-styled line-count tables
produced by coding models.
## Delivery order ## 2. Add A2UI local-chat surfaces
1. **Completed:** finish DS4 execution parity: detailed SSD cache telemetry, - Use the public [A2UI protocol](https://a2ui.org/) as the normative reference.
expert profiling, resident multi-session batching, GLM MTP, and the GLM/Pro Implement its versioned, streaming JSON messages and keep `../bDS2` only as
hardware matrix. a reference for the research-oriented components and presentation we want.
2. **Next:** product completion, exhaustive parity verification, and - Define a DS4Server catalog and native Iced renderer for the A2UI Basic Catalog
distribution. plus useful research surfaces such as charts, tables, metrics, timelines,
3. Optional extensions: Dev Brain and A2UI. maps, mind maps, tabs, cards, lists, and forms.
- Support `createSurface`, incremental `updateComponents` and `updateDataModel`
messages, and `deleteSurface`. The model must be able to add, replace, and
update components and data over multiple responses without recreating the
whole surface.
- Persist A2UI messages with the chat and restore the resulting surfaces after
relaunch. Validate messages against the selected catalog, report validation
failures back to the model, and retain a readable fallback in the transcript.
- Implement catalog-declared local functions and agent events. UI-local edits
update the surface immediately; agent events return the action and current
data to the conversation so the model can respond with further A2UI updates.
The existing approval flow applies only if the resulting agent turn invokes
an ordinary side-effecting tool such as shell or file mutation.
## 1. Completed — tool hardening and safety ## 3. Run local sessions independently
Goal: make the existing tool set safe and clear enough for productive daily - Give each active local session its own agent task so users can switch
use without weakening its ability to inspect, edit, build, and test a project. projects, start another session, and continue working while earlier turns are
still active. The shared model owner remains responsible for safe inference
scheduling and resident KV state. (#33)
- Show active state in the session list, route Stop and queued guidance to the
correct session, and isolate failures, tool jobs, approvals, compaction, and
checkpoints between sessions. (#33)
- Confirm application exit while any session still has generation or tool work
in progress. (#33)
- Keep file tools strictly inside the selected project after canonicalization, ## 4. Finish native project and chat controls
including symlinks and nonexistent write targets. Boundary escapes are
rejected, not approved as exceptions.
- Centralize tool approval. Reuse the existing visible-browser approval path
for actions that require consent instead of adding tool-specific modal
systems.
- Require confirmation before destructive shell commands or commands that
affect state outside the project, elevate privileges, launch external apps,
or create unrequested network side effects. Show the exact command and
working directory; support Allow once and Deny. Do not persist broad trust
silently.
- Define the shell environment deliberately so application credentials and
unrelated secrets are not exposed accidentally. Preserve the minimum normal
development environment needed for project commands.
- Render streamed tool calls as compact cards with their real lifecycle:
parsing, awaiting approval, queued, running, completed, failed, or stopped.
Show bounded parameters and results, provide copy/open actions where useful,
and never display raw DSML as ordinary assistant prose.
- Keep background shell jobs cancellable and observable through app stop,
session switches, compaction, and application shutdown. Retain bounded output
files only as long as needed and report cleanup failures.
- Continue enforcing context-aware read/search/bash result limits. Oversized
output must remain available through continuation or a file reference rather
than being inserted wholesale into the prompt.
- Test project-boundary and symlink escapes, destructive-command approval,
denial and cancellation, browser approval, output bounding, and session
switching with a live job.
Exit criterion: use the agent for a real inspect/edit/test cycle while every - Show the active project and Git branch in the chat status area. Add branch
side effect is visible, risky actions require consent, Stop works at every selection and safe branch switching in the project directory. (#23)
stage, and no file tool can escape the selected project. - Add functional File, View, and Help menus; update native menu enabled state
from the focused control; complete undo/redo behavior; and make transcript
text selectable and copyable.
- Explain model/checkpoint mismatches where a session must rebuild context
instead of resuming its existing checkpoint.
## 2. In progress — DS4 execution technology parity ## 5. Add Dev Brain support
Goal: finish the model-independent Metal/runtime capabilities in `ds4.c` - Let the user select one Obsidian vault and give the local agent bounded
before adding larger model families. Every capability must be shared by local Markdown search, read, create, and append operations outside project folders.
chat and the HTTP endpoint through the single process-wide model owner. - Load vault-backed skills on demand and advertise their Markdown entry points
in the initial local-session prompt. (#6)
- Maintain `memory/SOUL.md` and `memory/USER.md` in the vault and include them
in new local-session prompts. (#18)
### 2.1 Implemented — SSD streaming ## 6. Expand agent tools
Flash routed experts stream through the unchanged DS4 Metal kernels with - Add configurable structured-search providers while retaining the current
automatic or explicit cache budgets, cold start, model-specific DS4 hotlists, browser-backed Google search as a provider. (#20)
preload controls, asynchronous I/O, and bounded cache eviction. Resident and - Add local stdio and HTTP MCP server configuration and expose connected MCP
SSD generation share token oracles. Engine atomics expose resident/cache bytes, tools only to local project chat. (#5)
occupancy, preload progress, hits, misses, eviction, buffer reuse, VM advice, - Add a collapsed right sidebar with a project tree, Git status and per-file
direct-read bytes and latency; Stats samples them without inference-thread UI diffs, plus an inventory of built-in tools, MCP servers, and skills. (#8)
work.
SSD streaming is the capacity prerequisite for larger models and therefore
comes before GLM 5.2 and DeepSeek V4 Pro execution.
- Keep non-routed weights resident while loading routed MoE experts from the
GGUF into a bounded in-memory cache on demand.
- Port DS4's automatic cache sizing from the recommended Metal working set,
non-routed weights, KV/context requirements, graph scratch, and routed-prefill
headroom. Preserve explicit expert-count and GiB-budget modes.
- Port cold start, hot-expert preload, explicit preload count, full resident
layer policy, cache eviction, and the overlapped prefill/decode loading path.
- Keep inference correct under cache misses, short reads, cancellation, memory
pressure, unload, and model replacement. A streaming failure must produce a
bounded request error, not partial model state.
- Report resident bytes, cache capacity, hit/miss and load rates, SSD bytes and
latency, preload progress, and memory-pressure decisions in Stats.
- First prove resident-versus-streaming token parity on DeepSeek V4 Flash. Keep
the cache and I/O layer model-aware so later GLM/Pro milestones add policy and
graph support rather than a second streaming subsystem.
### 2.2 Completed for Flash — legacy MTP and DSpark
Both support GGUF formats are validated and executed by Rust. Draft generation,
confidence/margin gates, target verification, accepted-prefix commit, rollback,
strict/quality fallback, cancellation, checkpoint identity, streaming output,
usage accounting, and Stats counters use the existing generation lifecycle.
Sampled decoding remains target-only as in DS4. SSD streaming works with both
modes, and long-chat DSpark prefill has a dedicated hardware regression.
- Load and validate the optional Flash legacy-MTP or DSpark support GGUF without
treating either as a standalone model. Preserve exact support-kind and target
compatibility checks.
- Port draft generation, confidence/margin pruning, target-model verification,
accepted-prefix commit, rejected-suffix rollback/replay, cache counters, and
cancellation boundaries. The target model always owns the output stream.
- Honor MTP draft depth/margin and DSpark confidence/strict settings. Match the
reference rules for greedy, sampled, and quality modes rather than applying
speculation where DS4 falls back to target-only decode.
- Match current DS4 compatibility between SSD streaming and DSpark; remove
temporary Rust-only exclusions where the reference supports the combination.
- Expose accepted/drafted token counts, acceptance rate, verifier cost, and
effective speedup in Stats. Do not call the feature complete merely because
it produces correct tokens; it must also preserve checkpoints, tools,
streaming responses, usage accounting, and Stop behavior.
- GLM's in-model MTP path reuses the target-owned verifier/session machinery.
### 2.3 Completed — remaining Metal execution controls
Directional steering, power throttling, prefill chunking, quality mode, weight
warming, simulated memory pressure, and expert profiling now affect execution.
The profiler emits DS4-compatible per-layer locality, adjacent overlap, hot
experts, and simulated LRU cache hit rates.
- Port directional steering files and exact FFN/attention application,
including DS4 defaults, validation, zero-scale behavior, and checkpoint/model
identity effects.
- Audit every accepted execution preference against the Rust executor. Power,
prefill chunking, quality/exact kernels, warm weights, simulated memory, and
expert profiling must either affect execution as DS4 defines or be disabled
with a clear explanation; persisted no-op settings are not parity.
- Add hardware-backed token/activation fixtures for each mode and keep the
ordinary resident Flash path unchanged when optional features are off.
### 2.4 Completed — single-machine resident sessions
- A bounded resident pool swaps complete KV, logits, speculative state, and
checkpoint ownership between queued local and endpoint sessions. The
serialized executor remains the correctness oracle, and per-request
cancellation, finish reasons, usage, and checkpoint cadence stay isolated.
- Keep all scheduling, model state, KV state, and request handling within the
local process. Networked execution and non-Metal backends are outside the
product scope.
Exit criterion: DeepSeek V4 Flash matches DS4 token output and lifecycle in
resident, SSD-streamed, MTP, DSpark, steering, and batched-server
configurations, with optional modes off producing the same baseline behavior
as today.
## 3. Completed executors — additional-model validation matrix
GLM 5.2 has a dedicated DSA/MLA executor and DeepSeek V4 Pro uses the generalized
DeepSeek graph. Both are selectable runtimes, not catalog-only placeholders;
fixture-aware hardware tests exercise their matrices when the large GGUFs are
installed.
### GLM 5.2
- The GLM DSA/MLA graph covers dense-cache behavior, model-specific tensor and
quantization paths, sampling defaults, reasoning controls, prompt rendering,
and stop tokens.
- The defined GLM tool syntax uses the same durable local-agent loop and exposes
identical behavior through every HTTP route.
- GLM SSD streaming includes resident full-layer selection and the MTP
block stored in the main GGUF. Respect GLM restrictions on power, prefill
chunking, steering, and external support models.
- Resident, streamed, and MTP token paths share a hardware parity fixture.
### DeepSeek V4 Pro
- The generalized Flash graph isolates Pro's differing dimensions, layers,
routed experts, quantization layouts, and output path.
- Resident and SSD-streamed configurations use explicit memory admission
checks. Never begin a load that cannot leave room for the
configured KV/context and graph working set.
- Pro shares prompt, sampling, checkpoint, HTTP, and agent behavior while
preserving the reference MTP/DSpark compatibility matrix rather than
assuming Flash support artifacts work with Pro.
- Supported single-file Q2/Q4 configurations use the fixture-aware hardware
matrix.
Exit criterion: each advertised model passes the same local-agent, checkpoint,
HTTP, SSD-capacity, cancellation, and deterministic token-output matrix as
DeepSeek V4 Flash.
## 4. Product completion and verification
### Reference parity and regression coverage
- Complete the automated black-box C/Rust corpus for Models, Messages, Chat
Completions, Responses, Completions, and OPTIONS in streaming and
non-streaming modes. Cover aliases/defaults, malformed input, context limits,
tools, reasoning, cache reuse, disconnects, queueing, CORS, errors, usage, and
finish reasons.
- Add every new agent/runtime/model capability to both local and endpoint
regression coverage. HTTP conversations remain client-owned and may persist
only opaque content-addressed KV entries, never application transcripts.
- Reuse DS4 prompt, sampling, KV, server, agent, and hardware fixtures. Keep
formatting, Clippy, bundle, and unit tests as commit gates; run hardware
matrices only where the required GGUF and machine are available.
### macOS product and distribution
- Finish the remaining native menu and text behavior: functional File/View/Help
actions, dynamic enabled state, undo/redo parity, and selectable/copyable
transcript text.
- Finish session/model binding and recovery surfaces needed to explain why a
checkpoint is reused or rebuilt. Keep archive as metadata-only and never
delete project contents.
- Add production signing, hardened runtime, required entitlements,
notarization, update delivery, and release automation. Preserve all upstream
DwarfStar/llama.cpp notices for adapted code and kernels.
Exit criterion: a notarized build can be installed on a clean supported Mac,
run the full local-agent and endpoint smoke matrix, restart into its previous
sessions, and update without losing projects, transcripts, models, or KV data.
## 5. Optional future extensions
These are not DS4 baseline parity and must not delay the milestones above.
### Dev Brain
- Let the user opt into one existing Obsidian vault through a native folder
picker and persist a security-scoped bookmark for sandboxed restart access.
- Add one project-independent tool with bounded search, read, and create/append
operations for Markdown notes. Return vault-relative citations and exclude
`.obsidian`, hidden files, deletion, and arbitrary overwrite.
- Reuse the existing file/search implementation with a separate explicit vault
boundary. No vault means no tool and no effect on ordinary sessions.
Exit criterion: store a memory in one project, restart, and retrieve it with a
vault-relative citation from another project without granting broader disk
access.
### A2UI local-chat surfaces
- After the ordinary tool UI is stable, use `../bDS2` as the reference for
typed cards, charts, forms, lists, metrics, mind maps, tables, and tabs.
- Validate and persist versioned structured payloads with their messages.
Render with native Iced/SVG primitives; reject model-authored HTML,
JavaScript, styles, file URLs, and arbitrary commands.
- Keep A2UI local-chat-only until a real endpoint client requires a compatible
protocol. All mutating actions continue through the normal approval path.
Exit criterion: supported surfaces render accessibly, malformed payloads fall
back safely, interactions are validated, and the same surfaces rehydrate after
restart.

98
README.md
View File

@@ -1,58 +1,48 @@
# DS4Server # DS4Server
DS4Server is a native macOS coding-agent application that rewrites the DS4Server is a native macOS application for running DwarfStar models as a local
DwarfStar (`ds4`) inference engine in Rust. It uses Rust and Iced and will combine local model loading, an coding agent and as a localhost inference server. The inference engine, model
OpenAI-compatible localhost endpoint, and project-scoped agent chat in one app. lifecycle, agent loop, and application UI are implemented in Rust with Iced;
the Metal kernels and small native integration layers are adapted from
[DwarfStar (`ds4`)](https://github.com/antirez/ds4).
DS4Server vendors and adapts the Metal kernels, Objective-C Metal glue, and ## Capabilities
visible-Chrome web tool runtime from
[DwarfStar (`ds4`)](https://github.com/antirez/ds4). Their copyright and license
notices are retained in [`native/metal/LICENSE`](native/metal/LICENSE) and
[`native/web/LICENSE`](native/web/LICENSE).
The current milestone provides a Codex-inspired project/session layout. A native - **Project-based coding chat.** Add local project folders, keep multiple
macOS folder picker selects each workspace, then the app asks for its display persistent sessions per project, and rename, retitle, pin, archive, compact,
name. Projects and sessions are persisted through Diesel in SQLite, while rebuild, or delete sessions. Chats stream Markdown, reasoning, tool activity,
settings live in a YAML file beside it that lists only the values you changed context use, and generation speed.
and can be edited by hand. Open Preferences with `Command-,` to configure model, - **Local coding tools.** The agent can read, search, list, write, and edit
generation, runtime, local endpoint, and idle-unload settings. The separate Model Manager project files; run and monitor asynchronous shell commands; search Google;
(`Shift-Command-M`) and visit rendered web pages. File access is confined to the project, output
lists local main and DSpark artifacts, their on-disk sizes and state, and lets is bounded, and risky shell or visible-browser actions require approval.
you download, resume, validate, or delete them. Rust-native background work - **Long-running sessions.** Transcripts and summaries are stored in SQLite.
shows live byte progress, speed, and ETA in Model Manager and the app status Automatic and manual context compaction preserve the complete visible chat,
bar. Stopping or quitting keeps the partial file; the next Download/Resume while durable KV checkpoints make follow-up turns and relaunches resumable.
action continues from that exact byte after relaunch. Exact size and SHA-256 - **Native Metal inference.** DeepSeek V4 Flash, DeepSeek V4 Pro, and GLM 5.2
verification happen before an artifact becomes usable. run through model-specific Rust/Metal executors with long-context prefill,
sampling, cancellation, idle unload, and a bounded resident session pool.
Supported acceleration includes SSD expert streaming, DeepSeek legacy MTP,
DSpark, GLM MTP, directional steering, and model-specific memory admission.
- **Local HTTP APIs.** A configurable listener on `127.0.0.1:4000` exposes
Models, Chat Completions, Completions, Anthropic Messages, and Responses APIs
with streaming, tool calls, usage accounting, cancellation, and prefix-aware
KV reuse. Endpoint conversations remain client-owned and are not added to
project history.
- **Model and runtime management.** Model Manager downloads, resumes, verifies,
and deletes supported GGUF artifacts with byte progress, speed, and ETA.
Preferences cover generation, runtime, SSD, speculative decoding, steering,
KV storage, the local endpoint, and idle unload. The Stats view reports model,
cache, SSD, speculative, session, and HTTP activity.
The selected DeepSeek V4 Flash model can run directly from a project session. Projects, sessions, and transcripts use SQLite. Settings use a compact YAML file
The model, KV/compressor state, 43-layer graph, sampling, and lifecycle are that contains only values changed from their defaults and can also be edited by
owned by Rust; a fixed snapshot of the Objective-C Metal boundary and unchanged hand.
Metal kernels is vendored and built inside this repository. Tokens stream into the chat UI, Stop cancels generation,
follow-up turns reuse durable transcript and KV state, and the model unloads
after the idle timeout. The graph uses the full configured context with the
ratio-4 sparse indexer.
Project chat includes the native `ds4_agent.c` starting tool set: bounded file ## Build and run
read/continuation, write, anchored edit, search, directory listing, asynchronous
shell jobs, Google search, and rendered page visits. Tool calls and results are
persisted as transcript roles and automatically continue the same model turn.
File tools stay inside the selected project. Web tools ask before starting a
visible Chrome profile.
Long sessions compact automatically while retaining the complete scrollable DS4Server targets macOS 13 or newer and requires Rust 1.97 plus
chat. Each compaction appears in history with its durable summary, and a missing `cargo-packager` 0.11.8.
or incompatible KV checkpoint rebuilds from the latest summary and subsequent
chat. An idle session's `…` menu can force compaction after new chat has been
added since the latest marker, or discard its checkpoint for a clean rebuild.
The app also listens on `127.0.0.1:4000` by default for Models, Chat
Completions, Completions, Anthropic Messages, and Responses APIs. The listener,
port, and opt-in CORS are configurable in Preferences. The
endpoint and local chat share the single model owner. External conversations
are client-managed and never enter the project, session, message, or transcript
database; only opaque content-addressed KV cache files are retained. Model
discovery advertises only supported main artifacts that are fully downloaded
and verified on disk.
```sh ```sh
cargo install cargo-packager --locked --version 0.11.8 cargo install cargo-packager --locked --version 0.11.8
@@ -60,7 +50,7 @@ make bundle
open target/release/DS4Server.app open target/release/DS4Server.app
``` ```
State is stored at: ## Application data
```text ```text
~/Library/Application Support/DS4Server.rfc1437.de/data.sqlite3 ~/Library/Application Support/DS4Server.rfc1437.de/data.sqlite3
@@ -68,6 +58,10 @@ State is stored at:
~/Library/Application Support/DS4Server.rfc1437.de/kv-cache/ ~/Library/Application Support/DS4Server.rfc1437.de/kv-cache/
``` ```
Deleting a project or session removes only DS4Server metadata. It never deletes Deleting a project or session removes DS4Server metadata and its associated
the referenced project directory. See [PLAN.md](PLAN.md) for the implementation checkpoint; it never deletes the referenced project directory. See
roadmap. [PLAN.md](PLAN.md) for the remaining implementation work.
The adapted Metal and browser sources retain their upstream notices in
[`native/metal/LICENSE`](native/metal/LICENSE) and
[`native/web/LICENSE`](native/web/LICENSE).