hugo/DS4Server
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

chore: updated PLAN

Browse Source
This commit is contained in:
Georg Bauer
2026-07-26 22:28:59 +02:00
parent 47b405e72b
commit 21b9223706
2 changed files with 118 additions and 339 deletions
Download Patch File Download Diff File Expand all files Collapse all files

359
PLAN.md
View File

@@ -1,302 +1,87 @@
# DS4Server implementation plan
Bundle/application identifier: `DS4Server.rfc1437.de`
Only unfinished implementation work belongs here. DwarfStar remains the
behavioral oracle for model execution, token processing, context accounting,
KV-cache behavior, the HTTP API, and the built-in agent loop.
DS4Server is the native macOS Rust/Iced application counterpart to `../ds4`.
The reference implementation remains normative: `ds4.c`/`ds4.h` define model
and session behavior, `ds4_server.c` defines the HTTP API, and `ds4_agent.c`
defines local-agent behavior and tools. Parity means matching observable
behavior and fixtures, not merely exposing similar controls. All planned
execution targets one self-contained Mac.
Numbers in #xx format are issues that can be reaad with tea and have details
about the respective topic in it.
## Current state
## 1. Fix local-session behavior
- The macOS application shell, projects, session rename/pin/archive/delete,
SQLite transcripts, model preferences and downloads, Stats dashboard, native
Application/Edit/Window menu integration, development bundle, and shared lazy
model lifecycle are implemented.
- DeepSeek V4 Flash runs through the Rust/Metal executor with full configured
context, ratio-4 sparse attention, cold and resumed batched prefill, prefix
continuation, durable KV checkpoints, cancellation, and idle unload.
- The configurable localhost endpoint implements all `ds4_server.c` routes,
streaming and non-streaming envelopes, model aliases, CORS, tool replay and
continuation, malformed DSML recovery, transient prefix-aware KV reuse,
disconnect/queue cancellation, and shared model scheduling. The full
automated C/Rust differential corpus remains open.
- Local sessions now match the long-running `ds4_agent.c` baseline: the full
starting tool set, unlimited tool rounds, queued user guidance between tool
rounds, session date/time context, periodic tool-contract reminders,
cooperative Stop, and explicit activity/failure states are implemented.
- Local tools are hardened for daily use: canonical project boundaries reject
parent and symlink escapes, shell commands receive a deliberate environment,
risky shell and visible-browser actions share one cancellable Allow once/Deny
approval path, and compact tool cards expose bounded parameters, results, and
parsing/approval/queue/run/completion lifecycle state without showing DSML.
Background jobs and bounded output files stop and clean up with Stop, session
switches, and application shutdown.
- Context compaction uses the reference soft and exact token-counted hard
triggers, private live-model summaries, bounded summary and tool-result
retries, a recent verbatim tail, running-job observations, and compatible KV
checkpoints. Every summary is a visible durable history marker carrying its
tail boundary; the full chat remains scrollable, while missing or
incompatible KV state rebuilds from the last marker, its tail, and later
messages. Manual compact is available after new chat following the latest
marker, alongside a checkpoint-discard/rebuild action.
- Focused coverage exercises triggers, summary bounds and sanitizing, tail
selection, queued guidance, checkpoint identity, running jobs, durable
compaction markers, relaunch, and continued tool work after rebuild.
- DeepSeek V4 Flash now supports DS4-compatible SSD expert streaming, legacy
MTP, DSpark, directional steering, detailed native cache telemetry, expert
locality profiling, and a bounded resident multi-session pool in the Rust
executor. Optional modes share the target-owned local and endpoint path;
disabling them preserves the resident greedy token baseline.
- Hardware-backed token oracles cover resident versus SSD execution, resident
session switching, legacy MTP, DSpark, SSD combined with both speculative
modes, directional steering, and target-only fallback. Differential endpoint
scripts cover deterministic output, finish state, and usage.
- GLM 5.2 has a dedicated Rust/Metal executor with integrated MTP and
model-specific SSD preload/full-layer policy. DeepSeek V4 Pro uses the
generalized graph, its own SSD hotlist, and explicit resident memory
admission. Fixture-aware GLM/Pro hardware tests complete the model matrix.
- The native UI is on Iced 0.14. Chat transcripts use its table-aware Markdown
content and viewer path, with a regression for code-styled line-count tables
produced by coding models.
- Store model-dependent generation and runtime preferences per model and
restore them when switching models. (#35)
- Replace the default system prompt with a concise coding-agent prompt and edit
it through a multiline Preferences field. (#30)
- Add the selected project's `AGENTS.md` to the initial prompt when the first
user message starts a session. Keep title generation based on the user
conversation, and reuse a shared prompt KV prefix if measurement justifies
the extra cache lifecycle. (#29)
- Order ordinary sessions by most recent use while keeping pinned and archived
sections intact. (#31)
- Confirm every session deletion path before deleting metadata and its KV
checkpoint. (#32)
- Keep chat pinned to the end only while the user is already there; append new
streamed content without fighting manual scrolling and remove the current
jitter. (#34)
## Delivery order
## 2. Add A2UI local-chat surfaces
1. **Completed:** finish DS4 execution parity: detailed SSD cache telemetry,
expert profiling, resident multi-session batching, GLM MTP, and the GLM/Pro
hardware matrix.
2. **Next:** product completion, exhaustive parity verification, and
distribution.
3. Optional extensions: Dev Brain and A2UI.
- Use the public [A2UI protocol](https://a2ui.org/) as the normative reference.
Implement its versioned, streaming JSON messages and keep `../bDS2` only as
a reference for the research-oriented components and presentation we want.
- Define a DS4Server catalog and native Iced renderer for the A2UI Basic Catalog
plus useful research surfaces such as charts, tables, metrics, timelines,
maps, mind maps, tabs, cards, lists, and forms.
- Support `createSurface`, incremental `updateComponents` and `updateDataModel`
messages, and `deleteSurface`. The model must be able to add, replace, and
update components and data over multiple responses without recreating the
whole surface.
- Persist A2UI messages with the chat and restore the resulting surfaces after
relaunch. Validate messages against the selected catalog, report validation
failures back to the model, and retain a readable fallback in the transcript.
- Implement catalog-declared local functions and agent events. UI-local edits
update the surface immediately; agent events return the action and current
data to the conversation so the model can respond with further A2UI updates.
The existing approval flow applies only if the resulting agent turn invokes
an ordinary side-effecting tool such as shell or file mutation.
## 1. Completed — tool hardening and safety
## 3. Run local sessions independently
Goal: make the existing tool set safe and clear enough for productive daily
use without weakening its ability to inspect, edit, build, and test a project.
- Give each active local session its own agent task so users can switch
projects, start another session, and continue working while earlier turns are
still active. The shared model owner remains responsible for safe inference
scheduling and resident KV state. (#33)
- Show active state in the session list, route Stop and queued guidance to the
correct session, and isolate failures, tool jobs, approvals, compaction, and
checkpoints between sessions. (#33)
- Confirm application exit while any session still has generation or tool work
in progress. (#33)
- Keep file tools strictly inside the selected project after canonicalization,
including symlinks and nonexistent write targets. Boundary escapes are
rejected, not approved as exceptions.
- Centralize tool approval. Reuse the existing visible-browser approval path
for actions that require consent instead of adding tool-specific modal
systems.
- Require confirmation before destructive shell commands or commands that
affect state outside the project, elevate privileges, launch external apps,
or create unrequested network side effects. Show the exact command and
working directory; support Allow once and Deny. Do not persist broad trust
silently.
- Define the shell environment deliberately so application credentials and
unrelated secrets are not exposed accidentally. Preserve the minimum normal
development environment needed for project commands.
- Render streamed tool calls as compact cards with their real lifecycle:
parsing, awaiting approval, queued, running, completed, failed, or stopped.
Show bounded parameters and results, provide copy/open actions where useful,
and never display raw DSML as ordinary assistant prose.
- Keep background shell jobs cancellable and observable through app stop,
session switches, compaction, and application shutdown. Retain bounded output
files only as long as needed and report cleanup failures.
- Continue enforcing context-aware read/search/bash result limits. Oversized
output must remain available through continuation or a file reference rather
than being inserted wholesale into the prompt.
- Test project-boundary and symlink escapes, destructive-command approval,
denial and cancellation, browser approval, output bounding, and session
switching with a live job.
## 4. Finish native project and chat controls
Exit criterion: use the agent for a real inspect/edit/test cycle while every
side effect is visible, risky actions require consent, Stop works at every
stage, and no file tool can escape the selected project.
- Show the active project and Git branch in the chat status area. Add branch
selection and safe branch switching in the project directory. (#23)
- Add functional File, View, and Help menus; update native menu enabled state
from the focused control; complete undo/redo behavior; and make transcript
text selectable and copyable.
- Explain model/checkpoint mismatches where a session must rebuild context
instead of resuming its existing checkpoint.
## 2. In progress — DS4 execution technology parity
## 5. Add Dev Brain support
Goal: finish the model-independent Metal/runtime capabilities in `ds4.c`
before adding larger model families. Every capability must be shared by local
chat and the HTTP endpoint through the single process-wide model owner.
- Let the user select one Obsidian vault and give the local agent bounded
Markdown search, read, create, and append operations outside project folders.
- Load vault-backed skills on demand and advertise their Markdown entry points
in the initial local-session prompt. (#6)
- Maintain `memory/SOUL.md` and `memory/USER.md` in the vault and include them
in new local-session prompts. (#18)
### 2.1 Implemented — SSD streaming
## 6. Expand agent tools
Flash routed experts stream through the unchanged DS4 Metal kernels with
automatic or explicit cache budgets, cold start, model-specific DS4 hotlists,
preload controls, asynchronous I/O, and bounded cache eviction. Resident and
SSD generation share token oracles. Engine atomics expose resident/cache bytes,
occupancy, preload progress, hits, misses, eviction, buffer reuse, VM advice,
direct-read bytes and latency; Stats samples them without inference-thread UI
work.
SSD streaming is the capacity prerequisite for larger models and therefore
comes before GLM 5.2 and DeepSeek V4 Pro execution.
- Keep non-routed weights resident while loading routed MoE experts from the
GGUF into a bounded in-memory cache on demand.
- Port DS4's automatic cache sizing from the recommended Metal working set,
non-routed weights, KV/context requirements, graph scratch, and routed-prefill
headroom. Preserve explicit expert-count and GiB-budget modes.
- Port cold start, hot-expert preload, explicit preload count, full resident
layer policy, cache eviction, and the overlapped prefill/decode loading path.
- Keep inference correct under cache misses, short reads, cancellation, memory
pressure, unload, and model replacement. A streaming failure must produce a
bounded request error, not partial model state.
- Report resident bytes, cache capacity, hit/miss and load rates, SSD bytes and
latency, preload progress, and memory-pressure decisions in Stats.
- First prove resident-versus-streaming token parity on DeepSeek V4 Flash. Keep
the cache and I/O layer model-aware so later GLM/Pro milestones add policy and
graph support rather than a second streaming subsystem.
### 2.2 Completed for Flash — legacy MTP and DSpark
Both support GGUF formats are validated and executed by Rust. Draft generation,
confidence/margin gates, target verification, accepted-prefix commit, rollback,
strict/quality fallback, cancellation, checkpoint identity, streaming output,
usage accounting, and Stats counters use the existing generation lifecycle.
Sampled decoding remains target-only as in DS4. SSD streaming works with both
modes, and long-chat DSpark prefill has a dedicated hardware regression.
- Load and validate the optional Flash legacy-MTP or DSpark support GGUF without
treating either as a standalone model. Preserve exact support-kind and target
compatibility checks.
- Port draft generation, confidence/margin pruning, target-model verification,
accepted-prefix commit, rejected-suffix rollback/replay, cache counters, and
cancellation boundaries. The target model always owns the output stream.
- Honor MTP draft depth/margin and DSpark confidence/strict settings. Match the
reference rules for greedy, sampled, and quality modes rather than applying
speculation where DS4 falls back to target-only decode.
- Match current DS4 compatibility between SSD streaming and DSpark; remove
temporary Rust-only exclusions where the reference supports the combination.
- Expose accepted/drafted token counts, acceptance rate, verifier cost, and
effective speedup in Stats. Do not call the feature complete merely because
it produces correct tokens; it must also preserve checkpoints, tools,
streaming responses, usage accounting, and Stop behavior.
- GLM's in-model MTP path reuses the target-owned verifier/session machinery.
### 2.3 Completed — remaining Metal execution controls
Directional steering, power throttling, prefill chunking, quality mode, weight
warming, simulated memory pressure, and expert profiling now affect execution.
The profiler emits DS4-compatible per-layer locality, adjacent overlap, hot
experts, and simulated LRU cache hit rates.
- Port directional steering files and exact FFN/attention application,
including DS4 defaults, validation, zero-scale behavior, and checkpoint/model
identity effects.
- Audit every accepted execution preference against the Rust executor. Power,
prefill chunking, quality/exact kernels, warm weights, simulated memory, and
expert profiling must either affect execution as DS4 defines or be disabled
with a clear explanation; persisted no-op settings are not parity.
- Add hardware-backed token/activation fixtures for each mode and keep the
ordinary resident Flash path unchanged when optional features are off.
### 2.4 Completed — single-machine resident sessions
- A bounded resident pool swaps complete KV, logits, speculative state, and
checkpoint ownership between queued local and endpoint sessions. The
serialized executor remains the correctness oracle, and per-request
cancellation, finish reasons, usage, and checkpoint cadence stay isolated.
- Keep all scheduling, model state, KV state, and request handling within the
local process. Networked execution and non-Metal backends are outside the
product scope.
Exit criterion: DeepSeek V4 Flash matches DS4 token output and lifecycle in
resident, SSD-streamed, MTP, DSpark, steering, and batched-server
configurations, with optional modes off producing the same baseline behavior
as today.
## 3. Completed executors — additional-model validation matrix
GLM 5.2 has a dedicated DSA/MLA executor and DeepSeek V4 Pro uses the generalized
DeepSeek graph. Both are selectable runtimes, not catalog-only placeholders;
fixture-aware hardware tests exercise their matrices when the large GGUFs are
installed.
### GLM 5.2
- The GLM DSA/MLA graph covers dense-cache behavior, model-specific tensor and
quantization paths, sampling defaults, reasoning controls, prompt rendering,
and stop tokens.
- The defined GLM tool syntax uses the same durable local-agent loop and exposes
identical behavior through every HTTP route.
- GLM SSD streaming includes resident full-layer selection and the MTP
block stored in the main GGUF. Respect GLM restrictions on power, prefill
chunking, steering, and external support models.
- Resident, streamed, and MTP token paths share a hardware parity fixture.
### DeepSeek V4 Pro
- The generalized Flash graph isolates Pro's differing dimensions, layers,
routed experts, quantization layouts, and output path.
- Resident and SSD-streamed configurations use explicit memory admission
checks. Never begin a load that cannot leave room for the
configured KV/context and graph working set.
- Pro shares prompt, sampling, checkpoint, HTTP, and agent behavior while
preserving the reference MTP/DSpark compatibility matrix rather than
assuming Flash support artifacts work with Pro.
- Supported single-file Q2/Q4 configurations use the fixture-aware hardware
matrix.
Exit criterion: each advertised model passes the same local-agent, checkpoint,
HTTP, SSD-capacity, cancellation, and deterministic token-output matrix as
DeepSeek V4 Flash.
## 4. Product completion and verification
### Reference parity and regression coverage
- Complete the automated black-box C/Rust corpus for Models, Messages, Chat
Completions, Responses, Completions, and OPTIONS in streaming and
non-streaming modes. Cover aliases/defaults, malformed input, context limits,
tools, reasoning, cache reuse, disconnects, queueing, CORS, errors, usage, and
finish reasons.
- Add every new agent/runtime/model capability to both local and endpoint
regression coverage. HTTP conversations remain client-owned and may persist
only opaque content-addressed KV entries, never application transcripts.
- Reuse DS4 prompt, sampling, KV, server, agent, and hardware fixtures. Keep
formatting, Clippy, bundle, and unit tests as commit gates; run hardware
matrices only where the required GGUF and machine are available.
### macOS product and distribution
- Finish the remaining native menu and text behavior: functional File/View/Help
actions, dynamic enabled state, undo/redo parity, and selectable/copyable
transcript text.
- Finish session/model binding and recovery surfaces needed to explain why a
checkpoint is reused or rebuilt. Keep archive as metadata-only and never
delete project contents.
- Add production signing, hardened runtime, required entitlements,
notarization, update delivery, and release automation. Preserve all upstream
DwarfStar/llama.cpp notices for adapted code and kernels.
Exit criterion: a notarized build can be installed on a clean supported Mac,
run the full local-agent and endpoint smoke matrix, restart into its previous
sessions, and update without losing projects, transcripts, models, or KV data.
## 5. Optional future extensions
These are not DS4 baseline parity and must not delay the milestones above.
### Dev Brain
- Let the user opt into one existing Obsidian vault through a native folder
picker and persist a security-scoped bookmark for sandboxed restart access.
- Add one project-independent tool with bounded search, read, and create/append
operations for Markdown notes. Return vault-relative citations and exclude
`.obsidian`, hidden files, deletion, and arbitrary overwrite.
- Reuse the existing file/search implementation with a separate explicit vault
boundary. No vault means no tool and no effect on ordinary sessions.
Exit criterion: store a memory in one project, restart, and retrieve it with a
vault-relative citation from another project without granting broader disk
access.
### A2UI local-chat surfaces
- After the ordinary tool UI is stable, use `../bDS2` as the reference for
typed cards, charts, forms, lists, metrics, mind maps, tables, and tabs.
- Validate and persist versioned structured payloads with their messages.
Render with native Iced/SVG primitives; reject model-authored HTML,
JavaScript, styles, file URLs, and arbitrary commands.
- Keep A2UI local-chat-only until a real endpoint client requires a compatible
protocol. All mutating actions continue through the normal approval path.
Exit criterion: supported surfaces render accessibly, malformed payloads fall
back safely, interactions are validated, and the same surfaces rehydrate after
restart.
- Add configurable structured-search providers while retaining the current
browser-backed Google search as a provider. (#20)
- Add local stdio and HTTP MCP server configuration and expose connected MCP
tools only to local project chat. (#5)
- Add a collapsed right sidebar with a project tree, Git status and per-file
diffs, plus an inventory of built-in tools, MCP servers, and skills. (#8)

98
README.md
View File

@@ -1,58 +1,48 @@
# DS4Server
DS4Server is a native macOS coding-agent application that rewrites the
DwarfStar (`ds4`) inference engine in Rust. It uses Rust and Iced and will combine local model loading, an
OpenAI-compatible localhost endpoint, and project-scoped agent chat in one app.
DS4Server is a native macOS application for running DwarfStar models as a local
coding agent and as a localhost inference server. The inference engine, model
lifecycle, agent loop, and application UI are implemented in Rust with Iced;
the Metal kernels and small native integration layers are adapted from
[DwarfStar (`ds4`)](https://github.com/antirez/ds4).
DS4Server vendors and adapts the Metal kernels, Objective-C Metal glue, and
visible-Chrome web tool runtime from
[DwarfStar (`ds4`)](https://github.com/antirez/ds4). Their copyright and license
notices are retained in [`native/metal/LICENSE`](native/metal/LICENSE) and
[`native/web/LICENSE`](native/web/LICENSE).
## Capabilities
The current milestone provides a Codex-inspired project/session layout. A native
macOS folder picker selects each workspace, then the app asks for its display
name. Projects and sessions are persisted through Diesel in SQLite, while
settings live in a YAML file beside it that lists only the values you changed
and can be edited by hand. Open Preferences with `Command-,` to configure model,
generation, runtime, local endpoint, and idle-unload settings. The separate Model Manager
(`Shift-Command-M`)
lists local main and DSpark artifacts, their on-disk sizes and state, and lets
you download, resume, validate, or delete them. Rust-native background work
shows live byte progress, speed, and ETA in Model Manager and the app status
bar. Stopping or quitting keeps the partial file; the next Download/Resume
action continues from that exact byte after relaunch. Exact size and SHA-256
verification happen before an artifact becomes usable.
- **Project-based coding chat.** Add local project folders, keep multiple
persistent sessions per project, and rename, retitle, pin, archive, compact,
rebuild, or delete sessions. Chats stream Markdown, reasoning, tool activity,
context use, and generation speed.
- **Local coding tools.** The agent can read, search, list, write, and edit
project files; run and monitor asynchronous shell commands; search Google;
and visit rendered web pages. File access is confined to the project, output
is bounded, and risky shell or visible-browser actions require approval.
- **Long-running sessions.** Transcripts and summaries are stored in SQLite.
Automatic and manual context compaction preserve the complete visible chat,
while durable KV checkpoints make follow-up turns and relaunches resumable.
- **Native Metal inference.** DeepSeek V4 Flash, DeepSeek V4 Pro, and GLM 5.2
run through model-specific Rust/Metal executors with long-context prefill,
sampling, cancellation, idle unload, and a bounded resident session pool.
Supported acceleration includes SSD expert streaming, DeepSeek legacy MTP,
DSpark, GLM MTP, directional steering, and model-specific memory admission.
- **Local HTTP APIs.** A configurable listener on `127.0.0.1:4000` exposes
Models, Chat Completions, Completions, Anthropic Messages, and Responses APIs
with streaming, tool calls, usage accounting, cancellation, and prefix-aware
KV reuse. Endpoint conversations remain client-owned and are not added to
project history.
- **Model and runtime management.** Model Manager downloads, resumes, verifies,
and deletes supported GGUF artifacts with byte progress, speed, and ETA.
Preferences cover generation, runtime, SSD, speculative decoding, steering,
KV storage, the local endpoint, and idle unload. The Stats view reports model,
cache, SSD, speculative, session, and HTTP activity.
The selected DeepSeek V4 Flash model can run directly from a project session.
The model, KV/compressor state, 43-layer graph, sampling, and lifecycle are
owned by Rust; a fixed snapshot of the Objective-C Metal boundary and unchanged
Metal kernels is vendored and built inside this repository. Tokens stream into the chat UI, Stop cancels generation,
follow-up turns reuse durable transcript and KV state, and the model unloads
after the idle timeout. The graph uses the full configured context with the
ratio-4 sparse indexer.
Projects, sessions, and transcripts use SQLite. Settings use a compact YAML file
that contains only values changed from their defaults and can also be edited by
hand.
Project chat includes the native `ds4_agent.c` starting tool set: bounded file
read/continuation, write, anchored edit, search, directory listing, asynchronous
shell jobs, Google search, and rendered page visits. Tool calls and results are
persisted as transcript roles and automatically continue the same model turn.
File tools stay inside the selected project. Web tools ask before starting a
visible Chrome profile.
## Build and run
Long sessions compact automatically while retaining the complete scrollable
chat. Each compaction appears in history with its durable summary, and a missing
or incompatible KV checkpoint rebuilds from the latest summary and subsequent
chat. An idle session's `…` menu can force compaction after new chat has been
added since the latest marker, or discard its checkpoint for a clean rebuild.
The app also listens on `127.0.0.1:4000` by default for Models, Chat
Completions, Completions, Anthropic Messages, and Responses APIs. The listener,
port, and opt-in CORS are configurable in Preferences. The
endpoint and local chat share the single model owner. External conversations
are client-managed and never enter the project, session, message, or transcript
database; only opaque content-addressed KV cache files are retained. Model
discovery advertises only supported main artifacts that are fully downloaded
and verified on disk.
DS4Server targets macOS 13 or newer and requires Rust 1.97 plus
`cargo-packager` 0.11.8.
```sh
cargo install cargo-packager --locked --version 0.11.8
@@ -60,7 +50,7 @@ make bundle
open target/release/DS4Server.app
```
State is stored at:
## Application data
```text
~/Library/Application Support/DS4Server.rfc1437.de/data.sqlite3
@@ -68,6 +58,10 @@ State is stored at:
~/Library/Application Support/DS4Server.rfc1437.de/kv-cache/
```
Deleting a project or session removes only DS4Server metadata. It never deletes
the referenced project directory. See [PLAN.md](PLAN.md) for the implementation
roadmap.
Deleting a project or session removes DS4Server metadata and its associated
checkpoint; it never deletes the referenced project directory. See
[PLAN.md](PLAN.md) for the remaining implementation work.
The adapted Metal and browser sources retain their upstream notices in
[`native/metal/LICENSE`](native/metal/LICENSE) and
[`native/web/LICENSE`](native/web/LICENSE).