hugo/DS4Server
1
0
Fork 0
Code Issues 8 Pull Requests Actions Packages Projects Releases Wiki Activity

Refocus implementation roadmap

Browse Source
This commit is contained in:
Georg Bauer
2026-07-25 22:49:22 +02:00
parent 4a45735072
commit ae4222a573
1 changed files with 277 additions and 553 deletions
Download Patch File Download Diff File Expand all files Collapse all files

830
PLAN.md
View File

@@ -2,609 +2,333 @@
Bundle/application identifier: `DS4Server.rfc1437.de` Bundle/application identifier: `DS4Server.rfc1437.de`
The GUI is a Rust/Iced reimplementation of the user-facing model server and DS4Server is the native macOS Rust/Iced application counterpart to `../ds4`.
coding-agent workflows in `../ds4`. The existing engine remains the behavioral The reference implementation remains normative: `ds4.c`/`ds4.h` define model
reference: `ds4.c`/`ds4.h` define model and session behavior, and session behavior, `ds4_server.c` defines the HTTP API, and `ds4_agent.c`
`ds4_server.c` defines the compatible API, and `ds4_agent.c` defines agent defines local-agent behavior and tools. Parity means matching observable
sessions, tools, and turn orchestration. behavior and fixtures, not merely exposing similar controls. All planned
execution targets one self-contained Mac.
## Current status ## Current state
| Area | Status | Available now | Still open | - The macOS application shell, projects, session rename/pin/archive/delete,
| --- | --- | --- | --- | SQLite transcripts, model preferences and downloads, Stats dashboard, native
| App shell | Implemented | Native multi-window Iced app, Codex-inspired layout, Chat/Stats segmented workspace, native Application/Edit/Window menus, Preferences panel, Model Manager, streaming composer, and an ad-hoc-signed development `.app` bundle | Production signing, hardened runtime, entitlements, notarization, and update delivery | Application/Edit/Window menu integration, development bundle, and shared lazy
| Projects and sessions | Implemented with transcripts and KV resume | Native folder picker, project-name dialog, create/select/delete projects and sessions, durable structured chat history, and per-session KV checkpoints | Rename/archive and model binding | model lifecycle are implemented.
| Persistence | Implemented for local chat | SQLite in Application Support, Diesel ORM, embedded migrations, constrained defaults, streamed message updates, reopen tests, context/speed state, and local KV rehydration | Downloaded-artifact metadata and session-runtime migrations; HTTP chats deliberately never enter SQLite | - DeepSeek V4 Flash runs through the Rust/Metal executor with full configured
| Model runtime | DeepSeek Flash decode and batched prefill implemented | Rust-owned mmap/model/session graph, local Metal kernels, full configured sparse context, exact absolute-position chunk scheduling, cold and resumed layer-major prefill (including unaligned compressor/indexer frontiers), DS4 sampling, one process-wide UI/HTTP owner, cancellation, idle unloading, prefix continuation, KV checkpoint save/load, and lock-free runtime metrics | Remaining KV continuation/rewrite parity, DSpark/SSD/steering, GLM/Pro execution, and sampled per-kernel GPU timing if coarse phase metrics prove insufficient | context, ratio-4 sparse attention, cold and resumed batched prefill, prefix
| Local endpoint | Exact parity in progress | Configurable localhost listener, verified-on-disk model discovery, transient Chat/Completions/Messages/Responses routes, native JSON/SSE envelopes, exact tool schema/call/result replay, live tool continuation, canonical/short/plain DSML recovery, strict Responses replay validation, and differential C/Rust fixtures for tool use, cache accounting, reasoning, cold 6k-token prefill, resumed unaligned prefill, and real Pi tool execution | Remaining malformed-tool repair/error fallbacks, full KV-store policy, disconnect/cancellation, scheduling/lifecycle, and error/context-limit fixtures; enable/disable controls | continuation, durable KV checkpoints, cancellation, and idle unload.
| Agent | Solid local chat implemented | Multi-turn role rendering, structured reasoning disclosure, Markdown, automatic scrolling, context/speed display, durable transcript rehydration, KV-backed resume, and a live Stats dashboard | Tools, approvals, and compaction | - The configurable localhost endpoint implements all `ds4_server.c` routes,
| A2UI | Future extension | bDS2 provides the reference structured render-tool contract | Native inline surfaces for local chat after core chat and tools are stable | streaming and non-streaming envelopes, model aliases, CORS, tool replay and
| Dev brain | Not started | No vault access | Obsidian vault selection, durable access, and agent knowledge/memory tools | continuation, malformed DSML recovery, transient prefix-aware KV reuse,
| Release | Development bundle implemented | Release-mode `.app` packaging and ad-hoc code signing run through `make bundle` as a commit gate | Distribution identity/signing, hardened runtime, entitlements, notarization, update delivery, and release automation | disconnect/queue cancellation, and shared model scheduling. The full
automated C/Rust differential corpus remains open.
- Local sessions persist user, assistant, reasoning, and tool roles. The agent
executes the complete `ds4_agent.c` starting tool set and reinjects the same
model-specific tool contract when sessions are opened or continued.
- The remaining baseline gaps are long-running agent behavior and tool safety.
SSD streaming, speculative decoding, steering, GLM 5.2 execution, and
DeepSeek V4 Pro execution are not implemented in the Rust executor. Some
related catalog, validation, and preference plumbing already exists but must
not be treated as runtime support.
The application manages durable project/session metadata and model preferences. ## Delivery order
Its Model Manager downloads, validates, and deletes managed GGUF artifacts
without blocking the UI. Local chat now lazily acquires the Rust DeepSeek Flash
executor, streams generated text, supports cancellation and multi-turn role
replay, persists reasoning and answers while they stream, rehydrates selected
sessions, and unloads model resources after the configured idle timeout. The
current Rust graph supports the full configured context with ratio-4 sparse
attention, fixed-size cold/resumed batch chunks, unaligned compressor/indexer
continuation, and durable local KV checkpoints. The shared HTTP service exposes
every `ds4_server.c` route and passes the initial differential corpus for native
tool use/continuation, cache accounting, reasoning, and Pi execution. Full
reference parity remains in progress until malformed-output recovery, KV-store,
cancellation, scheduling, lifecycle, and error fixtures are ported.
## Phase 0 — project and session shell 1. **Next:** long-running agent stability and remaining `ds4_agent.c` parity.
2. Tool hardening, approvals, and productive tool presentation.
3. Remaining DS4 execution technology, starting with SSD streaming, then
speculative decoding and the other Metal/runtime parity work.
4. Additional model execution: GLM 5.2 and DeepSeek V4 Pro.
5. Product completion, exhaustive parity verification, and distribution.
6. Optional extensions: Dev Brain and A2UI.
Status: **implemented**. ## 1. Next — long-running agent stability and `ds4_agent.c` parity
- Show projects in a sidebar with their sessions nested underneath. Goal: a local agent session must be able to run long read/edit/test loops,
- Create projects with a native macOS folder picker and a project-name dialog. cross the context limit repeatedly, survive interruption or relaunch, and
- Create, select, and delete projects and sessions. continue with the same durable task state as `ds4-agent`.
- Use a Codex-inspired two-pane layout with embedded, action-specific SVG icons.
- Persist projects and sessions in SQLite under
`~/Library/Application Support/DS4Server.rfc1437.de/`, using Diesel ORM
queries and embedded Diesel migrations.
- Never delete workspace contents when removing app metadata.
Exit criterion: restart the app and see the same project/session tree. ### Context compaction
## Phase 1 — model library and on-demand loading - Port the reference soft trigger: compact before a user turn or tool
continuation at 85% context use, or when at most 8192 tokens remain, with the
free-token threshold capped to one eighth of small contexts.
- Port the hard trigger: before appending a tool result that would leave
insufficient answer room, compact once and retry. If it still does not fit,
return a bounded tool error that tells the model to request less output.
- Use the live model to generate an internal durable task-state summary. The
prompt and generated summary are private compaction work, never ordinary
user/assistant messages, and may not execute tools or retain thinking/DSML
control markup.
- Preserve goals, constraints, files touched, commands and important results,
decisions, known failures, and next steps. Prefer reloadable paths, ranges,
and commands over copying bulky data into the summary.
- Rebuild the model context exactly as the reference does: current system/tool
contract, durable summary, then a recent verbatim tail. Keep up to 10% of the
configured context as the tail, capped at 50000 tokens, and align it to a
user-turn boundary when possible.
- Generate at most the reference summary budget, stop at model control or tool
markers, and never let the private compaction exchange become the reusable
session prefix.
Status: **partially implemented**. The typed preference/download/intake path and ### Durable transition and recovery
a DeepSeek Flash Rust/Metal vertical are implemented, including full configured
context, ratio-4 sparse indexed attention, prefix continuation, KV checkpoint
save/load, DS4 sampling, cancellation, idle unload, and shared HTTP acquisition.
Other model families and optional execution modes remain open.
1. **Implemented:** the typed model/runtime preference contract described below - Treat transcript replacement and the new KV checkpoint as one logical
is complete before chat, the HTTP endpoint, or the engine lifecycle. transition. Persist the compacted semantic transcript and checkpoint
2. **Implemented for DeepSeek Flash:** mmap-backed loading, tokenizer/prompt metadata only after the rebuilt prefix is valid.
rendering, Rust-owned session creation/evaluation/sampling, prefix sync, and - On cancellation, summary failure, prefill failure, or application exit, keep
KV save/load are complete. the previous durable transcript, invalidate any KV state contaminated by the
3. **Implemented:** retain the tested model restrictions from DwarfStar. Main private compaction prompt, and make the next turn rebuild safely.
and DSpark artifacts reject invalid GGUF versions, catalog mismatches, and - Reopen a compacted session with the same summary and recent verbatim turns.
incompatible metadata, tensor shapes, offsets, or quantization before If its checkpoint is absent or incompatible, rebuild it from persisted
promotion and again before model use. messages without changing the visible conversation.
4. **Partially implemented:** reuse the existing `.metal` kernels and preserve mmap-backed resident - Preserve relevant live tool state across compaction. In particular, append a
loading plus the explicit SSD-streaming path. Keep Objective-C only at the compact observation for running shell jobs so the model can still inspect or
Metal interop boundary. stop them after the context rebuild.
5. Keep exactly one model resident process-wide. Local chat and every endpoint - Add a user-visible action equivalent to `/compact`, plus the reference
request must use the same owner and queue; model selection may replace the strip/rebuild behavior: discard a session KV payload without discarding its
resident model only after active work reaches a safe boundary. Multiple transcript, then rebuild on the next use.
simultaneously loaded models are out of scope because DS4Server targets - Bind checkpoint compatibility to the model identity, quantization, context,
models large enough to consume most available memory. rendered transcript, and payload ABI. A model/configuration change must
6. **Implemented for local chat:** move engine work off the UI thread and support cooperative cancellation at rebuild rather than reuse an invalid prefix.
the safe session boundaries already defined by `ds4_session_sync`.
### DS4 KV-cache port and shared session core ### Long-turn behavior
KV-cache behavior is part of the DS4 engine contract, not a new application - Accept user input while an assistant/tool loop is active. Queue it visibly
cache to redesign. Port the existing `ds4_session` and `ds4_kvstore` behavior and inject it after the current tool result, before the next assistant
to Rust while preserving its semantics and on-disk compatibility: continuation, matching `ds4_agent.c` instead of starting a competing turn.
- Preserve the reference rule that there is no arbitrary maximum tool-round
count. Completion, Stop, context pressure, or a real error ends the loop.
- Match the reference date/time context injection and periodic system/tool
prompt reminder so long or reopened sessions do not drift away from the tool
contract.
- Keep Stop cooperative across summarization, compacted-prefix prefill,
generation, and active tool work. A stop must always leave a transcript that
can be reopened.
- A live engine session owns one mutable inference timeline: exact rendered ### Presentation and verification
tokens, logits, and the DS4-specific KV graph state. Callers provide the full
token prefix and the Rust port of `ds4_session_sync` must retain the longest
compatible prefix, evaluate only its suffix, or safely rewind, invalidate,
and rebuild exactly when DS4 does.
- Carry over DS4's safe cancellation/checkpoint boundaries, common-prefix
matching, canonical rewrite and replay, snapshots, payload staging,
save/load, strip-to-transcript behavior, and atomic replacement. The engine
continues to own the opaque KV payload format; UI, HTTP, and persistence code
must not reinterpret tensor state.
- Port `ds4_kvstore` policy rather than replacing it with a generic cache:
exact token and rendered-text prefix lookup, cold and continued aligned
checkpoints, model/quantization/context/payload-ABI compatibility checks,
tool-call replay metadata, disk-budget enforcement, hit tracking, and
eviction behavior.
- Use one process-wide Rust model owner and KV checkpoint implementation for
local chat and the HTTP server. Model access is serialized so two callers
never mutate the single resident inference graph concurrently.
- Application projects and sessions belong exclusively to local chat. HTTP
requests are transient and must never create projects, sessions, messages,
transcript rows, request-identity mappings, or any other durable chat record.
The outside client owns conversation identity and resends the history needed
for each request, exactly as `ds4_server.c` expects.
- Transient HTTP requests may read and write engine-owned KV cache files for
compatible-prefix reuse. Those opaque cache files are the only server-side
persistence permitted for external conversations and do not establish an
application session or recoverable transcript.
- Store engine-owned checkpoints under Application Support in `kv-cache/`.
Local chats use their application session ID; HTTP uses content-addressed
entries under `kv-cache/http/`. The large opaque payload is never stored in
SQLite, and HTTP cache entries do not create application sessions.
- Cache sharing does not merge conversations or expose transcript text. It
shares only the DS4 implementation and compatible opaque prefix checkpoints;
the request body remains the source of truth for external conversation state.
- Engine unload must first leave every reusable live timeline in a valid
persisted checkpoint. Reload restores a compatible checkpoint immediately;
if model identity, quantization, context size, payload ABI, or rendered
history is incompatible, rebuild from the transcript and replace the stale
checkpoint only after the new state is valid.
### Lazy lifecycle - Show explicit `Compacting`, rebuilding/prefill, queued-input, stopped, and
failed states without blocking the Iced event loop. Keep the compacted
summary inspectable without presenting the private prompt as user history.
- Add focused tests for soft and hard triggers, tail selection, tool-result
retry, cancellation rollback, checkpoint invalidation, queued input between
tool rounds, running-job preservation, and reopen after compaction.
- Add a reference fixture that runs a long tool loop through compaction, saves,
relaunches, and continues without losing the active task.
- Start the application and the local HTTP endpoint with no model loaded. Exit criterion: repeat the reference `ds4-agent` long-context scenarios,
- Acquire the configured model only when generation starts, whether initiated including compaction forced by a large tool result, stop during compaction, and
by a local chat or an OpenAI-compatible endpoint request. Concurrent first restart after a successful compaction. The same task state, recent turns, tool
requests share the same load instead of opening the model more than once. contract, and running-job awareness must remain available.
- Expose `Unloaded → Downloading → Loading → Ready → Unloading` plus failure
states and progress in the GUI. A request that triggered loading waits for
the shared load and receives a clear error if download or initialization
fails.
- Treat active local generations, endpoint requests, prefills, and tool turns
as model activity. Start the idle timer only when all activity has stopped.
- Unload the engine after the configured idle timeout. New work cancels a
pending unload; work arriving after unload transparently loads the model
again. Persisted transcripts and KV files remain available when model memory
is released.
- Changing the selected model prevents new work from using the old model and
unloads it as soon as current work reaches a safe boundary.
### Preferences foundation (implemented) ## 2. Tool hardening and safety
- Provide a Preferences panel from the sidebar and the standard Goal: make the existing tool set safe and clear enough for productive daily
macOS `Command-,` shortcut. use without weakening its ability to inspect, edit, build, and test a project.
- Let the user choose the active model. Default to `deepseek-v4-flash`.
- Let the user enable or disable DSpark for the selected model. Disable the
control for models that do not support DSpark.
- Let the user configure the inactivity timeout; default to 10 minutes.
- Let the user configure the localhost endpoint port; default to 4000 and apply
a changed port when preferences are saved.
- Persist preferences in the application SQLite database. Changing preferences
never loads a model by itself.
### DS4 model and generation preferences (implemented) - Keep file tools strictly inside the selected project after canonicalization,
including symlinks and nonexistent write targets. Boundary escapes are
rejected, not approved as exceptions.
- Centralize tool approval. Reuse the existing visible-browser approval path
for actions that require consent instead of adding tool-specific modal
systems.
- Require confirmation before destructive shell commands or commands that
affect state outside the project, elevate privileges, launch external apps,
or create unrequested network side effects. Show the exact command and
working directory; support Allow once and Deny. Do not persist broad trust
silently.
- Define the shell environment deliberately so application credentials and
unrelated secrets are not exposed accidentally. Preserve the minimum normal
development environment needed for project commands.
- Render streamed tool calls as compact cards with their real lifecycle:
parsing, awaiting approval, queued, running, completed, failed, or stopped.
Show bounded parameters and results, provide copy/open actions where useful,
and never display raw DSML as ordinary assistant prose.
- Keep background shell jobs cancellable and observable through app stop,
session switches, compaction, and application shutdown. Retain bounded output
files only as long as needed and report cleanup failures.
- Continue enforcing context-aware read/search/bash result limits. Oversized
output must remain available through continuation or a file reference rather
than being inserted wholesale into the prompt.
- Test project-boundary and symlink escapes, destructive-command approval,
denial and cancellation, browser approval, output bounding, and session
switching with a live job.
The Preferences panel must expose every persistent model, runtime, and Exit criterion: use the agent for a real inspect/edit/test cycle while every
generation parameter accepted by the `ds4` command line. `../ds4/ds4_cli.c`, side effect is visible, risky actions require consent, Stop works at every
`ds4.h`, and the shared GPU/distributed parsers are the source of truth; use the stage, and no file tool can escape the selected project.
same defaults, ranges, units, dependencies, and model-family overrides rather
than inventing application-specific behavior.
- **Model and capacity:** selected main GGUF (`--model`), optional MTP/DSpark ## 3. DS4 execution technology parity
support GGUF (`--mtp`), context tokens (`--ctx`), maximum generated tokens
(`--tokens`), and system prompt (`--system`). Managed catalog selections map
to their installed files instead of requiring users to type paths.
- **Sampling and reasoning:** temperature (`--temp`), top-p, min-p, optional
seed, and one reasoning mode representing `--think`, `--think-max`, or
`--nothink`. Preserve DS4's explicit-vs-default distinction because GLM
applies model-family sampling defaults only when the user has not overridden
them.
- **Execution:** the macOS app uses Metal exclusively; omit the unnecessary CPU
backend and backend selector. Persist CPU helper threads for host-side work,
GPU power percentage, prefill chunk, exact/quality kernels, and warm weights.
- **Speculative decoding:** MTP draft-token count and margin, GLM MTP, GLM MTP
timing, DSpark enablement, DSpark confidence threshold, and DSpark strict
target-only decode. Enabling a dependent control must enable or require its
support artifact exactly as the CLI does.
- **SSD streaming:** enablement, cold start, expert count or GiB cache budget,
fully resident GLM layers, and explicit expert preload count. Retain the
distinction between unset/automatic and an explicit zero where DS4 does.
- **GPU placement:** per-device VRAM budgets, device indices, and CUDA tensor
parallelism when available. Hide or disable these controls in a Metal-only or
CPU-only build rather than accepting values the engine cannot honor.
- **Directional steering:** direction-vector file, FFN scale, and attention
scale, including the CLI rule that a file with no explicit scale defaults the
FFN scale to 1.
- **Advanced diagnostics that change model execution:** simulated used memory
and routed expert profile output. Keep these in an Advanced section with
clear diagnostic labels, but persist and pass them through like their CLI
equivalents.
Prompt sources, raw one-shot mode, inspect/dump/perplexity/imatrix/self-test Goal: finish the model-independent Metal/runtime capabilities in `ds4.c`
actions, and distributed coordinator/worker network topology are CLI workflows, before adding larger model families. Every capability must be shared by local
not persistent model parameters; they belong to their corresponding chat, chat and the HTTP endpoint through the single process-wide model owner.
diagnostic, or deployment surfaces rather than Preferences. Any future DS4 CLI
option must be classified explicitly as either a preference or one of these
non-preference actions.
Implement this before chat as one typed settings path shared by the local GUI, ### 3.1 SSD streaming — first runtime priority
the lazy engine lifecycle, and HTTP defaults:
- Persist unset/automatic values separately from explicit values and provide a SSD streaming is the capacity prerequisite for larger models and therefore
Reset to DS4 defaults action. Validate before saving with the same accepted comes before GLM 5.2 and DeepSeek V4 Pro execution.
ranges and cross-field rules as the CLI.
- Mark settings as load-time or turn-time. A load-time change unloads/reloads at
the next safe boundary; sampling, seed, token budget, and reasoning changes
apply to the next turn without duplicating the engine.
- Local chat uses these values directly. HTTP requests use them as defaults and
may override only request-scoped generation fields supported by the API; both
paths still resolve through the same effective-settings builder.
- Add a focused parity test that inventories every option branch in the DS4 CLI
model/runtime, sampling, and steering groups and fails until each option is
mapped to a typed preference or explicitly classified as a non-preference
action.
Exit criterion: save, restart, and recover every supported DS4 model/runtime - Keep non-routed weights resident while loading routed MoE experts from the
and generation value; show the same effective defaults and validation as the GGUF into a bounded in-memory cache on demand.
CLI; and construct one engine configuration plus one turn configuration without - Port DS4's automatic cache sizing from the recommended Metal working set,
chat or HTTP code re-parsing preference fields. non-routed weights, KV/context requirements, graph scratch, and routed-prefill
headroom. Preserve explicit expert-count and GiB-budget modes.
- Port cold start, hot-expert preload, explicit preload count, full resident
layer policy, cache eviction, and the overlapped prefill/decode loading path.
- Keep inference correct under cache misses, short reads, cancellation, memory
pressure, unload, and model replacement. A streaming failure must produce a
bounded request error, not partial model state.
- Report resident bytes, cache capacity, hit/miss and load rates, SSD bytes and
latency, preload progress, and memory-pressure decisions in Stats.
- First prove resident-versus-streaming token parity on DeepSeek V4 Flash. Keep
the cache and I/O layer model-aware so later GLM/Pro milestones add policy and
graph support rather than a second streaming subsystem.
### Runtime observability (implemented) ### 3.2 Speculative decoding: legacy MTP and DSpark
- Keep instrumentation in the shared model/runtime and HTTP paths so local chat - Load and validate the optional Flash legacy-MTP or DSpark support GGUF without
and endpoint work report through the same counters. treating either as a standalone model. Preserve exact support-kind and target
- Publish hot-path state with relaxed atomics only. Generation and prefill never compatibility checks.
wait for the UI, allocate telemetry events, write logs, or scan the KV cache. - Port draft generation, confidence/margin pruning, target-model verification,
- Sample snapshots in the UI every 200 ms and retain only a bounded in-memory accepted-prefix commit, rejected-suffix rollback/replay, cache counters, and
history for prefill, decode, request-rate, and KV read/write-rate graphs. cancellation boundaries. The target model always owns the output stream.
- Show model phase and lifecycle, prefill/decode throughput, context occupancy, - Honor MTP draft depth/margin and DSpark confidence/strict settings. Match the
request queue and source, exact memory/disk hits, misses, invalid entries, reference rules for greedy, sampled, and quality modes rather than applying
prefix reuse, checkpoint storage and I/O, endpoint traffic and latency, speculation where DS4 falls back to target-only decode.
errors, and model mapping details on the Stats tab. - Match current DS4 compatibility between SSD streaming and DSpark; remove
- Derive KV file sizes when checkpoints are already being saved and scan the temporary Rust-only exclusions where the reference supports the combination.
cache directory only once at startup. Add sampled Metal command-buffer or - Expose accepted/drafted token counts, acceptance rate, verifier cost, and
per-kernel timings later only if coarse phase counters cannot diagnose a effective speedup in Stats. Do not call the feature complete merely because
measured engine bottleneck. it produces correct tokens; it must also preserve checkpoints, tools,
streaming responses, usage accounting, and Stop behavior.
- GLM's in-model MTP path belongs to the GLM milestone, but it should reuse the
verifier/session machinery established here.
Exit criterion: model generation and endpoint traffic visibly update the Stats ### 3.3 Remaining Metal execution controls
dashboard without introducing locks, telemetry I/O, or unbounded history on the
prefill/decode path.
### Targeted downloads and storage - Port directional steering files and exact FFN/attention application,
including DS4 defaults, validation, zero-scale behavior, and checkpoint/model
identity effects.
- Audit every accepted execution preference against the Rust executor. Power,
prefill chunking, quality/exact kernels, warm weights, simulated memory, and
expert profiling must either affect execution as DS4 defines or be disabled
with a clear explanation; persisted no-op settings are not parity.
- Add hardware-backed token/activation fixtures for each mode and keep the
ordinary resident Flash path unchanged when optional features are off.
- Preferences only selects the model/runtime configuration. A separate Model ### 3.4 Single-machine server batching
Manager window, reachable from the native Window menu, lists every managed
main and DSpark artifact with its on-disk size and state, and provides
Download/Resume, Validate, Stop, and confirmed Delete actions.
- A dedicated worker thread performs HTTP and verification work so the Model
Manager and the rest of the application stay interactive.
- Use an in-process Rust HTTP client with Rustls rather than invoking a system
downloader. Stream response chunks directly into a `.part` file and use a
Rust SHA-256 implementation for verification.
- Report aggregate bytes present, total bytes, bytes missing, percentage,
smoothed transfer rate, current artifact/verification phase, and estimated
time remaining. Keep the detailed display in Model Manager and a persistent
status bar with a Stop action while the transfer runs.
- During SHA-256 verification, show a dedicated progress bar based on bytes
read and hashed rather than the already-complete file size. Update the bytes
verified, verification throughput, percentage, and estimated time remaining
throughout the pass so large models never appear stalled at 100%.
- Stop is non-destructive: close the active transfer and retain its `.part`
file. Starting the download again in the same process or after quitting and
relaunching must derive progress from that file and issue an HTTP range
request from its exact byte length. If the server ignores the range request,
safely restart that artifact from byte zero rather than appending corrupt
data.
- Promote a `.part` file only after its exact size and SHA-256 are verified. A
stopped verification keeps the complete partial file so the next run can
verify and promote it without downloading it again.
- Store managed model artifacts under
`~/Library/Application Support/DS4Server.rfc1437.de/models/<model-id>/` and
mmap/load them from there. Do not place managed models in the app bundle or a
project directory.
- Download only the artifact explicitly requested in Model Manager; never
prefetch other catalog models or optional artifacts.
- Remove checksum-invalid partial files without touching a valid model. Keep
partial files after cancellation, application exit, network failure, or an
interrupted verification so they remain restart-resumable.
- Delete model artifacts only through an explicit, confirmed Model Manager
action; changing Preferences must never remove files.
Exit criterion: select a supported DeepSeek V4 or GLM 5.2 GGUF, load it without - Port DS4's resident multi-session batching and server scheduling only after
blocking the UI only when a chat begins, run a deterministic prompt, unload it the serialized path remains the correctness oracle. Preserve per-request
after the configured idle timeout, and pass the matching `../ds4` token-output cancellation, finish reasons, usage, and KV ownership while batching prefill
fixture. Repeat through the local endpoint and verify both paths share the same or decode work.
engine lifecycle. - Keep all scheduling, model state, KV state, and request handling within the
local process. Networked execution and non-Metal backends are outside the
product scope.
## Phase 2 — exact `ds4_server.c` HTTP parity Exit criterion: DeepSeek V4 Flash matches DS4 token output and lifecycle in
resident, SSD-streamed, MTP, DSpark, steering, and batched-server
configurations, with optional modes off producing the same baseline behavior
as today.
Status: **exact parity in progress**. The shared single-model runtime, ## 4. Additional model execution
verified-model discovery, configurable localhost port, all four generation
routes, native JSON/SSE response envelopes, exact batched prefill, tool replay,
and live continuation exist. Canonical, short, and plain DSML tool output plus
loose nested parameters are recovered in final and streaming responses, and
Responses replay rejects unknown/partial state instead of silently dropping
it. Differential C/Rust fixtures cover tools, reasoning, usage/cache
accounting, and real Pi execution. Malformed-tool repair/error fallbacks, full
KV-store, cancellation, scheduling, lifecycle, and error fixture parity remain
open, so no route is considered complete yet.
`../ds4/ds4_server.c` is normative for every externally observable HTTP Start these only after the shared capacity and execution technology above is
behavior. “Compatible” approximations are not acceptable: routes, accepted stable. Catalog entries, settings, tokenizer work, or GGUF validation alone do
fields, defaults, aliases, prompt rendering, status codes, headers, JSON/SSE not make a model selectable for inference.
shapes and ordering, five-second prefill keepalives, reasoning/tool translation,
usage, finish reasons, errors, cancellation, and cache continuation must match
the C server exactly. Port its fixtures alongside each Rust slice and keep every
previous parity fixture passing.
This requirement covers the complete communication behavior, not only the HTTP ### GLM 5.2
wire format. The Rust request path must follow the same request parsing and
validation, model aliases, prompt/token construction, reasoning mode, sampling,
stop matching, tool schema/call/result translation, live tool continuation,
canonical replay, KV lookup/rewrite/checkpoint policy, scheduling, disconnect
cancellation, usage accounting, finish decisions, and model lifecycle invoked
by `ds4_server.c`. `ds4.c` is normative for engine operations reached from the
server. A route is complete only when its differential request corpus passes
against the C server; partial route support stays labeled incomplete.
1. **Partially implemented:** add a localhost-only HTTP service whose lifecycle is independent of the - Port the GLM DSA/MLA graph, dense-cache behavior, model-specific tensor and
engine. It can listen while the model is unloaded and acquires the configured quantization paths, sampling defaults, reasoning controls, prompt rendering,
model only for inference. It listens on configurable port 4000 by default; and stop tokens.
start/stop controls remain open. Never expose a LAN listener without an - Use the already defined GLM tool syntax through the same durable local-agent
explicit setting. loop and expose identical behavior through every HTTP route.
2. **Partially implemented:** expose the complete DwarfStar HTTP surface from - Port GLM SSD streaming policy, resident full-layer selection, and the MTP
`ds4_server.c` in this block stored in the main GGUF. Respect GLM restrictions on power, prefill
order, preserving its request parsing, prompt formatting, reasoning fields, chunking, steering, and external support models.
streaming events, errors, usage accounting, stop behavior, tool replay, and - Validate resident and streamed token output against DS4 fixtures before the
cancellation semantics rather than inventing an app-specific protocol: Model Manager advertises GLM as runnable.
- `GET /v1/models`
- `GET /v1/models/{known-alias}`
- `POST /v1/messages`
- `POST /v1/chat/completions`
- `POST /v1/responses`
- `POST /v1/completions`
- `OPTIONS` with the same CORS behavior when enabled
`GET /v1/models` scans managed artifacts without forcing a model to load and ### DeepSeek V4 Pro
reports only main model files that are fully downloaded and have a matching
on-disk verification marker. Missing, partial, unverified, or checksum-invalid
artifacts are never advertised.
3. **Partially implemented:** streaming SSE, reasoning output, usage accounting,
sampling parameters, tool schemas, and automatic/disabled tool choice work;
exact disconnect cancellation and remaining forced-choice errors are open.
4. **Partially implemented:** port exact sampled tool-call replay and deterministic canonicalization from
`ds4_server.c` so a client's normalized JSON does not destroy KV-prefix
reuse.
5. **Partially implemented:** route API work through the single process-wide model owner and engine-owned
KV store defined in Phase 1. External requests are always stateless at the
application layer: never write their messages to SQLite and never bind a
Responses `conversation`, request identifier, or client-supplied session key
to a local project/session. The client must replay its protocol history;
DS4-compatible prefix lookup may accelerate that replay using opaque KV cache
files. Add resident batching only if one-model serialized execution is later
proven insufficient.
6. **Implemented:** display endpoint address, model, active requests, token
rates, cache behavior, and errors in the GUI Stats dashboard.
Exit criterion: the complete `ds4_server.c` fixture suite and a differential - Generalize the Flash graph only where Pro's dimensions, layers, routed
black-box corpus pass against the Rust app for Models, Anthropic Messages, Chat experts, quantization layouts, or output path actually differ.
Completions, Responses, Completions, and OPTIONS, in streaming and non-streaming - Support resident and SSD-streamed single-machine configurations with explicit
forms where supported. The corpus covers valid requests, aliases/defaults, memory admission checks. Never begin a load that cannot leave room for the
context limits, tools and multi-turn continuation, reasoning, cache reuse, configured KV/context and graph working set.
cancellation/disconnects, queueing, errors, usage, and finish reasons. Verified - Match Pro prompt, sampling, checkpoint, HTTP, and agent behavior. Preserve
installed models are listed without loading one; missing/unverified models are the reference compatibility matrix for MTP/DSpark rather than assuming Flash
absent; local and HTTP generations cannot load two models concurrently; support artifacts work with Pro.
external calls leave no project, session, message, or transcript rows behind; - Validate supported single-file Q2/Q4 configurations against DS4 fixtures
and compatible KV cache files remain reusable. before advertising them as runnable.
## Phase 3 — durable agent sessions Exit criterion: each advertised model passes the same local-agent, checkpoint,
HTTP, SSD-capacity, cancellation, and deterministic token-output matrix as
DeepSeek V4 Flash.
Status: **partially implemented for local chat only**. Project/session metadata, ## 5. Product completion and verification
structured transcripts, context statistics, and per-session KV checkpoints are
durable. Model binding and archive behavior remain open. This phase never
applies to external endpoint conversations.
Extend each metadata-only session with transcript and shared-checkpoint ### Reference parity and regression coverage
metadata:
```text - Complete the automated black-box C/Rust corpus for Models, Messages, Chat
Application Support/DS4Server.rfc1437.de/ Completions, Responses, Completions, and OPTIONS in streaming and
data.sqlite3 non-streaming modes. Cover aliases/defaults, malformed input, context limits,
models/<model-id>/ tools, reasoning, cache reuse, disconnects, queueing, CORS, errors, usage, and
main.gguf finish reasons.
dspark.gguf # only when enabled - Add every new agent/runtime/model capability to both local and endpoint
kv-cache/ regression coverage. HTTP conversations remain client-owned and may persist
<local-session-id>.bin only opaque content-addressed KV entries, never application transcripts.
http/<conversation-tag>.bin - Reuse DS4 prompt, sampling, KV, server, agent, and hardware fixtures. Keep
``` formatting, Clippy, bundle, and unit tests as commit gates; run hardware
matrices only where the required GGUF and machine are available.
- **Implemented:** migrated message tables persist local user text, reasoning ### macOS product and distribution
state, assistant output, context usage, and generation speed while generation
streams. Local session KV payloads remain engine-owned files written with
atomic replacement; SQLite never stores opaque tensor data.
- Record model identity, context size, rendered token history, title,
timestamps, and working directory.
- Restore compatible KV immediately. If KV is absent or incompatible, rebuild
from the transcript and show prefill progress.
- Add strip/archive behavior equivalent to the native agent: retain the
transcript and clear its checkpoint reference. Remove the cache file only
when it is not reusable by another live/session prefix; otherwise normal DS4
disk-budget eviction reclaims it.
- Version formats before the first release and add migrations only when a real
format change exists.
Exit criterion: quit during normal use, relaunch, select a session, and resume - Finish the remaining native menu and text behavior: functional File/View/Help
the same conversation without prefill when its KV payload is compatible. actions, dynamic enabled state, undo/redo parity, and selectable/copyable
## Phase 4 — agent chat and tools
Status: **solid local agent base implemented**. The conversation area streams
multi-turn DeepSeek Flash output, persists and rehydrates structured reasoning
and answers, renders Markdown, follows new output, restores sessions at their
latest message, reports context use and generation speed, and supports Stop.
DeepSeek DSML and GLM tool turns now execute off the UI thread, persist semantic
tool roles, resume the same checkpoint, and expose the complete `ds4_agent.c`
starting tool set. Context compaction and richer partial-tool presentation
remain open.
Every chat feature must persist its semantic state and rehydrate it when a
session is reopened in the same implementation slice. In-memory-only chat
features are incomplete by definition; disclosure state and other purely
ephemeral presentation preferences are the only exception.
1. **Partially implemented:** build the classic chat surface. Transcript,
structured reasoning disclosure, Markdown rendering, composer, automatic
scrolling, streamed output, context/generation-speed status, and Stop are
complete. The shared Stats dashboard provides 200 ms runtime, prefill,
decode, HTTP, and KV telemetry; tool-call cards, queued input, and inline
chat-prefill progress remain. Local turns must call the same session registry
and KV path used by the HTTP endpoint.
2. **Implemented:** port the native agent turn loop from `ds4_agent.c`, including
model-specific system prompts and DeepSeek DSML/GLM tool-call parsing.
3. **Implemented starting set:** implement the local tool set with the project directory as its boundary:
- shell command execution
- file read and continuation
- file write
- anchored edit
- text/file search
- directory listing
- asynchronous shell status/stop
- approved visible-Chrome Google search and rendered page visits
- opt-in Dev Brain knowledge lookup and memory storage remains separate
4. **Partially implemented:** assistant tokens and compact completed tool calls
stream into Iced while inference and tools run on workers. Cooperative Stop
covers generation and tool work; richer partial-call cards remain.
5. **Partially implemented:** file tools enforce the project boundary and web
tools require visible-browser approval. Destructive shell-command approval
remains before broader distribution.
6. The full tool system prompt is derived into every local generation, including
reopened sessions. Port context compaction before prompts approach their
configured context limit.
### Native macOS menus (required with chat)
Status: **partially implemented**. Application, Edit, and Window menus are
installed; Cut, Copy, Paste, and Select All now bridge into the focused Iced
text field. File/View/Help menus, dynamic enabled state, the remaining Edit
actions, and selectable transcript text remain open.
The application must install and maintain a complete native macOS menu bar;
the application-name menu alone is not sufficient. Use platform menu roles,
ordering, selectors, and standard key equivalents rather than drawing menus
inside the Iced window.
- Provide the standard **Application**, **File**, **Edit**, **View**,
**Window**, and **Help** menus. Include the normal macOS Application actions
(About, Settings, Services, Hide, Hide Others, Show All, and Quit) and Window
actions (Minimize, Zoom, Bring All to Front) with native behavior.
- Make File actions operate on real application concepts: New Session, Add/Open
Project, and Close Window. Make View actions cover actual available views,
including sidebar visibility and Enter Full Screen. Help opens the local help
or documentation surface. Do not add permanently inert placeholder items.
- The Edit menu is part of the chat acceptance criteria. Undo, Redo, Cut, Copy,
Paste, Paste and Match Style where supported, Delete, and Select All must act
on the current first responder. They must work in the chat composer and every
other editable text field; Copy and Select All must also work for selectable
transcript text. transcript text.
- Prefer native first-responder selectors for native controls. Where an Iced - Finish session/model binding and recovery surfaces needed to explain why a
widget owns focus, bridge the same menu command into that focused widget so checkpoint is reused or rebuilt. Keep archive as metadata-only and never
menu clicks and `Command-Z`, `Shift-Command-Z`, `Command-X`, `Command-C`, delete project contents.
`Command-V`, and `Command-A` behave identically. - Add production signing, hardened runtime, required entitlements,
- Validate menu enabled state whenever focus or application state changes: notarization, update delivery, and release automation. Preserve all upstream
editing commands reflect the focused selection/clipboard and session/project DwarfStar/llama.cpp notices for adapted code and kernels.
commands reflect whether their action is currently possible. Menu actions
must use the same application messages as toolbar/sidebar actions instead of
duplicating business logic.
- Preserve conventional macOS labels, grouping, keyboard shortcuts, and
accessibility metadata. Keep platform-specific menu integration behind the
macOS application shell so the chat and session core remain platform-neutral.
Exit criterion: with the composer focused, every standard Edit command works Exit criterion: a notarized build can be installed on a clean supported Mac,
from both the menu and its keyboard shortcut, including undo/redo across run the full local-agent and endpoint smoke matrix, restart into its previous
multiple edits; transcript text can be copied; and Application, File, View, sessions, and update without losing projects, transcripts, models, or KV data.
Window, and Help expose functional, correctly enabled native actions.
### Dev Brain — Obsidian knowledge and memory ## 6. Optional future extensions
- Add an optional Dev Brain setting in Preferences. Use the native macOS folder These are not DS4 baseline parity and must not delay the milestones above.
picker to select an existing Obsidian vault, allow the vault to be changed or
disconnected, and persist its canonical path plus a security-scoped bookmark
in SQLite so sandboxed builds can regain access after restart.
- Keep the vault global to the application rather than tied to one project or
session. The selected vault remains in its original location; never copy it
into Application Support or a project directory.
- Add one local `dev_brain` agent tool with the minimum useful operations:
search Markdown notes, read a selected note, and create or append a memory
note. Return vault-relative paths with results so the agent can cite where
knowledge came from.
- Restrict every operation to the explicitly selected vault. Resolve and check
paths before access, ignore `.obsidian` and hidden files, and do not provide
deletion or arbitrary overwrite operations initially.
- Let the agent retrieve relevant global knowledge during a turn and record
durable conclusions after a turn. Vault access is opt-in, does not preload the
model, and must not block sessions when no vault is configured or available.
- Reuse the existing local file/search implementation underneath the new vault
boundary instead of building a second filesystem stack.
Exit criterion: create a project session, ask the model to inspect and edit a ### Dev Brain
small fixture repository, review each tool result in the GUI, restart, and
continue from the persisted session. With Dev Brain enabled, store a memory in
one session and retrieve it with a cited vault-relative note path from a session
in another project after restarting the app.
## Phase 5 — A2UI rich local-chat surfaces - Let the user opt into one existing Obsidian vault through a native folder
picker and persist a security-scoped bookmark for sandboxed restart access.
- Add one project-independent tool with bounded search, read, and create/append
operations for Markdown notes. Return vault-relative citations and exclude
`.obsidian`, hidden files, deletion, and arbitrary overwrite.
- Reuse the existing file/search implementation with a separate explicit vault
boundary. No vault means no tool and no effect on ordinary sessions.
Status: **future extension; not part of the core milestone**. Start only after Exit criterion: store a memory in one project, restart, and retrieve it with a
ordinary local chat, structured tool calls, transcript persistence, and vault-relative citation from another project without granting broader disk
cancellation are stable. Use the A2UI implementation in `../bDS2` as the access.
behavioral reference, especially its render-tool schemas, typed surface
normalization, inline message placement, chart geometry, and interaction state.
- Expose typed `render_card`, `render_chart`, `render_form`, `render_list`, ### A2UI local-chat surfaces
`render_metric`, `render_mindmap`, `render_table`, and `render_tabs` tools to
the local agent. Keep these tools presentation-only; they return structured
data and do not perform project mutations themselves.
- Support the bDS2 chart set initially: bar, stacked bar, line, area, pie,
donut, and heatmap. Render charts and mind maps with native Iced primitives
and SVG geometry inside the assistant message rather than adding a web view.
- Treat tool arguments as untrusted data. Validate type, nesting depth, row and
series counts, numeric values, labels, action names, and payload size before
rendering. Never accept model-authored HTML, JavaScript, styles, file URLs, or
arbitrary commands.
- Attach each validated surface to the assistant message/tool call that created
it and persist the versioned structured payload with the transcript. A
restarted session must reconstruct the same surface without asking the model
to emit it again.
- Build a surface only after its tool arguments are complete and valid while
allowing ordinary assistant text to continue streaming. Malformed or unknown
render payloads fall back to a bounded JSON/text representation and must not
fail the surrounding chat turn.
- Preserve local interaction state for forms and tabs. Dispatch form submits
and card actions through a small allowlist of application actions; any action
that mutates files, settings, or external state still follows the normal
approval path.
- Provide keyboard navigation, readable labels, theme-aware colors, and a text
or tabular fallback for chart data so the information remains accessible and
copyable.
- Keep A2UI local-chat-only initially. Do not extend the OpenAI-compatible HTTP
surface or invent a general plugin/component protocol until a real client
requires it.
Exit criterion: ask the local agent to present fixture data as each supported - After the ordinary tool UI is stable, use `../bDS2` as the reference for
surface type; verify a chart and nested tab surface render correctly, an typed cards, charts, forms, lists, metrics, mind maps, tables, and tabs.
interactive form returns validated values, malformed payloads degrade safely, - Validate and persist versioned structured payloads with their messages.
and all surfaces survive application restart with their message history. Render with native Iced/SVG primitives; reject model-authored HTML,
JavaScript, styles, file URLs, and arbitrary commands.
- Keep A2UI local-chat-only until a real endpoint client requires a compatible
protocol. All mutating actions continue through the normal approval path.
## Verification and packaging Exit criterion: supported surfaces render accessibly, malformed payloads fall
back safely, interactions are validated, and the same surfaces rehydrate after
Status: **partially implemented**. Formatting, Clippy, release build/application restart.
bundle, and unit-test commit gates are documented in `AGENTS.md` and pass.
Database migration/CRUD, preferences, engine scheduling, metrics, request
parsing, tool replay/recovery, and streaming behavior have focused unit
coverage. Live C/Rust endpoint scripts cover tool calls, multi-turn
continuation/cache accounting, reasoning, and real Pi execution. The complete
automated `ds4_server.c` differential corpus and production release signing,
notarization, and delivery remain open.
- Keep unit coverage narrow: persistence round trips, format compatibility,
DS4 KV prefix/rewrite/store/load parity, cross-surface session binding,
request parsing, and tool-boundary checks.
- Reuse `../ds4` fixtures for prompt rendering, sampling, server streaming, and
KV correctness; run live Metal tests only when a supported GGUF is available.
- Preserve the DwarfStar/llama.cpp notices required by `../ds4/LICENSE` for
reused or adapted source and kernels.
- Produce a signed `.app` with the exact identifier above, then add hardened
runtime, entitlements, notarization, and update delivery as release work.
## Recommended next milestones
1. Finish the Flash session core as one larger slice: port the ratio-4 indexer
and indexed attention for full configured context, then add prefix sync and
compatible KV checkpoint save/load. **Implemented.**
2. Complete exact `ds4_server.c` communication parity as one tracked program,
beginning with exact layer-major/chunked/resumed prefill and KV continuation,
then porting every parser/route/state machine and its fixtures. Keep HTTP
conversation state transient except for opaque KV cache files. **In
progress; no route is considered complete until differential tests pass.**
3. Local transcript/KV persistence, the conversation UI, and the native agent
starting tool set are implemented. Add context compaction and richer
partial-tool surfaces next.
4. Add the opt-in Dev Brain tool after local file/search boundaries and agent
approvals are stable.
5. Add bDS2-style A2UI render tools and native inline surfaces after the core
local chat/tool loop is complete.