DS4Server/README.md

# DS4Server

DS4Server is a native macOS application for running DwarfStar models as a local
coding agent and as a localhost inference server. The inference engine, model
lifecycle, agent loop, and application UI are implemented in Rust with Iced;
the Metal kernels and small native integration layers are adapted from
[DwarfStar (`ds4`)](https://github.com/antirez/ds4).

## Capabilities

- **Project-based coding chat.** Add local project folders, keep multiple
  persistent sessions per project, and rename, retitle, pin, archive, compact,
  rebuild, or delete sessions. Chats stream Markdown, reasoning, tool activity,
  context use, and generation speed.
- **Local coding tools.** The agent can read, search, list, write, and edit
  project files; run and monitor asynchronous shell commands; search Google;
  and visit rendered web pages. File access is confined to the project, output
  is bounded, risky shell or visible-browser actions require approval, and
  tool calls are validated against the same typed schemas shown to the model.
- **Fresh-agent Ralph loops.** For a bounded coding objective, the agent can
  run foreground rounds in fresh model contexts while preserving workspace
  changes between rounds. A validated structured report is the only
  conversational handoff to the next worker; the loop stops on reported
  completion, a blocker, failure, cancellation, or its configurable round
  limit.
- **Workspace instructions and reusable skills.** A global `AGENTS.md` plus
  project and nested `AGENTS.md`/`AGENTS.local.md` files provide hierarchical
  guidance that is reconciled as the agent enters or changes a workspace area.
  Standard `~/.agents/skills/<name>/SKILL.md` packages are advertised by
  metadata and loaded in full only when a task matches.
- **Project-backed Dev Brain.** An optional managed Obsidian vault provides
  source-verified wiki pages, ranked search, and on-demand skills. Each session
  receives the verified skill names, descriptions, and paths, then loads full
  instructions only when a task matches.
- **Long-running sessions.** Transcripts and summaries are stored in SQLite.
  Automatic and manual context compaction preserve the complete visible chat,
  while durable KV checkpoints make follow-up turns and relaunches resumable.
- **Native Metal inference.** DeepSeek V4 Flash, DeepSeek V4 Pro, and GLM 5.2
  run through model-specific Rust/Metal executors with long-context prefill,
  sampling, cancellation, idle unload, and a bounded resident session pool.
  Supported acceleration includes SSD expert streaming, DeepSeek legacy MTP,
  DSpark, GLM MTP, directional steering, and model-specific memory admission.
- **Local HTTP APIs.** A configurable listener on `127.0.0.1:4000` exposes
  Models, Chat Completions, Completions, Anthropic Messages, and Responses APIs
  with streaming, tool calls, usage accounting, cancellation, and prefix-aware
  KV reuse. Endpoint conversations remain client-owned and are not added to
  project history.
- **Model and runtime management.** Model Manager downloads, resumes, verifies,
  and deletes supported GGUF artifacts with byte progress, speed, and ETA.
  Preferences cover generation, runtime, SSD, speculative decoding, steering,
  KV storage, the local endpoint, and idle unload. The Stats view reports model,
  cache, SSD, speculative, session, and HTTP activity.

Projects, sessions, and transcripts use SQLite. Settings use a compact YAML file
that contains only values changed from their defaults and can also be edited by
hand.

## A2UI widgets

DS4Server renders A2UI v1.0 surfaces with these widgets:

- **Content:** Markdown text, images (including avatars), icons, video and audio
  players, and dividers.
- **Layout:** rows, columns, lists, cards, modals, and tabs.
- **Controls:** buttons, text fields, checkboxes, sliders, date/time inputs, and
  choice pickers.
- **Research:** bar, line, area, stacked-bar, pie, donut, and heatmap charts; tables,
  metrics, timelines, maps, mind maps, and forms.

Video posters render in the surface; video and audio playback uses native
macOS AVKit player windows with standard transport, volume, and fullscreen
controls.

The newest A2UI surface is pinned above the transcript in a default 50/50
workspace split. Dismissing it requires confirmation and records a durable
boundary in the chat database, so reopening the session restores only the
newest surface created after the last dismissal. Left and right arrow buttons
reconstruct earlier and later read-only surface epochs from the recorded A2UI
protocol calls; model updates remain routed only to the current surface.

### Live A2UI validation

With DS4Server running and its local endpoint enabled, run the opt-in model
validation suite with:

```sh
cargo run --release -- validate-a2ui
```

It sends natural-language chart, form, choice, and full-composition prompts
through the current A2UI tool description, then replays every returned protocol
message through the real catalog and component-graph validators. Use `--case
pie-natural`, `--case composed-basics`, `--attempts 3`, `--model MODEL_ID`, or
`--endpoint URL` to narrow or repeat runs. This slow, model-dependent suite is
intentionally separate from `cargo test`.

### Headless model evaluation

Run a bounded, instrumented generation through the production runtime without
opening the UI:

```sh
cargo run --release -- model-eval \
  --model qwen3.8-flash-next \
  --reasoning low \
  --acceleration off \
  --power-percent 100
```

The evaluator first performs a short unmeasured warmup through the same system-prompt
builder and production runtime as the UI, then resets its GPU counters and starts
the measured request. `warmup_result.elapsed_ms` includes initial model loading;
its `timing` object separates `model_load_ms` and `non_loading_elapsed_ms` (the
latter still includes setup, inference, and checkpoint work). Do not add the
reported load time to the warmup total. A `model_loaded` event reports the same
load duration as the shared runtime metrics. Canary samples retain their start
`phase` and record `completion_phase`, actual completion `elapsed_ms`, and host
observation `observed_elapsed_ms`; per-phase summaries group by probe start.
`result.timing.decode_ms` measures the shared engine decode loop, including
sampling and streamed output callbacks but excluding subsequent pending-token
finalization and checkpoint work. Stats shows the same measurement as "Decode
loop (without checkpoint)". The older host-observed interval through request
completion is retained as `decode_to_request_end_ms`; it is not pure decode time.
`decode_loop_tokens_per_second` uses completion tokens divided by the loop duration;
`decode_tokens_per_second` remains the UI's last progress rate. `checkpoint_io`
reports request operation counts and the last read/write duration, not summed I/O
time. A missing decode duration is `null`, not zero.
The default test
prompt identifies a prose-generation test and asks the model to choose
its own genre, setting, and characters, then output only a complete short story
without questions, choices, limitations, or coding discussion. JSON Lines report
runtime phases, prefill and decode speed, Metal busy time,
command-buffer duration, CPU, physical memory, disk I/O, and the complete reasoning
and answer. `--input-file PATH` appends a UTF-8 file to the supplied prompt for
realistic long-prefill tests. Repeat `--prompt` to run multiple turns in one
ongoing chat: every later turn includes all preceding user prompts and complete
assistant responses and continues from the same session checkpoint. An independent
Metal canary runs inside the same
process as the production runtime through
loading, prefill, decode, and finishing, reporting per-phase scheduling and
completion latency plus stalls that could make a UI unresponsive. Use
`--acceleration on|off` to compare DSpark on DeepSeek or integrated MTP on
GLM/Qwen. The canary itself submits GPU work, so use `--canary off` for clean
throughput measurements and `--canary on` for responsiveness diagnostics. The UI
canary is off by default and can be enabled in Stats. A supervisor terminates
the worker only when startup or token progress stalls, or when the memory limit
is exceeded; there is no whole-run timeout. See `model-eval --help` for all
overrides. The evaluator only opens already installed model artifacts and never
downloads missing models.

Evaluation defaults to `--plain-chat on`: the shared UI prompt builder omits
agent/tool schemas, skill instructions and the built-in coding-agent system
prompt; an explicitly customized system prompt is preserved. The corresponding
UI configuration is `plain_chat: true` in its YAML configuration. In that mode
the UI also skips workspace/date injections, extension hooks, A2UI handling,
tool execution, automatic response retries and automatic first-turn titles.
Use a **new chat** in both frontends: existing history is never silently removed.
The normal UI default remains agent mode (`plain_chat: false`). `--plain-chat off`
restores agent prompt construction for diagnostics, but does not make the harness
an agent/tool executor. The effective mode and complete system prompt are reported
in the start receipt; do not compare runs with different modes.

For a summary, story, and Python one-shot in a single chat, pass three prompts:

```sh
target/release/ds4-server model-eval \
  --model qwen3.8-flash-next --reasoning low \
  --power-percent 100 --acceleration off --canary on \
  --input-file README.md \
  --prompt 'Give a concise summary of the following project README. Return the summary directly without asking questions.' \
  --prompt 'This is a prose-generation test. Respond only with a complete short fictional story in ordinary prose. Choose the genre, setting, and characters yourself and begin the story immediately. Do not ask questions, offer choices, explain limitations, or discuss coding.' \
  --prompt 'This is an offline code-generation benchmark. Do not invoke tools or emit tool-call markup. Return exactly one Python code block defining has_close_elements(numbers: list[float], threshold: float) -> bool. It must return True if any two distinct numbers are closer than threshold. Include no explanation outside the code block.'
```

The input file is appended only to the first prompt. Each turn starts after the
previous response finishes; `measurement_start` records the turn number and the
number of history messages (1, 3, 5), and each `result` reports its own timings,
cached tokens, reasoning, and answer. The warmup uses a separate session and is
not included in this history.

## Build and run

DS4Server targets Apple silicon with macOS 15 or newer and requires Rust 1.97 plus
`cargo-packager` 0.11.8.

```sh
cargo install cargo-packager --locked --version 0.11.8
make bundle
open target/release/DS4Server.app
```

Tagged versions are also published as Apple Silicon DMG files on the
[repository releases page](https://git.rfc1437.de/hugo/DS4Server/releases).

## Application data

```text
~/Library/Application Support/de.rfc1437.ds4server/data.sqlite3
~/Library/Application Support/de.rfc1437.ds4server/config.yaml
~/Library/Application Support/de.rfc1437.ds4server/AGENTS.md
~/Library/Application Support/de.rfc1437.ds4server/models/
~/Library/Application Support/de.rfc1437.ds4server/kv-cache/
```

Deleting a project or session removes DS4Server metadata and its associated
checkpoint; it never deletes the referenced project directory. Standard agent
skills live separately under `~/.agents/skills/`. See the
[issue tracker](https://git.rfc1437.de/hugo/DS4Server/issues) for remaining
implementation work.

The adapted Metal and browser sources retain their upstream notices in
[`native/metal/LICENSE`](native/metal/LICENSE) and
[`native/web/LICENSE`](native/web/LICENSE).