DS4Server

DS4Server is a native macOS application for running DwarfStar models as a local coding agent and as a localhost inference server. The inference engine, model lifecycle, agent loop, and application UI are implemented in Rust with Iced; the Metal kernels and small native integration layers are adapted from DwarfStar (ds4).

Capabilities

Project-based coding chat. Add local project folders, keep multiple persistent sessions per project, and rename, retitle, pin, archive, compact, rebuild, or delete sessions. Chats stream Markdown, reasoning, tool activity, context use, and generation speed.
Local coding tools. The agent can read, search, list, write, and edit project files; run and monitor asynchronous shell commands; search Google; and visit rendered web pages. File access is confined to the project, output is bounded, risky shell or visible-browser actions require approval, and tool calls are validated against the same typed schemas shown to the model.
Fresh-agent Ralph loops. For a bounded coding objective, the agent can run foreground rounds in fresh model contexts while preserving workspace changes between rounds. A validated structured report is the only conversational handoff to the next worker; the loop stops on reported completion, a blocker, failure, cancellation, or its configurable round limit.
Workspace instructions and reusable skills. A global AGENTS.md plus project and nested AGENTS.md/AGENTS.local.md files provide hierarchical guidance that is reconciled as the agent enters or changes a workspace area. Standard ~/.agents/skills/<name>/SKILL.md packages are advertised by metadata and loaded in full only when a task matches.
Project-backed Dev Brain. An optional managed Obsidian vault provides source-verified wiki pages, ranked search, and on-demand skills. Each session receives the verified skill names, descriptions, and paths, then loads full instructions only when a task matches.
Long-running sessions. Transcripts and summaries are stored in SQLite. Automatic and manual context compaction preserve the complete visible chat, while durable KV checkpoints make follow-up turns and relaunches resumable.
Native Metal inference. DeepSeek V4 Flash, DeepSeek V4 Pro, and GLM 5.2 run through model-specific Rust/Metal executors with long-context prefill, sampling, cancellation, idle unload, and a bounded resident session pool. Supported acceleration includes SSD expert streaming, DeepSeek legacy MTP, DSpark, GLM MTP, directional steering, and model-specific memory admission.
Local HTTP APIs. A configurable listener on 127.0.0.1:4000 exposes Models, Chat Completions, Completions, Anthropic Messages, and Responses APIs with streaming, tool calls, usage accounting, cancellation, and prefix-aware KV reuse. Endpoint conversations remain client-owned and are not added to project history.
Model and runtime management. Model Manager downloads, resumes, verifies, and deletes supported GGUF artifacts with byte progress, speed, and ETA. Preferences cover generation, runtime, SSD, speculative decoding, steering, KV storage, the local endpoint, and idle unload. The Stats view reports model, cache, SSD, speculative, session, and HTTP activity.

Projects, sessions, and transcripts use SQLite. Settings use a compact YAML file that contains only values changed from their defaults and can also be edited by hand.

A2UI widgets

DS4Server renders A2UI v1.0 surfaces with these widgets:

Content: Markdown text, images (including avatars), icons, video and audio players, and dividers.
Layout: rows, columns, lists, cards, modals, and tabs.
Controls: buttons, text fields, checkboxes, sliders, date/time inputs, and choice pickers.
Research: bar, line, area, stacked-bar, pie, donut, and heatmap charts; tables, metrics, timelines, maps, mind maps, and forms.

Video posters render in the surface; video and audio playback uses native macOS AVKit player windows with standard transport, volume, and fullscreen controls.

The newest A2UI surface is pinned above the transcript in a default 50/50 workspace split. Dismissing it requires confirmation and records a durable boundary in the chat database, so reopening the session restores only the newest surface created after the last dismissal. Left and right arrow buttons reconstruct earlier and later read-only surface epochs from the recorded A2UI protocol calls; model updates remain routed only to the current surface.

Live A2UI validation

With DS4Server running and its local endpoint enabled, run the opt-in model validation suite with:

cargo run --release -- validate-a2ui

It sends natural-language chart, form, choice, and full-composition prompts through the current A2UI tool description, then replays every returned protocol message through the real catalog and component-graph validators. Use --case pie-natural, --case composed-basics, --attempts 3, --model MODEL_ID, or --endpoint URL to narrow or repeat runs. This slow, model-dependent suite is intentionally separate from cargo test.

Headless model evaluation

Run a bounded, instrumented generation through the production runtime without opening the UI:

cargo run --release -- model-eval \
  --model qwen3.8-flash-next \
  --reasoning low \
  --acceleration off \
  --power-percent 100

The evaluator first performs a short unmeasured warmup through the same system-prompt builder and production runtime as the UI, then resets its GPU counters and starts the measured request. warmup_result.elapsed_ms includes initial model loading; its timing object separates model_load_ms and non_loading_elapsed_ms (the latter still includes setup, inference, and checkpoint work). Do not add the reported load time to the warmup total. A model_loaded event reports the same load duration as the shared runtime metrics. Canary samples retain their start phase and record completion_phase, actual completion elapsed_ms, and host observation observed_elapsed_ms; per-phase summaries group by probe start. result.timing.decode_ms measures the shared engine decode loop, including sampling and streamed output callbacks but excluding subsequent pending-token finalization and checkpoint work. Stats shows the same measurement as "Decode loop (without checkpoint)". The older host-observed interval through request completion is retained as decode_to_request_end_ms; it is not pure decode time. decode_loop_tokens_per_second uses completion tokens divided by the loop duration; decode_tokens_per_second remains the UI's last progress rate. checkpoint_io reports request operation counts and the last read/write duration, not summed I/O time. A missing decode duration is null, not zero. The default test prompt identifies a prose-generation test and asks the model to choose its own genre, setting, and characters, then output only a complete short story without questions, choices, limitations, or coding discussion. JSON Lines report runtime phases, prefill and decode speed, Metal busy time, command-buffer duration, CPU, physical memory, disk I/O, and the complete reasoning and answer. --input-file PATH appends a UTF-8 file to the supplied prompt for realistic long-prefill tests. Repeat --prompt to run multiple turns in one ongoing chat: every later turn includes all preceding user prompts and complete assistant responses and continues from the same session checkpoint. An independent Metal canary runs inside the same process as the production runtime through loading, prefill, decode, and finishing, reporting per-phase scheduling and completion latency plus stalls that could make a UI unresponsive. Use --acceleration on|off to compare DSpark on DeepSeek or integrated MTP on GLM/Qwen. The canary itself submits GPU work, so use --canary off for clean throughput measurements and --canary on for responsiveness diagnostics. The UI canary is off by default and can be enabled in Stats. A supervisor terminates the worker only when startup or token progress stalls, or when the memory limit is exceeded; there is no whole-run timeout. See model-eval --help for all overrides. The evaluator only opens already installed model artifacts and never downloads missing models.

Evaluation defaults to --plain-chat on: the shared UI prompt builder omits agent/tool schemas, skill instructions and the built-in coding-agent system prompt; an explicitly customized system prompt is preserved. The corresponding UI configuration is plain_chat: true in its YAML configuration. In that mode the UI also skips workspace/date injections, extension hooks, A2UI handling, tool execution, automatic response retries and automatic first-turn titles. Use a new chat in both frontends: existing history is never silently removed. The normal UI default remains agent mode (plain_chat: false). --plain-chat off restores agent prompt construction for diagnostics, but does not make the harness an agent/tool executor. The effective mode and complete system prompt are reported in the start receipt; do not compare runs with different modes.

For a summary, story, and Python one-shot in a single chat, pass three prompts:

target/release/ds4-server model-eval \
  --model qwen3.8-flash-next --reasoning low \
  --power-percent 100 --acceleration off --canary on \
  --input-file README.md \
  --prompt 'Give a concise summary of the following project README. Return the summary directly without asking questions.' \
  --prompt 'This is a prose-generation test. Respond only with a complete short fictional story in ordinary prose. Choose the genre, setting, and characters yourself and begin the story immediately. Do not ask questions, offer choices, explain limitations, or discuss coding.' \
  --prompt 'This is an offline code-generation benchmark. Do not invoke tools or emit tool-call markup. Return exactly one Python code block defining has_close_elements(numbers: list[float], threshold: float) -> bool. It must return True if any two distinct numbers are closer than threshold. Include no explanation outside the code block.'

The input file is appended only to the first prompt. Each turn starts after the previous response finishes; measurement_start records the turn number and the number of history messages (1, 3, 5), and each result reports its own timings, cached tokens, reasoning, and answer. The warmup uses a separate session and is not included in this history.

Build and run

DS4Server targets Apple silicon with macOS 15 or newer and requires Rust 1.97 plus cargo-packager 0.11.8.

cargo install cargo-packager --locked --version 0.11.8
make bundle
open target/release/DS4Server.app

Tagged versions are also published as Apple Silicon DMG files on the repository releases page.

Application data

~/Library/Application Support/de.rfc1437.ds4server/data.sqlite3
~/Library/Application Support/de.rfc1437.ds4server/config.yaml
~/Library/Application Support/de.rfc1437.ds4server/AGENTS.md
~/Library/Application Support/de.rfc1437.ds4server/models/
~/Library/Application Support/de.rfc1437.ds4server/kv-cache/

Deleting a project or session removes DS4Server metadata and its associated checkpoint; it never deletes the referenced project directory. Standard agent skills live separately under ~/.agents/skills/. See the issue tracker for remaining implementation work.

The adapted Metal and browser sources retain their upstream notices in native/metal/LICENSE and native/web/LICENSE.

12 KiB Raw Blame History