hugo/DS4Server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement DS4 model intake

Browse Source
This commit is contained in:
Georg Bauer
2026-07-24 16:53:03 +02:00
parent 87dfc01e74
commit 616b550849
13 changed files with 2418 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
PLAN.md
View File

@@ -15,7 +15,7 @@ sessions, tools, and turn orchestration.
| App shell | Implemented | Native multi-window Iced app, Codex-inspired layout, native Application/Edit/Window menus, Preferences panel, and Model Manager window | Functional chat content and native app packaging |
| Projects and sessions | Implemented for metadata | Native folder picker, project-name dialog, create/select/delete projects and sessions | Transcripts, KV payloads, rename/archive, and model binding |
| Persistence | Implemented for metadata and preferences | SQLite in Application Support, Diesel ORM, embedded migrations, constrained defaults and CRUD tests | Messages, downloaded-artifact metadata, and session-runtime migrations |
| Model runtime | Preferences and downloads implemented | Complete typed DS4 preference contract plus a Model Manager for background download, validation, deletion, progress, ETA, stop, and restart-resume | Loading, inference, idle unloading, and Metal integration |
| Model runtime | Model intake implemented | Typed DS4 preferences, managed downloads, mmap-backed GGUF v3 loading, complete DS4 shape/quantization validation, DeepSeek/GLM tokenization, and prompt rendering | Metal inference, sessions/KV state, lifecycle coordination, and idle unloading |
| Local endpoint | Not started | Nothing listening | OpenAI-compatible HTTP and streaming surfaces |
| Agent | UI shell only | Empty conversation pane and composer placeholder | Messages, generation, tools, approvals, compaction, and cancellation |
| A2UI | Future extension | bDS2 provides the reference structured render-tool contract | Native inline surfaces for local chat after core chat and tools are stable |
@@ -23,10 +23,12 @@ sessions, tools, and turn orchestration.
| Release | Not started | Development binary builds and tests | `.app` packaging, signing, entitlements, and notarization |
The application currently manages durable project/session metadata and model
preferences. Its separate Model Manager can download, validate, and delete
main and DSpark GGUF artifacts in managed Application Support storage without
blocking the UI. It does not yet load a model, serve HTTP, save chat messages,
or run an agent.
preferences. Its separate Model Manager can download, fully validate, and
delete main and DSpark GGUF artifacts in managed Application Support storage
without blocking the UI. The engine can mmap a configured model, validate its
metadata and complete tensor layout, load its tokenizer, and render a prompt;
the app does not yet acquire that engine for inference, serve HTTP, save chat
messages, or run an agent.
## Phase 0 — project and session shell
@@ -45,17 +47,21 @@ Exit criterion: restart the app and see the same project/session tree.
## Phase 1 — model library and on-demand loading
Status: **partially implemented**. The typed DS4 preference contract and the
complete background download workflow are implemented; loading, inference,
and idle unloading remain open.
Status: **partially implemented**. The typed DS4 preference contract, shared
effective-settings builder, background download workflow, and model-intake
boundary are implemented; Metal inference, session state, and idle lifecycle
coordination remain open.
1. **Implemented:** the typed model/runtime preference contract described below
is complete before chat, the HTTP endpoint, or the engine lifecycle.
2. Port the narrow GGUF loader, tokenizer, prompt renderer, and Metal session
API behind a small Rust `Engine` surface modeled on `ds4.h`:
`open`, `close`, `create_session`, `sync`, `sample`, and KV save/load.
3. Retain the tested model restrictions from DwarfStar; reject unsupported
shapes and quantizations before allocating model state.
2. **Partially implemented:** the mmap-backed GGUF loader, DeepSeek/GLM
tokenizer, prompt renderer, and model `open`/`close` ownership boundary are
complete. The Metal session API modeled on `ds4.h` remains:
`create_session`, `sync`, `sample`, and KV save/load.
3. **Implemented:** retain the tested model restrictions from DwarfStar. Main
and DSpark artifacts reject invalid GGUF versions, catalog mismatches, and
incompatible metadata, tensor shapes, offsets, or quantization before
promotion and again before model use.
4. Reuse the existing `.metal` kernels and preserve mmap-backed resident
loading plus the explicit SSD-streaming path. Keep Objective-C only at the
Metal interop boundary.
@@ -475,15 +481,13 @@ tests, and release packaging remain open.
## Recommended next milestones
1. Complete DS4 CLI model/runtime/generation preference parity and its shared
typed effective-settings builder.
2. Implement the single-engine lifecycle and local generation path: lazy load,
1. Implement the single-engine lifecycle and local generation path: lazy load,
shared concurrent acquisition, activity tracking, and idle unload.
3. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
2. Put the OpenAI-compatible endpoint on that same engine lifecycle so local
chats and HTTP requests cannot load duplicate engines.
4. Add transcript/KV persistence, then connect the existing conversation UI and
3. Add transcript/KV persistence, then connect the existing conversation UI and
finally port the agent tools.
5. Add the opt-in Dev Brain tool after local file/search boundaries and agent
4. Add the opt-in Dev Brain tool after local file/search boundaries and agent
approvals are stable.
6. Add bDS2-style A2UI render tools and native inline surfaces after the core
5. Add bDS2-style A2UI render tools and native inline surfaces after the core
local chat/tool loop is complete.