Implement batched server parity and observability

PLAN.md

@@ -12,12 +12,12 @@ sessions, tools, and turn orchestration.
 
 | Area | Status | Available now | Still open |
 | --- | --- | --- | --- |
-| App shell | Implemented | Native multi-window Iced app, Codex-inspired layout, native Application/Edit/Window menus, Preferences panel, Model Manager, and streaming composer | Native app release packaging |
+| App shell | Implemented | Native multi-window Iced app, Codex-inspired layout, Chat/Stats segmented workspace, native Application/Edit/Window menus, Preferences panel, Model Manager, and streaming composer | Native app release packaging |
 | Projects and sessions | Implemented with transcripts and KV resume | Native folder picker, project-name dialog, create/select/delete projects and sessions, durable structured chat history, and per-session KV checkpoints | Rename/archive and model binding |
 | Persistence | Implemented for local chat | SQLite in Application Support, Diesel ORM, embedded migrations, constrained defaults, streamed message updates, reopen tests, context/speed state, and local KV rehydration | Downloaded-artifact metadata and session-runtime migrations; HTTP chats deliberately never enter SQLite |
-| Model runtime | DeepSeek Flash vertical implemented | Rust-owned mmap/model/session graph, local Metal kernels, full configured sparse context, DS4 sampling, one process-wide UI/HTTP owner, cancellation, idle unloading, prefix continuation, and KV checkpoint save/load | DSpark/SSD/steering and GLM/Pro execution |
-| Local endpoint | Chat Completions vertical implemented | Automatic configurable localhost listener (port 4000 by default), verified-on-disk model discovery, transient requests, non-streaming and SSE Chat Completions, reasoning, usage, sampling, stops, cancellation, tools, and exact in-process tool replay | GUI status and enable/disable controls, Responses, legacy Completions, Anthropic Messages, full `ds4_server.c` fixture parity, and cross-restart exact tool replay |
-| Agent | Solid local chat implemented | Multi-turn role rendering, structured reasoning disclosure, Markdown, automatic scrolling, context/speed display, durable transcript rehydration, and KV-backed resume | Tools, approvals, compaction, and richer statistics |
+| Model runtime | DeepSeek Flash decode and batched prefill implemented | Rust-owned mmap/model/session graph, local Metal kernels, full configured sparse context, exact absolute-position chunk scheduling, cold and resumed layer-major prefill (including unaligned compressor/indexer frontiers), DS4 sampling, one process-wide UI/HTTP owner, cancellation, idle unloading, prefix continuation, KV checkpoint save/load, and lock-free runtime metrics | Remaining KV continuation/rewrite parity, DSpark/SSD/steering, GLM/Pro execution, and sampled per-kernel GPU timing if coarse phase metrics prove insufficient |
+| Local endpoint | Exact parity in progress | Configurable localhost listener, verified-on-disk model discovery, transient Chat/Completions/Messages/Responses routes, native JSON/SSE envelopes, exact tool schema/call/result replay, live tool continuation, and differential C/Rust fixtures for tool use, cache accounting, reasoning, cold 6k-token prefill, resumed unaligned prefill, and real Pi tool execution | Remaining `ds4_server.c` parser/tool recovery, KV-store policy, disconnect/cancellation, scheduling/lifecycle, and error/context-limit fixtures; enable/disable controls |
+| Agent | Solid local chat implemented | Multi-turn role rendering, structured reasoning disclosure, Markdown, automatic scrolling, context/speed display, durable transcript rehydration, KV-backed resume, and a live Stats dashboard | Tools, approvals, and compaction |
 | A2UI | Future extension | bDS2 provides the reference structured render-tool contract | Native inline surfaces for local chat after core chat and tools are stable |
 | Dev brain | Not started | No vault access | Obsidian vault selection, durable access, and agent knowledge/memory tools |
 | Release | Not started | Development binary builds and tests | `.app` packaging, signing, entitlements, and notarization |
@@ -29,8 +29,12 @@ executor, streams generated text, supports cancellation and multi-turn role
 replay, persists reasoning and answers while they stream, rehydrates selected
 sessions, and unloads model resources after the configured idle timeout. The
 current Rust graph supports the full configured context with ratio-4 sparse
-attention and durable local KV checkpoints. The shared HTTP Chat Completions
-service is available; the remaining protocol routes and agent tools remain open.
+attention, fixed-size cold/resumed batch chunks, unaligned compressor/indexer
+continuation, and durable local KV checkpoints. The shared HTTP service exposes
+every `ds4_server.c` route and passes the initial differential corpus for native
+tool use/continuation, cache accounting, reasoning, and Pi execution. Full
+reference parity remains in progress until the remaining parser, KV-store,
+cancellation, scheduling, and error fixtures are ported.
 
 ## Phase 0 — project and session shell
 
@@ -220,6 +224,27 @@ and generation value; show the same effective defaults and validation as the
 CLI; and construct one engine configuration plus one turn configuration without
 chat or HTTP code re-parsing preference fields.
 
+### Runtime observability (implemented)
+
+- Keep instrumentation in the shared model/runtime and HTTP paths so local chat
+  and endpoint work report through the same counters.
+- Publish hot-path state with relaxed atomics only. Generation and prefill never
+  wait for the UI, allocate telemetry events, write logs, or scan the KV cache.
+- Sample snapshots in the UI every 200 ms and retain only a bounded in-memory
+  history for prefill, decode, request-rate, and KV read/write-rate graphs.
+- Show model phase and lifecycle, prefill/decode throughput, context occupancy,
+  request queue and source, exact memory/disk hits, misses, invalid entries,
+  prefix reuse, checkpoint storage and I/O, endpoint traffic and latency,
+  errors, and model mapping details on the Stats tab.
+- Derive KV file sizes when checkpoints are already being saved and scan the
+  cache directory only once at startup. Add sampled Metal command-buffer or
+  per-kernel timings later only if coarse phase counters cannot diagnose a
+  measured engine bottleneck.
+
+Exit criterion: model generation and endpoint traffic visibly update the Stats
+dashboard without introducing locks, telemetry I/O, or unbounded history on the
+prefill/decode path.
+
 ### Targeted downloads and storage
 
 - Preferences only selects the model/runtime configuration. A separate Model
@@ -266,37 +291,63 @@ after the configured idle timeout, and pass the matching `../ds4` token-output
 fixture. Repeat through the local endpoint and verify both paths share the same
 engine lifecycle.
 
-## Phase 2 — OpenAI-compatible local endpoint
+## Phase 2 — exact `ds4_server.c` HTTP parity
 
-Status: **partially implemented**. The shared single-model runtime,
-verified-model discovery, configurable localhost port, and the first
-`ds4_server.c`-compatible Chat Completions vertical are implemented. The
-remaining protocol routes plus GUI status and enable/disable controls remain
-open.
+Status: **exact parity in progress**. The shared single-model runtime,
+verified-model discovery, configurable localhost port, all four generation
+routes, native JSON/SSE response envelopes, exact batched prefill, tool replay,
+and live continuation exist. Differential C/Rust fixtures cover tools,
+reasoning, usage/cache accounting, and real Pi execution. Full parser/tool
+recovery, KV-store, cancellation, scheduling, lifecycle, and error fixture
+parity remain open, so no route is considered complete yet.
+
+`../ds4/ds4_server.c` is normative for every externally observable HTTP
+behavior. “Compatible” approximations are not acceptable: routes, accepted
+fields, defaults, aliases, prompt rendering, status codes, headers, JSON/SSE
+shapes and ordering, five-second prefill keepalives, reasoning/tool translation,
+usage, finish reasons, errors, cancellation, and cache continuation must match
+the C server exactly. Port its fixtures alongside each Rust slice and keep every
+previous parity fixture passing.
+
+This requirement covers the complete communication behavior, not only the HTTP
+wire format. The Rust request path must follow the same request parsing and
+validation, model aliases, prompt/token construction, reasoning mode, sampling,
+stop matching, tool schema/call/result translation, live tool continuation,
+canonical replay, KV lookup/rewrite/checkpoint policy, scheduling, disconnect
+cancellation, usage accounting, finish decisions, and model lifecycle invoked
+by `ds4_server.c`. `ds4.c` is normative for engine operations reached from the
+server. A route is complete only when its differential request corpus passes
+against the C server; partial route support stays labeled incomplete.
 
 1. **Partially implemented:** add a localhost-only HTTP service whose lifecycle is independent of the
    engine. It can listen while the model is unloaded and acquires the configured
    model only for inference. It listens on configurable port 4000 by default;
    start/stop controls remain open. Never expose a LAN listener without an
    explicit setting.
-2. Implement the DwarfStar compatibility surface from `ds4_server.c` in this
+2. **Partially implemented:** expose the complete DwarfStar HTTP surface from
+   `ds4_server.c` in this
    order, preserving its request parsing, prompt formatting, reasoning fields,
    streaming events, errors, usage accounting, stop behavior, tool replay, and
    cancellation semantics rather than inventing an app-specific protocol:
    - `GET /v1/models`
+   - `GET /v1/models/{known-alias}`
+   - `POST /v1/messages`
    - `POST /v1/chat/completions`
    - `POST /v1/responses`
    - `POST /v1/completions`
+   - `OPTIONS` with the same CORS behavior when enabled
+
    `GET /v1/models` scans managed artifacts without forcing a model to load and
    reports only main model files that are fully downloaded and have a matching
    on-disk verification marker. Missing, partial, unverified, or checksum-invalid
    artifacts are never advertised.
-3. Support streaming SSE, reasoning output, usage accounting, cancellation,
-   sampling parameters, tool schemas, and tool choice.
-4. Port exact sampled tool-call replay and deterministic canonicalization from
+3. **Partially implemented:** streaming SSE, reasoning output, usage accounting,
+   sampling parameters, tool schemas, and automatic/disabled tool choice work;
+   exact disconnect cancellation and remaining forced-choice errors are open.
+4. **Partially implemented:** port exact sampled tool-call replay and deterministic canonicalization from
    `ds4_server.c` so a client's normalized JSON does not destroy KV-prefix
    reuse.
-5. Route API work through the single process-wide model owner and engine-owned
+5. **Partially implemented:** route API work through the single process-wide model owner and engine-owned
    KV store defined in Phase 1. External requests are always stateless at the
    application layer: never write their messages to SQLite and never bind a
    Responses `conversation`, request identifier, or client-supplied session key
@@ -304,15 +355,19 @@ open.
    DS4-compatible prefix lookup may accelerate that replay using opaque KV cache
    files. Add resident batching only if one-model serialized execution is later
    proven insufficient.
-6. Display endpoint address, model, active requests, token rates, and errors in
-   the GUI.
+6. **Implemented:** display endpoint address, model, active requests, token
+   rates, cache behavior, and errors in the GUI Stats dashboard.
 
-Exit criterion: the upstream DwarfStar server tests pass against the app for
-non-streaming and streaming Chat Completions and Responses calls, including a
-multi-turn tool call. Verified installed models are listed without loading one;
-missing/unverified models are absent; local and HTTP generations cannot load two
-models concurrently; external calls leave no project, session, message, or
-transcript rows behind; and compatible KV cache files remain reusable.
+Exit criterion: the complete `ds4_server.c` fixture suite and a differential
+black-box corpus pass against the Rust app for Models, Anthropic Messages, Chat
+Completions, Responses, Completions, and OPTIONS, in streaming and non-streaming
+forms where supported. The corpus covers valid requests, aliases/defaults,
+context limits, tools and multi-turn continuation, reasoning, cache reuse,
+cancellation/disconnects, queueing, errors, usage, and finish reasons. Verified
+installed models are listed without loading one; missing/unverified models are
+absent; local and HTTP generations cannot load two models concurrently;
+external calls leave no project, session, message, or transcript rows behind;
+and compatible KV cache files remain reusable.
 
 ## Phase 3 — durable agent sessions
 
@@ -524,10 +579,11 @@ tests, and release packaging remain open.
 1. Finish the Flash session core as one larger slice: port the ratio-4 indexer
    and indexed attention for full configured context, then add prefix sync and
    compatible KV checkpoint save/load. **Implemented.**
-2. Put the OpenAI-compatible endpoint on the same engine lifecycle so local
-   chats and HTTP requests cannot load duplicate models. Keep HTTP conversation
-   state transient except for opaque KV cache files. **Chat Completions vertical
-   implemented; remaining reference routes are open.**
+2. Complete exact `ds4_server.c` communication parity as one tracked program,
+   beginning with exact layer-major/chunked/resumed prefill and KV continuation,
+   then porting every parser/route/state machine and its fixtures. Keep HTTP
+   conversation state transient except for opaque KV cache files. **In
+   progress; no route is considered complete until differential tests pass.**
 3. Local transcript/KV persistence and the conversation UI are implemented;
    port the agent tools after the shared endpoint lifecycle is stable.
 4. Add the opt-in Dev Brain tool after local file/search boundaries and agent