From 859eecb459b8b6d8263f7144e82cb294822e931a Mon Sep 17 00:00:00 2001 From: Georg Bauer Date: Fri, 24 Jul 2026 13:22:46 +0200 Subject: [PATCH] Plan A2UI chat surfaces --- PLAN.md | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/PLAN.md b/PLAN.md index c2c1ee9..4b415a5 100644 --- a/PLAN.md +++ b/PLAN.md @@ -18,6 +18,7 @@ sessions, tools, and turn orchestration. | Model runtime | Download workflow implemented | Persisted model choice, DSpark toggle, idle timeout, and Rust-native background main/DSpark downloads with progress, ETA, stop, restart-resume, size checks, and SHA-256 verification | Loading, inference, idle unloading, and Metal integration | | Local endpoint | Not started | Nothing listening | OpenAI-compatible HTTP and streaming surfaces | | Agent | UI shell only | Empty conversation pane and composer placeholder | Messages, generation, tools, approvals, compaction, and cancellation | +| A2UI | Future extension | bDS2 provides the reference structured render-tool contract | Native inline surfaces for local chat after core chat and tools are stable | | Dev brain | Not started | No vault access | Obsidian vault selection, durable access, and agent knowledge/memory tools | | Release | Not started | Development binary builds and tests | `.app` packaging, signing, entitlements, and notarization | @@ -369,6 +370,49 @@ continue from the persisted session. With Dev Brain enabled, store a memory in one session and retrieve it with a cited vault-relative note path from a session in another project after restarting the app. +## Phase 5 — A2UI rich local-chat surfaces + +Status: **future extension; not part of the core milestone**. Start only after +ordinary local chat, structured tool calls, transcript persistence, and +cancellation are stable. Use the A2UI implementation in `../bDS2` as the +behavioral reference, especially its render-tool schemas, typed surface +normalization, inline message placement, chart geometry, and interaction state. + +- Expose typed `render_card`, `render_chart`, `render_form`, `render_list`, + `render_metric`, `render_mindmap`, `render_table`, and `render_tabs` tools to + the local agent. Keep these tools presentation-only; they return structured + data and do not perform project mutations themselves. +- Support the bDS2 chart set initially: bar, stacked bar, line, area, pie, + donut, and heatmap. Render charts and mind maps with native Iced primitives + and SVG geometry inside the assistant message rather than adding a web view. +- Treat tool arguments as untrusted data. Validate type, nesting depth, row and + series counts, numeric values, labels, action names, and payload size before + rendering. Never accept model-authored HTML, JavaScript, styles, file URLs, or + arbitrary commands. +- Attach each validated surface to the assistant message/tool call that created + it and persist the versioned structured payload with the transcript. A + restarted session must reconstruct the same surface without asking the model + to emit it again. +- Build a surface only after its tool arguments are complete and valid while + allowing ordinary assistant text to continue streaming. Malformed or unknown + render payloads fall back to a bounded JSON/text representation and must not + fail the surrounding chat turn. +- Preserve local interaction state for forms and tabs. Dispatch form submits + and card actions through a small allowlist of application actions; any action + that mutates files, settings, or external state still follows the normal + approval path. +- Provide keyboard navigation, readable labels, theme-aware colors, and a text + or tabular fallback for chart data so the information remains accessible and + copyable. +- Keep A2UI local-chat-only initially. Do not extend the OpenAI-compatible HTTP + surface or invent a general plugin/component protocol until a real client + requires it. + +Exit criterion: ask the local agent to present fixture data as each supported +surface type; verify a chart and nested tab surface render correctly, an +interactive form returns validated values, malformed payloads degrade safely, +and all surfaces survive application restart with their message history. + ## Verification and packaging Status: **partially implemented**. Formatting, Clippy, build, and unit-test @@ -398,3 +442,5 @@ tests, and release packaging remain open. finally port the agent tools. 5. Add the opt-in Dev Brain tool after local file/search boundaries and agent approvals are stable. +6. Add bDS2-style A2UI render tools and native inline surfaces after the core + local chat/tool loop is complete.