Plan A2UI chat surfaces

This commit is contained in:
Georg Bauer
2026-07-24 13:22:46 +02:00
parent 2e4b811f48
commit 859eecb459

46
PLAN.md
View File

@@ -18,6 +18,7 @@ sessions, tools, and turn orchestration.
| Model runtime | Download workflow implemented | Persisted model choice, DSpark toggle, idle timeout, and Rust-native background main/DSpark downloads with progress, ETA, stop, restart-resume, size checks, and SHA-256 verification | Loading, inference, idle unloading, and Metal integration |
| Local endpoint | Not started | Nothing listening | OpenAI-compatible HTTP and streaming surfaces |
| Agent | UI shell only | Empty conversation pane and composer placeholder | Messages, generation, tools, approvals, compaction, and cancellation |
| A2UI | Future extension | bDS2 provides the reference structured render-tool contract | Native inline surfaces for local chat after core chat and tools are stable |
| Dev brain | Not started | No vault access | Obsidian vault selection, durable access, and agent knowledge/memory tools |
| Release | Not started | Development binary builds and tests | `.app` packaging, signing, entitlements, and notarization |
@@ -369,6 +370,49 @@ continue from the persisted session. With Dev Brain enabled, store a memory in
one session and retrieve it with a cited vault-relative note path from a session
in another project after restarting the app.
## Phase 5 — A2UI rich local-chat surfaces
Status: **future extension; not part of the core milestone**. Start only after
ordinary local chat, structured tool calls, transcript persistence, and
cancellation are stable. Use the A2UI implementation in `../bDS2` as the
behavioral reference, especially its render-tool schemas, typed surface
normalization, inline message placement, chart geometry, and interaction state.
- Expose typed `render_card`, `render_chart`, `render_form`, `render_list`,
`render_metric`, `render_mindmap`, `render_table`, and `render_tabs` tools to
the local agent. Keep these tools presentation-only; they return structured
data and do not perform project mutations themselves.
- Support the bDS2 chart set initially: bar, stacked bar, line, area, pie,
donut, and heatmap. Render charts and mind maps with native Iced primitives
and SVG geometry inside the assistant message rather than adding a web view.
- Treat tool arguments as untrusted data. Validate type, nesting depth, row and
series counts, numeric values, labels, action names, and payload size before
rendering. Never accept model-authored HTML, JavaScript, styles, file URLs, or
arbitrary commands.
- Attach each validated surface to the assistant message/tool call that created
it and persist the versioned structured payload with the transcript. A
restarted session must reconstruct the same surface without asking the model
to emit it again.
- Build a surface only after its tool arguments are complete and valid while
allowing ordinary assistant text to continue streaming. Malformed or unknown
render payloads fall back to a bounded JSON/text representation and must not
fail the surrounding chat turn.
- Preserve local interaction state for forms and tabs. Dispatch form submits
and card actions through a small allowlist of application actions; any action
that mutates files, settings, or external state still follows the normal
approval path.
- Provide keyboard navigation, readable labels, theme-aware colors, and a text
or tabular fallback for chart data so the information remains accessible and
copyable.
- Keep A2UI local-chat-only initially. Do not extend the OpenAI-compatible HTTP
surface or invent a general plugin/component protocol until a real client
requires it.
Exit criterion: ask the local agent to present fixture data as each supported
surface type; verify a chart and nested tab surface render correctly, an
interactive form returns validated values, malformed payloads degrade safely,
and all surfaces survive application restart with their message history.
## Verification and packaging
Status: **partially implemented**. Formatting, Clippy, build, and unit-test
@@ -398,3 +442,5 @@ tests, and release packaging remain open.
finally port the agent tools.
5. Add the opt-in Dev Brain tool after local file/search boundaries and agent
approvals are stable.
6. Add bDS2-style A2UI render tools and native inline surfaces after the core
local chat/tool loop is complete.