diff --git a/AGUI.md b/AGUI.md index dccfe41..5b7573f 100644 --- a/AGUI.md +++ b/AGUI.md @@ -18,6 +18,38 @@ Build a **protocol-first chat assistant** that: 4. Works identically in editor chat and assistant sidebar. 5. Is observable, testable, and versioned for internal iteration. +## Implementation Status (This Branch) + +- ✅ Phase 0 — Foundation Contracts (**completed 2026-02-25**) +- ✅ Phase 1 — Server-Side Enforcement + Repair Loop (**completed 2026-02-25**) +- ✅ Phase 2 — Capability Registry + Negotiation (**completed 2026-02-25**) +- ✅ Phase 3 — Workflow Engine and Clarification UX (**completed 2026-02-25**) +- ✅ Phase 4 — Action Policy and Safety (**completed 2026-02-25**) +- ✅ Phase 5 — Observability and QA (**completed 2026-02-25**) + +### Implemented Artifacts + +- `src/main/agentic/protocol/types.ts` +- `src/main/agentic/protocol/errors.ts` +- `src/main/agentic/protocol/validator.ts` +- `src/main/agentic/protocol/uiSchema.ts` +- `src/main/agentic/protocol/uiSpecParser.ts` +- `src/main/agentic/protocol/responseBuilder.ts` +- `src/main/agentic/capabilities/registry.ts` +- `src/main/agentic/workflow/turnStateMachine.ts` +- `src/main/agentic/workflow/checkpointStore.ts` +- `src/main/agentic/policy/actionPolicy.ts` +- `src/main/agentic/observability/protocolTelemetry.ts` + +### Branch Completion Notes + +- Main-process protocol validation/envelope building now runs server-side before renderer consumption. +- Renderer consumes protocol envelope directly; legacy text parsing remains as compatibility fallback only. +- Capability snapshots are injected into each model request and unsupported widgets/actions are filtered with diagnostics. +- Workflow checkpoints are persisted per conversation for resumable needs-input turns. +- Action policy levels (`silent`, `confirm`, `danger`) are enforced at render-time before dispatch. +- Protocol telemetry is emitted and available via `chat:getProtocolHealth` IPC. + --- ## Current State (Baseline) @@ -150,6 +182,56 @@ Rules: - `actions` are declarative, validated against capability registry. - Unknown properties are rejected in strict mode. +### Canonical Request Envelope Example + +```json +{ + "protocolVersion": "2.0", + "surface": "tab", + "messages": [ + { "role": "user", "content": "Show posting trend by month" } + ], + "context": { + "projectId": "project-1" + }, + "capabilities": { + "widgets": ["chart", "form", "tabs"], + "actions": ["openPost", "openSettings"], + "tools": ["search_posts", "list_posts"], + "disabled": [] + } +} +``` + +### Invalid Envelope Example (strict mode) + +```json +{ + "protocolVersion": "2.0", + "assistantText": "Please provide missing fields", + "intent": "ask_input", + "needsInput": { + "required": true, + "fields": [] + }, + "actions": [], + "confidence": 0.9, + "traceId": "trace-123", + "extra": "not-allowed" +} +``` + +Reason invalid: + +- `needsInput.required=true` but `needsInput.fields` is empty. +- `extra` is an unknown property in strict mode. + +### Protocol Error Codes + +- `AGUI_PROTOCOL_VALIDATION_ERROR` + - Emitted for request/response envelope validation failures. + - Includes human-readable `message` and per-field `details`. + --- ## Implementation Plan (Phased) diff --git a/API.md b/API.md index 53ec1ea..b5ab8ac 100644 --- a/API.md +++ b/API.md @@ -1,6 +1,6 @@ # API Documentation -Contract version: 1.3.0 +Contract version: 1.4.0 This reference documents all Python runtime API calls available through `bds_api` in embedded Pyodide. @@ -3073,6 +3073,7 @@ result = await bds.tags.sync_from_posts() - [chat.validateApiKey](#chatvalidateapikey) - [chat.setApiKey](#chatsetapikey) - [chat.getApiKey](#chatgetapikey) +- [chat.getProtocolHealth](#chatgetprotocolhealth) - [chat.getAvailableModels](#chatgetavailablemodels) - [chat.setDefaultModel](#chatsetdefaultmodel) - [chat.getSystemPrompt](#chatgetsystemprompt) @@ -3206,6 +3207,41 @@ result = await bds.chat.get_api_key() } ``` +### chat.getProtocolHealth + +Get AGUI protocol telemetry health snapshot. + +**Parameters** + +- None + +**Response specification** + +- Return type: `ProtocolTelemetrySnapshot` +- Data structures: `ProtocolTelemetrySnapshot` + +**Example call** + +```python +from bds_api import bds +result = await bds.chat.get_protocol_health() +``` + +**Example response** + +```python +{ + 'totalTurns': 0, + 'validEnvelopeTurns': 0, + 'repairAttempts': 0, + 'fallbackTurns': 0, + 'blockedActionCount': 0, + 'parseValidityRate': 0, + 'repairRate': 0, + 'fallbackRate': 0 +} +``` + ### chat.getAvailableModels Get available chat models and selected default. @@ -4049,6 +4085,23 @@ Stored API key state for chat provider. [↑ Back to Table of contents](#table-of-contents) +### ProtocolTelemetrySnapshot + +Aggregated protocol telemetry metrics for AGUI response health. + +**Fields** + +- totalTurns (`number`, required): Total number of recorded assistant turns. +- validEnvelopeTurns (`number`, required): Turns with schema-valid protocol envelopes. +- repairAttempts (`number`, required): Number of response repair attempts. +- fallbackTurns (`number`, required): Turns that used protocol fallback response. +- blockedActionCount (`number`, required): Count of actions blocked by policy. +- parseValidityRate (`number`, required): Ratio of valid envelopes to total turns. +- repairRate (`number`, required): Ratio of repair attempts to total turns. +- fallbackRate (`number`, required): Ratio of fallback turns to total turns. + +[↑ Back to Table of contents](#table-of-contents) + --- -Generated from contract at 2026-02-24T00:00:00.000Z. +Generated from contract at 2026-02-25T00:00:00.000Z. diff --git a/src/main/agentic/capabilities/registry.ts b/src/main/agentic/capabilities/registry.ts new file mode 100644 index 0000000..27a46db --- /dev/null +++ b/src/main/agentic/capabilities/registry.ts @@ -0,0 +1,94 @@ +import type { AgentSurface, ProtocolCapabilitySnapshot } from '../protocol/types'; + +interface CapabilityRegistryOptions { + disabledActions?: string[]; + disabledWidgets?: string[]; + disabledTools?: string[]; +} + +interface CapabilitySnapshotInput { + surface: AgentSurface; +} + +const COMMON_WIDGETS = [ + 'text', + 'metric', + 'list', + 'table', + 'action', + 'chart', + 'form', + 'input', + 'datePicker', + 'card', + 'image', + 'tabs', +] as const; + +const COMMON_ACTIONS = [ + 'openSettings', + 'openPost', + 'openMedia', + 'openPanel', + 'setActiveView', + 'toggleSidebar', + 'togglePanel', + 'toggleAssistantSidebar', +] as const; + +const COMMON_TOOLS = [ + 'search_posts', + 'read_post', + 'list_posts', + 'get_media', + 'list_media', + 'update_post_metadata', + 'update_media_metadata', + 'list_tags', + 'list_categories', + 'view_image', + 'get_post_backlinks', + 'get_post_outlinks', + 'get_post_media', + 'get_media_posts', +] as const; + +function unique(values: string[]): string[] { + return Array.from(new Set(values)); +} + +export class CapabilityRegistryService { + private readonly disabledActions: Set; + private readonly disabledWidgets: Set; + private readonly disabledTools: Set; + + constructor(options: CapabilityRegistryOptions = {}) { + this.disabledActions = new Set(options.disabledActions ?? []); + this.disabledWidgets = new Set(options.disabledWidgets ?? []); + this.disabledTools = new Set(options.disabledTools ?? []); + } + + getSnapshot(input: CapabilitySnapshotInput): ProtocolCapabilitySnapshot { + const widgets = COMMON_WIDGETS.filter((widget) => !this.disabledWidgets.has(widget)); + + const surfaceActions = input.surface === 'tab' + ? COMMON_ACTIONS.filter((action) => action !== 'toggleAssistantSidebar') + : COMMON_ACTIONS.filter((action) => action !== 'toggleSidebar'); + + const actions = surfaceActions.filter((action) => !this.disabledActions.has(action)); + const tools = COMMON_TOOLS.filter((tool) => !this.disabledTools.has(tool)); + + const disabled = unique([ + ...Array.from(this.disabledActions).map((action) => `action:${action}`), + ...Array.from(this.disabledWidgets).map((widget) => `widget:${widget}`), + ...Array.from(this.disabledTools).map((tool) => `tool:${tool}`), + ]); + + return { + widgets: [...widgets], + actions: [...actions], + tools: [...tools], + disabled, + }; + } +} diff --git a/src/main/agentic/observability/protocolTelemetry.ts b/src/main/agentic/observability/protocolTelemetry.ts new file mode 100644 index 0000000..6b62ffb --- /dev/null +++ b/src/main/agentic/observability/protocolTelemetry.ts @@ -0,0 +1,63 @@ +export interface ProtocolTurnTelemetryInput { + validEnvelope: boolean; + repairAttempted: boolean; + fallbackUsed: boolean; + blockedActions: number; +} + +export interface ProtocolTelemetrySnapshot { + totalTurns: number; + validEnvelopeTurns: number; + repairAttempts: number; + fallbackTurns: number; + blockedActionCount: number; + parseValidityRate: number; + repairRate: number; + fallbackRate: number; +} + +export class ProtocolTelemetryService { + private totalTurns = 0; + private validEnvelopeTurns = 0; + private repairAttempts = 0; + private fallbackTurns = 0; + private blockedActionCount = 0; + + recordTurn(input: ProtocolTurnTelemetryInput): void { + this.totalTurns += 1; + if (input.validEnvelope) { + this.validEnvelopeTurns += 1; + } + if (input.repairAttempted) { + this.repairAttempts += 1; + } + if (input.fallbackUsed) { + this.fallbackTurns += 1; + } + this.blockedActionCount += input.blockedActions; + } + + getSnapshot(): ProtocolTelemetrySnapshot { + const denominator = this.totalTurns || 1; + return { + totalTurns: this.totalTurns, + validEnvelopeTurns: this.validEnvelopeTurns, + repairAttempts: this.repairAttempts, + fallbackTurns: this.fallbackTurns, + blockedActionCount: this.blockedActionCount, + parseValidityRate: this.validEnvelopeTurns / denominator, + repairRate: this.repairAttempts / denominator, + fallbackRate: this.fallbackTurns / denominator, + }; + } +} + +let protocolTelemetryService: ProtocolTelemetryService | null = null; + +export function getProtocolTelemetryService(): ProtocolTelemetryService { + if (!protocolTelemetryService) { + protocolTelemetryService = new ProtocolTelemetryService(); + } + + return protocolTelemetryService; +} diff --git a/src/main/agentic/policy/actionPolicy.ts b/src/main/agentic/policy/actionPolicy.ts new file mode 100644 index 0000000..60e828a --- /dev/null +++ b/src/main/agentic/policy/actionPolicy.ts @@ -0,0 +1,30 @@ +export type ActionPolicyLevel = 'silent' | 'confirm' | 'danger'; + +export interface ActionPolicyResolution { + level: ActionPolicyLevel; + requiresConfirmation: boolean; +} + +const ACTION_POLICY_MAP: Record = { + openPost: 'silent', + openMedia: 'silent', + openPanel: 'silent', + setActiveView: 'silent', + toggleSidebar: 'silent', + togglePanel: 'silent', + toggleAssistantSidebar: 'silent', + openSettings: 'confirm', + updatePostMetadata: 'confirm', + updateMediaMetadata: 'confirm', + submitNeedsInput: 'confirm', + deletePost: 'danger', + deleteMedia: 'danger', +}; + +export function resolveActionPolicy(action: string): ActionPolicyResolution { + const level = ACTION_POLICY_MAP[action] ?? 'danger'; + return { + level, + requiresConfirmation: level !== 'silent', + }; +} diff --git a/src/main/agentic/protocol/errors.ts b/src/main/agentic/protocol/errors.ts new file mode 100644 index 0000000..aed9910 --- /dev/null +++ b/src/main/agentic/protocol/errors.ts @@ -0,0 +1,9 @@ +import type { ProtocolValidationError } from './types'; + +export function createProtocolValidationError(message: string, details?: string[]): ProtocolValidationError { + return { + code: 'AGUI_PROTOCOL_VALIDATION_ERROR', + message, + details, + }; +} diff --git a/src/main/agentic/protocol/responseBuilder.ts b/src/main/agentic/protocol/responseBuilder.ts new file mode 100644 index 0000000..4c12a6c --- /dev/null +++ b/src/main/agentic/protocol/responseBuilder.ts @@ -0,0 +1,312 @@ +import { randomUUID } from 'crypto'; +import type { + AgentSurface, + ProtocolCapabilitySnapshot, + ProtocolIntent, + ProtocolResponseEnvelope, + ProtocolValidationError, +} from './types'; +import { validateProtocolResponseEnvelope } from './validator'; +import { extractAssistantUiSpec } from './uiSpecParser'; +import { resolveActionPolicy } from '../policy/actionPolicy'; + +export interface ProtocolResponseBuildInput { + rawAssistantOutput: string; + surface: AgentSurface; + capabilities: ProtocolCapabilitySnapshot; +} + +export interface ProtocolResponseBuildResult { + envelope: ProtocolResponseEnvelope; + traceId: string; + repairAttempted: boolean; + warnings: string[]; + validationError?: ProtocolValidationError; +} + +export class ProtocolResponseBuilder { + build(input: ProtocolResponseBuildInput): ProtocolResponseBuildResult { + const warnings: string[] = []; + + const directEnvelope = this.parseCanonicalEnvelope(input.rawAssistantOutput); + if (directEnvelope) { + const normalizedDirectEnvelope = this.applyActionPolicies(directEnvelope); + const { filteredEnvelope, warnings: capabilityWarnings } = this.applyCapabilityGuards(normalizedDirectEnvelope, input.capabilities); + warnings.push(...capabilityWarnings); + const validated = validateProtocolResponseEnvelope(filteredEnvelope); + if (validated.ok && validated.value) { + return { + envelope: validated.value, + traceId: validated.value.traceId, + repairAttempted: false, + warnings, + }; + } + + const fallback = this.fallbackEnvelope(input.rawAssistantOutput); + return { + envelope: fallback, + traceId: fallback.traceId, + repairAttempted: true, + warnings, + validationError: validated.error, + }; + } + + const repaired = this.repairRawEnvelope(input.rawAssistantOutput); + if (repaired) { + const normalizedRepairedEnvelope = this.applyActionPolicies(repaired); + const { filteredEnvelope, warnings: capabilityWarnings } = this.applyCapabilityGuards(normalizedRepairedEnvelope, input.capabilities); + warnings.push(...capabilityWarnings); + const validated = validateProtocolResponseEnvelope(filteredEnvelope); + if (validated.ok && validated.value) { + return { + envelope: validated.value, + traceId: validated.value.traceId, + repairAttempted: true, + warnings, + }; + } + } + + const parsedUi = extractAssistantUiSpec(input.rawAssistantOutput); + const jsonLikeOutput = input.rawAssistantOutput.trim().startsWith('{') + || input.rawAssistantOutput.trim().startsWith('['); + const baseEnvelope: ProtocolResponseEnvelope = { + protocolVersion: '2.0', + assistantText: parsedUi.assistantText, + ui: parsedUi.ui || undefined, + intent: jsonLikeOutput + ? 'summarize' + : this.deriveIntent(parsedUi.assistantText, Boolean(parsedUi.ui), false), + needsInput: { + required: false, + fields: [], + }, + actions: [], + confidence: 0.7, + traceId: randomUUID(), + }; + + const normalizedBaseEnvelope = this.applyActionPolicies(baseEnvelope); + const { filteredEnvelope, warnings: capabilityWarnings } = this.applyCapabilityGuards(normalizedBaseEnvelope, input.capabilities); + warnings.push(...capabilityWarnings); + + const validated = validateProtocolResponseEnvelope(filteredEnvelope); + if (validated.ok && validated.value) { + return { + envelope: validated.value, + traceId: validated.value.traceId, + repairAttempted: false, + warnings, + }; + } + + const fallback = this.fallbackEnvelope(input.rawAssistantOutput); + return { + envelope: fallback, + traceId: fallback.traceId, + repairAttempted: true, + warnings, + validationError: validated.error, + }; + } + + private parseCanonicalEnvelope(raw: string): ProtocolResponseEnvelope | null { + try { + const parsed = JSON.parse(raw); + const validated = validateProtocolResponseEnvelope(parsed); + return validated.ok && validated.value ? validated.value : null; + } catch { + return null; + } + } + + private repairRawEnvelope(raw: string): ProtocolResponseEnvelope | null { + try { + const parsed = JSON.parse(raw) as Record; + const looksLikeEnvelope = Boolean( + parsed.assistantText + || parsed.assistant_text + || parsed.intent + || parsed.needsInput + || parsed.needs_input + || parsed.actions, + ); + + if (!looksLikeEnvelope) { + return null; + } + + const repaired: Record = { + protocolVersion: parsed.protocolVersion ?? parsed.protocol_version ?? '2.0', + assistantText: parsed.assistantText ?? parsed.assistant_text ?? '', + ui: parsed.ui, + intent: parsed.intent ?? 'summarize', + needsInput: parsed.needsInput ?? parsed.needs_input ?? { required: false, fields: [] }, + actions: parsed.actions ?? [], + confidence: typeof parsed.confidence === 'number' ? parsed.confidence : 0.6, + traceId: parsed.traceId ?? parsed.trace_id ?? randomUUID(), + }; + + const validated = validateProtocolResponseEnvelope(repaired); + return validated.ok && validated.value ? validated.value : null; + } catch { + return null; + } + } + + private deriveIntent(text: string, hasUi: boolean, needsInput: boolean): ProtocolIntent { + if (needsInput) { + return 'ask_input'; + } + + if (hasUi) { + return 'propose_action'; + } + + if (text.trim().length === 0) { + return 'summarize'; + } + + return 'analyze'; + } + + private applyCapabilityGuards( + envelope: ProtocolResponseEnvelope, + capabilities: ProtocolCapabilitySnapshot, + ): { filteredEnvelope: ProtocolResponseEnvelope; warnings: string[] } { + const warnings: string[] = []; + + const filteredActions = envelope.actions.filter((action) => { + const supported = capabilities.actions.includes(action.action); + if (!supported) { + warnings.push(`Blocked unsupported action: ${action.action}`); + } + return supported; + }); + + const filteredUiElements = envelope.ui?.elements.filter((element) => { + const typedElement = element as { type?: string }; + const elementType = typedElement?.type; + if (!elementType) { + return true; + } + + const supported = capabilities.widgets.includes(elementType); + if (!supported) { + warnings.push(`Blocked unsupported widget: ${elementType}`); + } + return supported; + }); + + return { + filteredEnvelope: { + ...envelope, + ui: envelope.ui && filteredUiElements + ? { + specVersion: '1', + elements: filteredUiElements, + } + : envelope.ui, + actions: filteredActions, + }, + warnings, + }; + } + + private applyActionPolicies(envelope: ProtocolResponseEnvelope): ProtocolResponseEnvelope { + const actionList = envelope.actions.length > 0 + ? envelope.actions + : this.extractActionsFromUi(envelope.ui?.elements ?? []); + + const normalizedActions = actionList.map((action, index) => { + const policy = resolveActionPolicy(action.action); + return { + id: action.id || `agui-action-${index + 1}`, + action: action.action, + label: action.label, + payload: action.payload, + policy: policy.level, + requiresConfirmation: policy.requiresConfirmation, + }; + }); + + return { + ...envelope, + actions: normalizedActions, + }; + } + + private extractActionsFromUi(elements: unknown[]): Array<{ + id: string; + action: string; + label?: string; + payload?: Record; + }> { + const extracted: Array<{ + id: string; + action: string; + label?: string; + payload?: Record; + }> = []; + + const walk = (nodes: unknown[], parentId: string) => { + nodes.forEach((node, index) => { + const typedNode = node as Record; + const type = typeof typedNode?.type === 'string' ? typedNode.type : ''; + const nodeId = `${parentId}-${index + 1}`; + + if ((type === 'action' || type === 'input' || type === 'datePicker' || type === 'form' || type === 'image') && typeof typedNode.action === 'string') { + extracted.push({ + id: `ui-${nodeId}`, + action: typedNode.action, + label: typeof typedNode.label === 'string' ? typedNode.label : undefined, + payload: typedNode.payload as Record | undefined, + }); + } + + if (type === 'card' && Array.isArray(typedNode.actions)) { + typedNode.actions.forEach((cardAction, cardActionIndex) => { + const typedCardAction = cardAction as Record; + if (typeof typedCardAction.action === 'string') { + extracted.push({ + id: `ui-${nodeId}-card-${cardActionIndex + 1}`, + action: typedCardAction.action, + label: typeof typedCardAction.label === 'string' ? typedCardAction.label : undefined, + payload: typedCardAction.payload as Record | undefined, + }); + } + }); + } + + if (type === 'tabs' && Array.isArray(typedNode.tabs)) { + typedNode.tabs.forEach((tabNode, tabIndex) => { + const typedTab = tabNode as Record; + if (Array.isArray(typedTab.elements)) { + walk(typedTab.elements, `${nodeId}-tab-${tabIndex + 1}`); + } + }); + } + }); + }; + + walk(elements, 'root'); + return extracted; + } + + private fallbackEnvelope(rawAssistantOutput: string): ProtocolResponseEnvelope { + return { + protocolVersion: '2.0', + assistantText: rawAssistantOutput, + intent: 'summarize', + needsInput: { + required: false, + fields: [], + }, + actions: [], + confidence: 0.3, + traceId: randomUUID(), + }; + } +} diff --git a/src/main/agentic/protocol/types.ts b/src/main/agentic/protocol/types.ts new file mode 100644 index 0000000..c915427 --- /dev/null +++ b/src/main/agentic/protocol/types.ts @@ -0,0 +1,82 @@ +export type AgentSurface = 'tab' | 'sidebar'; + +export type ProtocolIntent = + | 'analyze' + | 'ask_input' + | 'propose_action' + | 'execute_action' + | 'summarize'; + +export type ActionPolicyLevel = 'silent' | 'confirm' | 'danger'; + +export interface ProtocolNeedsInputField { + key: string; + label: string; + inputType: 'text' | 'textarea' | 'select' | 'checkbox' | 'date' | 'number'; + required?: boolean; + options?: Array<{ label: string; value: string }>; + placeholder?: string; + defaultValue?: string | number | boolean; +} + +export interface ProtocolNeedsInput { + required: boolean; + fields: ProtocolNeedsInputField[]; +} + +export interface ProtocolAction { + id: string; + action: string; + label?: string; + payload?: Record; + policy: ActionPolicyLevel; + requiresConfirmation: boolean; +} + +export interface ProtocolUiSpec { + specVersion: '1'; + elements: unknown[]; +} + +export interface ProtocolResponseEnvelope { + protocolVersion: '2.0'; + assistantText: string; + ui?: ProtocolUiSpec; + intent: ProtocolIntent; + needsInput: ProtocolNeedsInput; + actions: ProtocolAction[]; + confidence: number; + traceId: string; +} + +export interface ProtocolRequestMessage { + role: 'user' | 'assistant' | 'system' | 'tool'; + content: string; +} + +export interface ProtocolCapabilitySnapshot { + widgets: string[]; + actions: string[]; + tools: string[]; + disabled?: string[]; +} + +export interface ProtocolRequestEnvelope { + protocolVersion: '2.0'; + surface: AgentSurface; + messages: ProtocolRequestMessage[]; + context: Record; + capabilities: ProtocolCapabilitySnapshot; +} + +export interface ProtocolValidationError { + code: 'AGUI_PROTOCOL_VALIDATION_ERROR'; + message: string; + details?: string[]; +} + +export interface ProtocolValidationResult { + ok: boolean; + value?: T; + error?: ProtocolValidationError; +} diff --git a/src/main/agentic/protocol/uiSchema.ts b/src/main/agentic/protocol/uiSchema.ts new file mode 100644 index 0000000..a5c7dc3 --- /dev/null +++ b/src/main/agentic/protocol/uiSchema.ts @@ -0,0 +1,154 @@ +import { z } from 'zod'; + +const inputTypeSchema = z.enum(['text', 'textarea', 'select', 'checkbox', 'date', 'number']); + +const inputOptionSchema = z.object({ + label: z.string().min(1), + value: z.string(), +}).strict(); + +const textElementSchema = z.object({ + type: z.literal('text'), + text: z.string().min(1), +}).strict(); + +const metricElementSchema = z.object({ + type: z.literal('metric'), + label: z.string().min(1), + value: z.string().min(1), +}).strict(); + +const listElementSchema = z.object({ + type: z.literal('list'), + title: z.string().optional(), + items: z.array(z.string().min(1)).min(1), +}).strict(); + +const tableElementSchema = z.object({ + type: z.literal('table'), + columns: z.array(z.string().min(1)).min(1), + rows: z.array(z.array(z.string())).min(1), +}).strict(); + +const actionElementSchema = z.object({ + type: z.literal('action'), + label: z.string().min(1), + action: z.string().min(1), + payload: z.record(z.string(), z.unknown()).optional(), +}).strict(); + +const chartElementSchema = z.object({ + type: z.literal('chart'), + chartType: z.enum(['bar', 'line', 'pie']), + title: z.string().min(1).optional(), + series: z.array(z.object({ + label: z.string().min(1), + value: z.number(), + }).strict()).min(1), +}).strict(); + +const inputElementSchema = z.object({ + type: z.literal('input'), + key: z.string().min(1), + label: z.string().min(1), + inputType: inputTypeSchema, + placeholder: z.string().optional(), + defaultValue: z.union([z.string(), z.number(), z.boolean()]).optional(), + options: z.array(inputOptionSchema).optional(), + action: z.string().min(1).optional(), + submitLabel: z.string().min(1).optional(), + payload: z.record(z.string(), z.unknown()).optional(), +}).strict(); + +const datePickerElementSchema = z.object({ + type: z.literal('datePicker'), + key: z.string().min(1), + label: z.string().min(1), + defaultValue: z.string().optional(), + min: z.string().optional(), + max: z.string().optional(), + action: z.string().min(1).optional(), + submitLabel: z.string().min(1).optional(), + payload: z.record(z.string(), z.unknown()).optional(), +}).strict(); + +const formFieldSchema = z.object({ + key: z.string().min(1), + label: z.string().min(1), + inputType: inputTypeSchema, + placeholder: z.string().optional(), + defaultValue: z.union([z.string(), z.number(), z.boolean()]).optional(), + options: z.array(inputOptionSchema).optional(), + required: z.boolean().optional(), +}).strict(); + +const formElementSchema = z.object({ + type: z.literal('form'), + formId: z.string().min(1), + title: z.string().optional(), + submitLabel: z.string().min(1), + action: z.string().min(1), + payload: z.record(z.string(), z.unknown()).optional(), + fields: z.array(formFieldSchema).min(1), +}).strict(); + +const cardActionSchema = z.object({ + label: z.string().min(1), + action: z.string().min(1), + payload: z.record(z.string(), z.unknown()).optional(), +}).strict(); + +const cardElementSchema = z.object({ + type: z.literal('card'), + title: z.string().min(1), + body: z.string().min(1), + subtitle: z.string().optional(), + actions: z.array(cardActionSchema).optional(), +}).strict(); + +const imageElementSchema = z.object({ + type: z.literal('image'), + src: z.string().min(1), + alt: z.string().optional(), + caption: z.string().optional(), + action: z.string().min(1).optional(), + payload: z.record(z.string(), z.unknown()).optional(), +}).strict(); + +let assistantPanelElementSchemaRef: z.ZodTypeAny; + +const tabsElementSchema: z.ZodTypeAny = z.lazy(() => z.object({ + type: z.literal('tabs'), + widgetId: z.string().min(1).optional(), + defaultTabId: z.string().min(1).optional(), + tabs: z.array(z.object({ + id: z.string().min(1), + label: z.string().min(1), + elements: z.array(assistantPanelElementSchemaRef).min(1), + }).strict()).min(1), +}).strict()); + +assistantPanelElementSchemaRef = z.union([ + textElementSchema, + metricElementSchema, + listElementSchema, + tableElementSchema, + actionElementSchema, + chartElementSchema, + inputElementSchema, + formElementSchema, + datePickerElementSchema, + cardElementSchema, + imageElementSchema, + tabsElementSchema, +]); + +export const assistantPanelElementSchema = assistantPanelElementSchemaRef; + +export const assistantPanelSpecSchema = z.object({ + specVersion: z.literal('1'), + elements: z.array(assistantPanelElementSchema).min(1), +}).strict(); + +export type AssistantPanelElement = z.infer; +export type AssistantPanelSpec = z.infer; diff --git a/src/main/agentic/protocol/uiSpecParser.ts b/src/main/agentic/protocol/uiSpecParser.ts new file mode 100644 index 0000000..9f054a6 --- /dev/null +++ b/src/main/agentic/protocol/uiSpecParser.ts @@ -0,0 +1,235 @@ +import { assistantPanelSpecSchema, type AssistantPanelSpec } from './uiSchema'; + +function toRecord(value: unknown): Record | null { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + return null; + } + + return value as Record; +} + +function normalizeChartElement(record: Record): Record { + const normalized: Record = { + ...record, + }; + + const dataRecord = toRecord(record.data); + if (Array.isArray(record.series)) { + return normalized; + } + + if (!dataRecord) { + return normalized; + } + + const labels = Array.isArray(dataRecord.labels) ? dataRecord.labels : []; + const datasets = Array.isArray(dataRecord.datasets) ? dataRecord.datasets : []; + const firstDataset = toRecord(datasets[0]); + const values = Array.isArray(firstDataset?.data) ? firstDataset?.data : []; + + if (labels.length === 0 || values.length === 0) { + return normalized; + } + + const series = labels + .map((label, index) => ({ + label: String(label), + value: Number(values[index]), + })) + .filter((entry) => Number.isFinite(entry.value)); + + if (series.length === 0) { + return normalized; + } + + normalized.series = series; + delete normalized.data; + return normalized; +} + +function normalizeTabContent(tabValue: unknown): Record[] { + if (Array.isArray(tabValue)) { + return tabValue + .map((entry) => normalizeElement(entry)) + .filter((entry): entry is Record => Boolean(entry)); + } + + const normalized = normalizeElement(tabValue); + return normalized ? [normalized] : []; +} + +function normalizeTabsElement(record: Record): Record | null { + const tabs = Array.isArray(record.tabs) ? record.tabs : []; + const normalizedTabs = tabs + .map((tabValue, tabIndex) => { + const tabRecord = toRecord(tabValue); + if (!tabRecord) { + return null; + } + + const id = typeof tabRecord.id === 'string' && tabRecord.id.trim().length > 0 + ? tabRecord.id + : `tab-${tabIndex + 1}`; + + const label = typeof tabRecord.label === 'string' && tabRecord.label.trim().length > 0 + ? tabRecord.label + : typeof tabRecord.title === 'string' && tabRecord.title.trim().length > 0 + ? tabRecord.title + : id; + + const elements = Array.isArray(tabRecord.elements) + ? normalizeTabContent(tabRecord.elements) + : normalizeTabContent(tabRecord.content); + + if (elements.length === 0) { + return null; + } + + return { id, label, elements }; + }) + .filter((entry): entry is { id: string; label: string; elements: Record[] } => Boolean(entry)); + + if (normalizedTabs.length === 0) { + return null; + } + + return { + ...record, + tabs: normalizedTabs, + }; +} + +function normalizeElement(value: unknown): Record | null { + const record = toRecord(value); + if (!record) { + return null; + } + + const type = typeof record.type === 'string' ? record.type : ''; + if (type === 'markdown') { + const textValue = typeof record.content === 'string' + ? record.content + : typeof record.text === 'string' + ? record.text + : ''; + + if (!textValue.trim()) { + return null; + } + + return { + type: 'text', + text: textValue, + }; + } + + if (type === 'chart') { + return normalizeChartElement(record); + } + + if (type === 'tabs') { + return normalizeTabsElement(record); + } + + return record; +} + +function normalizeCandidate(parsed: unknown): AssistantPanelSpec | null { + const canonicalResult = assistantPanelSpecSchema.safeParse(parsed); + if (canonicalResult.success) { + return canonicalResult.data; + } + + const record = toRecord(parsed); + if (!record) { + return null; + } + + if (record.type === 'tab' && record.content) { + return normalizeCandidate(record.content); + } + + if (record.type === 'tabs') { + const tabsElement = normalizeTabsElement(record); + if (!tabsElement) { + return null; + } + + const asSpec = { + specVersion: '1' as const, + elements: [tabsElement], + }; + const normalizedResult = assistantPanelSpecSchema.safeParse(asSpec); + return normalizedResult.success ? normalizedResult.data : null; + } + + if (Array.isArray(record.elements)) { + const normalizedElements = record.elements + .map((element) => normalizeElement(element)) + .filter((element): element is Record => Boolean(element)); + + if (normalizedElements.length === 0) { + return null; + } + + const asSpec = { + specVersion: '1' as const, + elements: normalizedElements, + }; + const normalizedResult = assistantPanelSpecSchema.safeParse(asSpec); + return normalizedResult.success ? normalizedResult.data : null; + } + + const normalizedElement = normalizeElement(record); + if (!normalizedElement) { + return null; + } + + const asSpec = { + specVersion: '1' as const, + elements: [normalizedElement], + }; + const normalizedResult = assistantPanelSpecSchema.safeParse(asSpec); + return normalizedResult.success ? normalizedResult.data : null; +} + +function parseSpecCandidate(raw: string): AssistantPanelSpec | null { + try { + const parsed = JSON.parse(raw); + return normalizeCandidate(parsed); + } catch { + return null; + } +} + +export interface ParsedAssistantUiResult { + assistantText: string; + ui: AssistantPanelSpec | null; +} + +export function extractAssistantUiSpec(message: string): ParsedAssistantUiResult { + const trimmed = message.trim(); + + const fencedMatches = [...trimmed.matchAll(/```(json)?\s*([\s\S]*?)```/gi)]; + for (const match of fencedMatches) { + const candidate = match[2]?.trim(); + if (!candidate) { + continue; + } + + const parsed = parseSpecCandidate(candidate); + if (parsed) { + const assistantText = trimmed.replace(match[0], '').trim(); + return { + assistantText, + ui: parsed, + }; + } + } + + const parsedWholeMessage = parseSpecCandidate(trimmed); + return { + assistantText: parsedWholeMessage ? '' : trimmed, + ui: parsedWholeMessage, + }; +} diff --git a/src/main/agentic/protocol/validator.ts b/src/main/agentic/protocol/validator.ts new file mode 100644 index 0000000..a572254 --- /dev/null +++ b/src/main/agentic/protocol/validator.ts @@ -0,0 +1,112 @@ +import { z } from 'zod'; +import type { + ProtocolRequestEnvelope, + ProtocolResponseEnvelope, + ProtocolValidationResult, +} from './types'; +import { createProtocolValidationError } from './errors'; + +const needsInputFieldSchema = z.object({ + key: z.string().min(1), + label: z.string().min(1), + inputType: z.enum(['text', 'textarea', 'select', 'checkbox', 'date', 'number']), + required: z.boolean().optional(), + options: z.array(z.object({ label: z.string().min(1), value: z.string() })).optional(), + placeholder: z.string().optional(), + defaultValue: z.union([z.string(), z.number(), z.boolean()]).optional(), +}).strict(); + +const needsInputSchema = z.object({ + required: z.boolean(), + fields: z.array(needsInputFieldSchema), +}).strict(); + +const protocolActionSchema = z.object({ + id: z.string().min(1), + action: z.string().min(1), + label: z.string().optional(), + payload: z.record(z.string(), z.unknown()).optional(), + policy: z.enum(['silent', 'confirm', 'danger']), + requiresConfirmation: z.boolean(), +}).strict(); + +const protocolUiSchema = z.object({ + specVersion: z.literal('1'), + elements: z.array(z.unknown()), +}).strict(); + +const protocolResponseEnvelopeSchema = z.object({ + protocolVersion: z.literal('2.0'), + assistantText: z.string(), + ui: protocolUiSchema.optional(), + intent: z.enum(['analyze', 'ask_input', 'propose_action', 'execute_action', 'summarize']), + needsInput: needsInputSchema, + actions: z.array(protocolActionSchema), + confidence: z.number().min(0).max(1), + traceId: z.string().min(1), +}).strict().superRefine((value, context) => { + if (value.needsInput.required && value.needsInput.fields.length === 0) { + context.addIssue({ + code: z.ZodIssueCode.custom, + path: ['needsInput', 'fields'], + message: 'needsInput.fields must include at least one field when needsInput.required is true', + }); + } +}); + +const protocolRequestEnvelopeSchema = z.object({ + protocolVersion: z.literal('2.0'), + surface: z.enum(['tab', 'sidebar']), + messages: z.array(z.object({ role: z.enum(['user', 'assistant', 'system', 'tool']), content: z.string() }).strict()), + context: z.record(z.string(), z.unknown()), + capabilities: z.object({ + widgets: z.array(z.string().min(1)), + actions: z.array(z.string().min(1)), + tools: z.array(z.string().min(1)), + disabled: z.array(z.string().min(1)).optional(), + }).strict(), +}).strict(); + +function toErrorMessage(prefix: string, issues: z.ZodIssue[]): string { + const firstIssue = issues[0]; + const issuePath = firstIssue.path.length > 0 ? firstIssue.path.join('.') : 'root'; + return `${prefix}: ${issuePath} ${firstIssue.message}`; +} + +export function validateProtocolResponseEnvelope(input: unknown): ProtocolValidationResult { + const parsed = protocolResponseEnvelopeSchema.safeParse(input); + if (!parsed.success) { + return { + ok: false, + error: createProtocolValidationError( + toErrorMessage('Invalid protocol response envelope', parsed.error.issues), + parsed.error.issues.map((issue) => `${issue.path.join('.')}: ${issue.message}`), + ), + }; + } + + return { + ok: true, + value: parsed.data, + }; +} + +export function validateProtocolRequestEnvelope(input: unknown): ProtocolValidationResult { + const parsed = protocolRequestEnvelopeSchema.safeParse(input); + if (!parsed.success) { + return { + ok: false, + error: createProtocolValidationError( + toErrorMessage('Invalid protocol request envelope', parsed.error.issues), + parsed.error.issues.map((issue) => `${issue.path.join('.')}: ${issue.message}`), + ), + }; + } + + return { + ok: true, + value: parsed.data, + }; +} + +export type { ProtocolRequestEnvelope, ProtocolResponseEnvelope } from './types'; diff --git a/src/main/agentic/workflow/checkpointStore.ts b/src/main/agentic/workflow/checkpointStore.ts new file mode 100644 index 0000000..e1e9faa --- /dev/null +++ b/src/main/agentic/workflow/checkpointStore.ts @@ -0,0 +1,50 @@ +import type { AgentTurnState } from './turnStateMachine'; + +export interface WorkflowCheckpoint { + conversationId: string; + state: AgentTurnState; + pendingFields: string[]; + lastTraceId: string; + updatedAt: string; +} + +export interface WorkflowCheckpointSettingsAdapter { + getSetting(key: string): Promise; + setSetting(key: string, value: string): Promise; +} + +function keyForConversation(conversationId: string): string { + return `agui.workflow.${conversationId}`; +} + +export class WorkflowCheckpointStore { + private readonly adapter: WorkflowCheckpointSettingsAdapter; + + constructor(adapter: WorkflowCheckpointSettingsAdapter) { + this.adapter = adapter; + } + + async save(checkpoint: WorkflowCheckpoint): Promise { + await this.adapter.setSetting( + keyForConversation(checkpoint.conversationId), + JSON.stringify(checkpoint), + ); + } + + async load(conversationId: string): Promise { + const raw = await this.adapter.getSetting(keyForConversation(conversationId)); + if (!raw) { + return null; + } + + try { + const parsed = JSON.parse(raw) as WorkflowCheckpoint; + if (!parsed || parsed.conversationId !== conversationId) { + return null; + } + return parsed; + } catch { + return null; + } + } +} diff --git a/src/main/agentic/workflow/turnStateMachine.ts b/src/main/agentic/workflow/turnStateMachine.ts new file mode 100644 index 0000000..2310aaa --- /dev/null +++ b/src/main/agentic/workflow/turnStateMachine.ts @@ -0,0 +1,45 @@ +export type AgentTurnState = + | 'planning' + | 'awaiting_input' + | 'executing' + | 'observing' + | 'completed'; + +interface TurnStateEnvelopeInput { + intent: 'analyze' | 'ask_input' | 'propose_action' | 'execute_action' | 'summarize'; + needsInput: { + required: boolean; + fields: Array<{ key: string }>; + }; +} + +interface TransitionInput { + previousState: AgentTurnState; + envelope: TurnStateEnvelopeInput; +} + +export class AgentTurnStateMachine { + transition(input: TransitionInput): AgentTurnState { + if (input.envelope.needsInput.required && input.envelope.needsInput.fields.length > 0) { + return 'awaiting_input'; + } + + if (input.envelope.intent === 'execute_action') { + return 'executing'; + } + + if (input.envelope.intent === 'propose_action') { + return 'observing'; + } + + if (input.envelope.intent === 'summarize') { + return 'completed'; + } + + if (input.previousState === 'awaiting_input') { + return 'executing'; + } + + return 'planning'; + } +} diff --git a/src/main/engine/OpenCodeManager.ts b/src/main/engine/OpenCodeManager.ts index a6f82a2..5a6f149 100644 --- a/src/main/engine/OpenCodeManager.ts +++ b/src/main/engine/OpenCodeManager.ts @@ -16,6 +16,13 @@ import { ChatEngine } from './ChatEngine'; import { PostEngine } from './PostEngine'; import { MediaEngine } from './MediaEngine'; import { getPostMediaEngine } from './PostMediaEngine'; +import { ProtocolResponseBuilder } from '../agentic/protocol/responseBuilder'; +import { CapabilityRegistryService } from '../agentic/capabilities/registry'; +import { validateProtocolRequestEnvelope } from '../agentic/protocol/validator'; +import type { ProtocolResponseEnvelope } from '../agentic/protocol/types'; +import { AgentTurnStateMachine, type AgentTurnState } from '../agentic/workflow/turnStateMachine'; +import { WorkflowCheckpointStore } from '../agentic/workflow/checkpointStore'; +import { getProtocolTelemetryService } from '../agentic/observability/protocolTelemetry'; // OpenCode Zen API endpoints const ZEN_ANTHROPIC_URL = 'https://opencode.ai/zen/v1/messages'; @@ -77,6 +84,10 @@ export interface SendMessageOptions { export interface SendMessageResult { success: boolean; message?: string; + envelope?: ProtocolResponseEnvelope; + protocolVersion?: '2.0'; + traceId?: string; + warnings?: string[]; error?: string; toolCalls?: Array<{ name: string; args: unknown }>; } @@ -131,6 +142,10 @@ export class OpenCodeManager { private postEngine: PostEngine; private mediaEngine: MediaEngine; private getMainWindow: () => BrowserWindow | null; + private protocolResponseBuilder: ProtocolResponseBuilder; + private capabilityRegistry: CapabilityRegistryService; + private turnStateMachine: AgentTurnStateMachine; + private workflowCheckpointStore: WorkflowCheckpointStore; private apiKey: string = ''; private abortControllers: Map = new Map(); @@ -144,6 +159,13 @@ export class OpenCodeManager { this.postEngine = postEngine; this.mediaEngine = mediaEngine; this.getMainWindow = getMainWindow; + this.protocolResponseBuilder = new ProtocolResponseBuilder(); + this.capabilityRegistry = new CapabilityRegistryService(); + this.turnStateMachine = new AgentTurnStateMachine(); + this.workflowCheckpointStore = new WorkflowCheckpointStore({ + getSetting: async (key: string) => this.chatEngine.getSetting(key), + setSetting: async (key: string, value: string) => this.chatEngine.setSetting(key, value), + }); } /** @@ -275,10 +297,37 @@ export class OpenCodeManager { // Build message history from DB (excluding system messages) const dbMessages = conversation.messages.filter(m => m.role !== 'system'); + const surface = metadata?.surface || 'tab'; + const capabilities = this.capabilityRegistry.getSnapshot({ surface }); + const requestEnvelope = { + protocolVersion: '2.0' as const, + surface, + messages: dbMessages + .filter((message) => message.role === 'user' || message.role === 'assistant' || message.role === 'system' || message.role === 'tool') + .map((message) => ({ + role: message.role, + content: message.content || '', + })), + context: { + conversationId, + modelId, + }, + capabilities, + }; + + const requestValidation = validateProtocolRequestEnvelope(requestEnvelope); + if (!requestValidation.ok) { + return { + success: false, + error: requestValidation.error?.message || 'Invalid protocol request envelope', + }; + } + const surfaceHint = metadata?.surface ? `\n\n[Client UI surface: ${metadata.surface}. Render response UI for this surface while keeping content functionally equivalent.]` : ''; - const userMessageForModel = `${userMessage}${surfaceHint}`; + const capabilityHint = `\n\n[Protocol request envelope]\n${JSON.stringify(requestEnvelope, null, 2)}`; + const userMessageForModel = `${userMessage}${surfaceHint}${capabilityHint}`; // Add the new user message dbMessages.push({ conversationId, @@ -336,6 +385,38 @@ export class OpenCodeManager { }); } + const protocolResult = this.protocolResponseBuilder.build({ + rawAssistantOutput: fullResponse, + surface, + capabilities, + }); + + const previousCheckpoint = await this.workflowCheckpointStore.load(conversationId); + const previousState: AgentTurnState = previousCheckpoint?.state || 'planning'; + const nextState = this.turnStateMachine.transition({ + previousState, + envelope: { + intent: protocolResult.envelope.intent, + needsInput: protocolResult.envelope.needsInput, + }, + }); + + await this.workflowCheckpointStore.save({ + conversationId, + state: nextState, + pendingFields: protocolResult.envelope.needsInput.fields.map((field) => field.key), + lastTraceId: protocolResult.envelope.traceId, + updatedAt: new Date().toISOString(), + }); + + const blockedActionWarnings = protocolResult.warnings.filter((warning) => warning.includes('Blocked unsupported action')); + getProtocolTelemetryService().recordTurn({ + validEnvelope: !protocolResult.validationError, + repairAttempted: protocolResult.repairAttempted, + fallbackUsed: Boolean(protocolResult.validationError), + blockedActions: blockedActionWarnings.length, + }); + // Generate title after first exchange const userMsgCount = conversation.messages.filter(m => m.role === 'user').length; if (userMsgCount === 0 && fullResponse) { @@ -346,7 +427,11 @@ export class OpenCodeManager { return { success: true, - message: fullResponse, + message: protocolResult.envelope.assistantText, + envelope: protocolResult.envelope, + protocolVersion: protocolResult.envelope.protocolVersion, + traceId: protocolResult.traceId, + warnings: protocolResult.warnings, toolCalls: toolCallsCollected.length > 0 ? toolCallsCollected : undefined, }; } catch (error) { diff --git a/src/main/ipc/chatHandlers.ts b/src/main/ipc/chatHandlers.ts index 4f080c1..1433592 100644 --- a/src/main/ipc/chatHandlers.ts +++ b/src/main/ipc/chatHandlers.ts @@ -8,6 +8,7 @@ import { OpenCodeManager } from '../engine/OpenCodeManager'; import { getPostEngine } from '../engine/PostEngine'; import { getMediaEngine } from '../engine/MediaEngine'; import { getDatabase } from '../database'; +import { getProtocolTelemetryService } from '../agentic/observability/protocolTelemetry'; let chatEngine: ChatEngine | null = null; let openCodeManager: OpenCodeManager | null = null; @@ -135,6 +136,10 @@ export function registerChatHandlers(): void { // ============ Chat Settings ============ + ipcMain.handle('chat:getProtocolHealth', async () => { + return getProtocolTelemetryService().getSnapshot(); + }); + // Get available models ipcMain.handle('chat:getAvailableModels', async () => { try { diff --git a/src/main/preload.ts b/src/main/preload.ts index b3a3e2b..b7fe30a 100644 --- a/src/main/preload.ts +++ b/src/main/preload.ts @@ -286,6 +286,7 @@ export const electronAPI: ElectronAPI = { getApiKey: () => ipcRenderer.invoke('chat:getApiKey'), // Settings + getProtocolHealth: () => ipcRenderer.invoke('chat:getProtocolHealth'), getAvailableModels: () => ipcRenderer.invoke('chat:getAvailableModels'), setDefaultModel: (modelId: string) => ipcRenderer.invoke('chat:setDefaultModel', modelId), getSystemPrompt: () => ipcRenderer.invoke('chat:getSystemPrompt'), diff --git a/src/main/shared/electronApi.ts b/src/main/shared/electronApi.ts index 59b59b7..012af51 100644 --- a/src/main/shared/electronApi.ts +++ b/src/main/shared/electronApi.ts @@ -435,6 +435,53 @@ export interface ChatSendMetadata { surface?: 'tab' | 'sidebar'; } +export interface ProtocolNeedsInputField { + key: string; + label: string; + inputType: 'text' | 'textarea' | 'select' | 'checkbox' | 'date' | 'number'; + required?: boolean; + options?: Array<{ label: string; value: string }>; + placeholder?: string; + defaultValue?: string | number | boolean; +} + +export interface ProtocolAction { + id: string; + action: string; + label?: string; + payload?: Record; + policy: 'silent' | 'confirm' | 'danger'; + requiresConfirmation: boolean; +} + +export interface ProtocolResponseEnvelope { + protocolVersion: '2.0'; + assistantText: string; + ui?: { + specVersion: '1'; + elements: unknown[]; + }; + intent: 'analyze' | 'ask_input' | 'propose_action' | 'execute_action' | 'summarize'; + needsInput: { + required: boolean; + fields: ProtocolNeedsInputField[]; + }; + actions: ProtocolAction[]; + confidence: number; + traceId: string; +} + +export interface ProtocolTelemetrySnapshot { + totalTurns: number; + validEnvelopeTurns: number; + repairAttempts: number; + fallbackTurns: number; + blockedActionCount: number; + parseValidityRate: number; + repairRate: number; + fallbackRate: number; +} + export interface SiteValidationReport { sitemapPath: string; sitemapChanged: boolean; @@ -717,6 +764,7 @@ export interface ElectronAPI { getApiKey: () => Promise; // Settings + getProtocolHealth: () => Promise; getAvailableModels: () => Promise<{ success: boolean; models?: ChatModel[]; selectedModel?: string; error?: string }>; setDefaultModel: (modelId: string) => Promise<{ success: boolean; error?: string }>; getSystemPrompt: () => Promise<{ success: boolean; prompt?: string; error?: string }>; @@ -730,7 +778,7 @@ export interface ElectronAPI { deleteConversation: (id: string) => Promise; // Messaging - sendMessage: (conversationId: string, message: string, metadata?: ChatSendMetadata) => Promise<{ success: boolean; message?: string; error?: string }>; + sendMessage: (conversationId: string, message: string, metadata?: ChatSendMetadata) => Promise<{ success: boolean; message?: string; envelope?: ProtocolResponseEnvelope; protocolVersion?: '2.0'; traceId?: string; warnings?: string[]; error?: string }>; addSystemEvent: (conversationId: string, content: string) => Promise<{ success: boolean; error?: string }>; abortMessage: (conversationId: string) => Promise; getHistory: (conversationId: string) => Promise; diff --git a/src/renderer/components/AssistantPanelControls/AssistantPanelControls.tsx b/src/renderer/components/AssistantPanelControls/AssistantPanelControls.tsx index d5da72b..727739c 100644 --- a/src/renderer/components/AssistantPanelControls/AssistantPanelControls.tsx +++ b/src/renderer/components/AssistantPanelControls/AssistantPanelControls.tsx @@ -5,9 +5,10 @@ import './AssistantPanelControls.css'; interface AssistantPanelControlsProps { elements: AssistantPanelElement[]; onAction: (action: string, payload?: Record) => void; + actionPolicies?: Record; } -export const AssistantPanelControls: React.FC = ({ elements, onAction }) => { +export const AssistantPanelControls: React.FC = ({ elements, onAction, actionPolicies = {} }) => { const [widgetValues, setWidgetValues] = useState>({}); const [activeTabByWidget, setActiveTabByWidget] = useState>({}); @@ -21,6 +22,20 @@ export const AssistantPanelControls: React.FC = ({ const getWidgetValue = (key: string, defaultValue?: unknown) => Object.prototype.hasOwnProperty.call(widgetValues, key) ? widgetValues[key] : defaultValue; + const triggerAction = (action: string, payload?: Record, label?: string) => { + const policy = actionPolicies[action] || 'silent'; + + if (policy !== 'silent') { + const confirmationText = label || action; + const confirmed = window.confirm(confirmationText); + if (!confirmed) { + return; + } + } + + onAction(action, payload); + }; + const renderInputControl = ( key: string, label: string, @@ -150,7 +165,7 @@ export const AssistantPanelControls: React.FC = ({ {element.action && element.submitLabel && ( @@ -175,7 +190,7 @@ export const AssistantPanelControls: React.FC = ({ {element.action && element.submitLabel && ( @@ -191,11 +206,11 @@ export const AssistantPanelControls: React.FC = ({ return accumulator; }, {}); - onAction(element.action, { + triggerAction(element.action, { ...(element.payload ?? {}), formId: element.formId, values, - }); + }, element.submitLabel); }; return ( @@ -224,7 +239,7 @@ export const AssistantPanelControls: React.FC = ({ @@ -243,7 +258,7 @@ export const AssistantPanelControls: React.FC = ({ alt={element.alt || ''} onClick={() => { if (element.action) { - onAction(element.action, element.payload); + triggerAction(element.action, element.payload, element.caption || element.alt || element.action); } }} /> @@ -279,7 +294,7 @@ export const AssistantPanelControls: React.FC = ({ } return ( - ); diff --git a/src/renderer/components/AssistantSidebar/AssistantSidebar.tsx b/src/renderer/components/AssistantSidebar/AssistantSidebar.tsx index 61f6a0e..fefc154 100644 --- a/src/renderer/components/AssistantSidebar/AssistantSidebar.tsx +++ b/src/renderer/components/AssistantSidebar/AssistantSidebar.tsx @@ -4,6 +4,7 @@ import { resolveAssistantEditorContext } from '../../navigation/assistantPromptC import { planAssistantRequest } from '../../navigation/assistantConversation'; import { dispatchAssistantAction } from '../../navigation/assistantActionDispatcher'; import { extractAssistantResponseContent, type AssistantPanelElement } from '../../navigation/assistantPanelSpec'; +import { toClarificationElements } from '../../navigation/protocolNeedsInput'; import { ensureConversationId } from '../../navigation/chatSession'; import { getChatSurfaceMode } from '../../navigation/chatSurfaceMode'; import { useChatMessageSender } from '../../navigation/useChatMessageSender'; @@ -22,6 +23,7 @@ export const AssistantSidebar: React.FC = () => { const [errorMessage, setErrorMessage] = useState(null); const [conversationId, setConversationId] = useState(null); const [panelElements, setPanelElements] = useState([]); + const [actionPolicies, setActionPolicies] = useState>({}); const [actionError, setActionError] = useState(null); const { @@ -127,10 +129,23 @@ export const AssistantSidebar: React.FC = () => { throw new Error(sendResult.error || 'Failed to send assistant message'); } - if (sendResult.message) { + if (sendResult.envelope) { + finalizeAssistantTurn(resolvedConversationId, sendResult.envelope.assistantText); + const uiElements = Array.isArray(sendResult.envelope.ui?.elements) + ? (sendResult.envelope.ui?.elements as AssistantPanelElement[]) + : toClarificationElements(sendResult.envelope.needsInput); + setPanelElements(uiElements); + setActionPolicies( + sendResult.envelope.actions.reduce>((accumulator, action) => { + accumulator[action.action] = action.policy; + return accumulator; + }, {}), + ); + } else if (sendResult.message) { const parsedResponse = extractAssistantResponseContent(sendResult.message); finalizeAssistantTurn(resolvedConversationId, parsedResponse.displayText); setPanelElements(parsedResponse.panelSpec?.elements ?? []); + setActionPolicies({}); } else { appendAssistantMessage(resolvedConversationId, tr('chat.errorEmptyResponse')); stopStreaming(); @@ -146,6 +161,61 @@ export const AssistantSidebar: React.FC = () => { }; const handleAssistantAction = (action: string, payload?: Record) => { + if (action === 'submitNeedsInput' && conversationId) { + const values = payload?.values; + if (!values || typeof values !== 'object') { + setActionError(tr('assistantSidebar.error.actionFailed')); + return; + } + + const clarificationMessage = `needs_input_response: ${JSON.stringify(values)}`; + + beginUserTurn(conversationId, clarificationMessage); + + void sendChatMessage({ + conversationId, + message: clarificationMessage, + metadata: { surface: 'sidebar' }, + }).then((sendResult) => { + if (!sendResult.success) { + appendAssistantMessage( + conversationId, + tr('chat.errorPrefix', { error: sendResult.error || tr('chat.errorNoResponse') }), + ); + stopStreaming(); + return; + } + + if (sendResult.envelope) { + finalizeAssistantTurn(conversationId, sendResult.envelope.assistantText); + const uiElements = Array.isArray(sendResult.envelope.ui?.elements) + ? (sendResult.envelope.ui?.elements as AssistantPanelElement[]) + : toClarificationElements(sendResult.envelope.needsInput); + setPanelElements(uiElements); + setActionPolicies( + sendResult.envelope.actions.reduce>((accumulator, action) => { + accumulator[action.action] = action.policy; + return accumulator; + }, {}), + ); + return; + } + + if (sendResult.message) { + const parsedResponse = extractAssistantResponseContent(sendResult.message); + finalizeAssistantTurn(conversationId, parsedResponse.displayText); + setPanelElements(parsedResponse.panelSpec?.elements ?? []); + setActionPolicies({}); + } + }).catch((error) => { + console.error('Failed to submit assistant clarification:', error); + appendAssistantMessage(conversationId, tr('chat.errorGeneric')); + stopStreaming(); + }); + + return; + } + const result = dispatchAssistantAction( { action, @@ -230,7 +300,7 @@ export const AssistantSidebar: React.FC = () => { )} {panelElements.length > 0 && ( - + )} ); diff --git a/src/renderer/components/ChatPanel/ChatPanel.tsx b/src/renderer/components/ChatPanel/ChatPanel.tsx index bda0ff9..d442704 100644 --- a/src/renderer/components/ChatPanel/ChatPanel.tsx +++ b/src/renderer/components/ChatPanel/ChatPanel.tsx @@ -5,6 +5,7 @@ import { useChatSurfaceState } from '../../navigation/useChatSurfaceState'; import { getChatSurfaceMode } from '../../navigation/chatSurfaceMode'; import { dispatchAssistantAction } from '../../navigation/assistantActionDispatcher'; import { extractAssistantResponseContent, type AssistantPanelElement } from '../../navigation/assistantPanelSpec'; +import { toClarificationElements } from '../../navigation/protocolNeedsInput'; import { useAppStore } from '../../store'; import { ChatTranscript } from '../ChatSurface'; import { AssistantPanelControls } from '../AssistantPanelControls'; @@ -28,6 +29,7 @@ export const ChatPanel: React.FC = ({ conversationId }) => { const [apiKeyError, setApiKeyError] = useState(''); const [isValidating, setIsValidating] = useState(false); const [panelElements, setPanelElements] = useState([]); + const [actionPolicies, setActionPolicies] = useState>({}); const [actionError, setActionError] = useState(null); const messagesEndRef = useRef(null); const inputRef = useRef(null); @@ -190,21 +192,36 @@ export const ChatPanel: React.FC = ({ conversationId }) => { // Fall back to the backend result message if streaming didn't capture the content const assistantContent = getStreamingContent() || (result.success ? result.message : ''); - if (assistantContent) { + if (result.envelope) { + finalizeAssistantTurn(conversationId, result.envelope.assistantText); + const uiElements = Array.isArray(result.envelope.ui?.elements) + ? (result.envelope.ui?.elements as AssistantPanelElement[]) + : toClarificationElements(result.envelope.needsInput); + setPanelElements(uiElements); + setActionPolicies( + result.envelope.actions.reduce>((accumulator, action) => { + accumulator[action.action] = action.policy; + return accumulator; + }, {}), + ); + } else if (assistantContent) { const parsedResponse = extractAssistantResponseContent(assistantContent); finalizeAssistantTurn(conversationId, parsedResponse.displayText); setPanelElements(parsedResponse.panelSpec?.elements ?? []); + setActionPolicies({}); } else if (!result.success) { // Backend returned an error (API failure, model unavailable, etc.) appendAssistantMessage(conversationId, tr('chat.errorPrefix', { error: result.error || tr('chat.errorNoResponse') })); stopStreaming(); setPanelElements([]); + setActionPolicies({}); } else { // No content from streaming AND no error, but also no success message // This can happen with some models that don't return content properly appendAssistantMessage(conversationId, tr('chat.errorEmptyResponse')); stopStreaming(); setPanelElements([]); + setActionPolicies({}); } } catch (error) { console.error('Failed to send message:', error); @@ -226,7 +243,64 @@ export const ChatPanel: React.FC = ({ conversationId }) => { } }; + const handleNeedsInputSubmit = async (payload?: Record) => { + const values = payload?.values; + if (!values || typeof values !== 'object') { + setActionError(tr('assistantSidebar.error.actionFailed')); + return; + } + + const clarificationMessage = `needs_input_response: ${JSON.stringify(values)}`; + beginUserTurn(conversationId, clarificationMessage); + + try { + const result = await sendChatMessage({ + conversationId, + message: clarificationMessage, + metadata: { surface: 'tab' }, + }); + + if (!result.success) { + appendAssistantMessage(conversationId, tr('chat.errorPrefix', { error: result.error || tr('chat.errorNoResponse') })); + stopStreaming(); + return; + } + + if (result.envelope) { + finalizeAssistantTurn(conversationId, result.envelope.assistantText); + const uiElements = Array.isArray(result.envelope.ui?.elements) + ? (result.envelope.ui?.elements as AssistantPanelElement[]) + : toClarificationElements(result.envelope.needsInput); + setPanelElements(uiElements); + setActionPolicies( + result.envelope.actions.reduce>((accumulator, action) => { + accumulator[action.action] = action.policy; + return accumulator; + }, {}), + ); + return; + } + + const assistantContent = getStreamingContent() || result.message; + if (assistantContent) { + const parsedResponse = extractAssistantResponseContent(assistantContent); + finalizeAssistantTurn(conversationId, parsedResponse.displayText); + setPanelElements(parsedResponse.panelSpec?.elements ?? []); + setActionPolicies({}); + } + } catch (error) { + console.error('Failed to submit clarification:', error); + appendAssistantMessage(conversationId, tr('chat.errorGeneric')); + stopStreaming(); + } + }; + const handleAssistantAction = (action: string, payload?: Record) => { + if (action === 'submitNeedsInput') { + void handleNeedsInputSubmit(payload); + return; + } + const result = dispatchAssistantAction( { action, @@ -377,7 +451,7 @@ export const ChatPanel: React.FC = ({ conversationId }) => { /> {panelElements.length > 0 && ( - + )} {actionError &&

{actionError}

} diff --git a/src/renderer/components/Editor/Editor.tsx b/src/renderer/components/Editor/Editor.tsx index c7b27db..e28eeea 100644 --- a/src/renderer/components/Editor/Editor.tsx +++ b/src/renderer/components/Editor/Editor.tsx @@ -1478,6 +1478,12 @@ interface CategoryCount { count: number; } +interface DashboardProtocolHealth { + blockedActionCount: number; + parseValidityRate: number; + fallbackTurns: number; +} + const Dashboard: React.FC = () => { const { t: tr, language } = useI18n(); const { posts, media } = useAppStore(); @@ -1486,6 +1492,7 @@ const Dashboard: React.FC = () => { const [tagCounts, setTagCounts] = useState([]); const [tagColors, setTagColors] = useState>(new Map()); const [categoryCounts, setCategoryCounts] = useState([]); + const [protocolHealth, setProtocolHealth] = useState(null); const uiDateLocale = UI_DATE_LOCALE[language] || UI_DATE_LOCALE.en; const monthFormatter = useMemo( @@ -1496,17 +1503,25 @@ const Dashboard: React.FC = () => { useEffect(() => { const loadStats = async () => { try { - const [ds, ym, tc, cc, colorMap] = await Promise.all([ + const [ds, ym, tc, cc, colorMap, protocolHealthSnapshot] = await Promise.all([ window.electronAPI?.posts.getDashboardStats(), window.electronAPI?.posts.getByYearMonth(), window.electronAPI?.posts.getTagsWithCounts(), window.electronAPI?.posts.getCategoriesWithCounts(), loadTagColorMap(), + window.electronAPI?.chat.getProtocolHealth(), ]); if (ds) setStats(ds); if (ym) setYearMonthData(ym); if (tc) setTagCounts(tc); if (cc) setCategoryCounts(cc); + if (protocolHealthSnapshot) { + setProtocolHealth({ + blockedActionCount: protocolHealthSnapshot.blockedActionCount, + parseValidityRate: protocolHealthSnapshot.parseValidityRate, + fallbackTurns: protocolHealthSnapshot.fallbackTurns, + }); + } setTagColors(colorMap); } catch (e) { console.error('Failed to load dashboard stats:', e); @@ -1551,6 +1566,9 @@ const Dashboard: React.FC = () => { const displayDraftCount = stats?.draftCount ?? 0; const displayPublishedCount = stats?.publishedCount ?? 0; const displayArchivedCount = stats?.archivedCount ?? 0; + const parseValidityPercent = protocolHealth + ? `${Math.round(protocolHealth.parseValidityRate * 100)}%` + : '—'; const getPostCountLabel = useCallback((count: number) => { return tr(count === 1 ? 'dashboard.postCount.one' : 'dashboard.postCount.other', { count }); @@ -1597,6 +1615,14 @@ const Dashboard: React.FC = () => { {tr('dashboard.stats.categories', { count: categoryCounts.length })} +
+
{parseValidityPercent}
+
{tr('dashboard.stats.protocolHealth')}
+
+ {tr('dashboard.stats.blockedActions', { count: protocolHealth?.blockedActionCount ?? 0 })} + {tr('dashboard.stats.fallbackTurns', { count: protocolHealth?.fallbackTurns ?? 0 })} +
+
{timelineEntries.length > 0 && ( diff --git a/src/renderer/i18n/locales/de.json b/src/renderer/i18n/locales/de.json index 8efa43f..7e114b7 100644 --- a/src/renderer/i18n/locales/de.json +++ b/src/renderer/i18n/locales/de.json @@ -528,6 +528,9 @@ "dashboard.stats.images": "{count} Bilder", "dashboard.stats.tags": "Schlagwörter", "dashboard.stats.categories": "{count} Kategorien", + "dashboard.stats.protocolHealth": "Protokollzustand", + "dashboard.stats.blockedActions": "{count} blockierte Aktionen", + "dashboard.stats.fallbackTurns": "{count} Fallback-Durchläufe", "dashboard.section.postsOverTime": "Beiträge im Zeitverlauf", "dashboard.section.tags": "Schlagwörter", "dashboard.section.categories": "Kategorien", diff --git a/src/renderer/i18n/locales/en.json b/src/renderer/i18n/locales/en.json index 9f819f2..3f61762 100644 --- a/src/renderer/i18n/locales/en.json +++ b/src/renderer/i18n/locales/en.json @@ -528,6 +528,9 @@ "dashboard.stats.images": "{count} images", "dashboard.stats.tags": "Tags", "dashboard.stats.categories": "{count} categories", + "dashboard.stats.protocolHealth": "Protocol Health", + "dashboard.stats.blockedActions": "{count} blocked actions", + "dashboard.stats.fallbackTurns": "{count} fallback turns", "dashboard.section.postsOverTime": "Posts Over Time", "dashboard.section.tags": "Tags", "dashboard.section.categories": "Categories", diff --git a/src/renderer/i18n/locales/es.json b/src/renderer/i18n/locales/es.json index dd09a89..c444d5b 100644 --- a/src/renderer/i18n/locales/es.json +++ b/src/renderer/i18n/locales/es.json @@ -528,6 +528,9 @@ "dashboard.stats.images": "{count} imágenes", "dashboard.stats.tags": "Etiquetas", "dashboard.stats.categories": "{count} categorías", + "dashboard.stats.protocolHealth": "Salud del protocolo", + "dashboard.stats.blockedActions": "{count} acciones bloqueadas", + "dashboard.stats.fallbackTurns": "{count} respuestas de respaldo", "dashboard.section.postsOverTime": "Entradas a lo largo del tiempo", "dashboard.section.tags": "Etiquetas", "dashboard.section.categories": "Categorías", diff --git a/src/renderer/i18n/locales/fr.json b/src/renderer/i18n/locales/fr.json index 5c9fa60..31346f2 100644 --- a/src/renderer/i18n/locales/fr.json +++ b/src/renderer/i18n/locales/fr.json @@ -528,6 +528,9 @@ "dashboard.stats.images": "{count} images", "dashboard.stats.tags": "Étiquettes", "dashboard.stats.categories": "{count} catégories", + "dashboard.stats.protocolHealth": "Santé du protocole", + "dashboard.stats.blockedActions": "{count} actions bloquées", + "dashboard.stats.fallbackTurns": "{count} tours de secours", "dashboard.section.postsOverTime": "Articles dans le temps", "dashboard.section.tags": "Étiquettes", "dashboard.section.categories": "Catégories", diff --git a/src/renderer/i18n/locales/it.json b/src/renderer/i18n/locales/it.json index a912c18..6895876 100644 --- a/src/renderer/i18n/locales/it.json +++ b/src/renderer/i18n/locales/it.json @@ -528,6 +528,9 @@ "dashboard.stats.images": "{count} immagini", "dashboard.stats.tags": "Tag", "dashboard.stats.categories": "{count} categorie", + "dashboard.stats.protocolHealth": "Salute del protocollo", + "dashboard.stats.blockedActions": "{count} azioni bloccate", + "dashboard.stats.fallbackTurns": "{count} risposte di fallback", "dashboard.section.postsOverTime": "Post nel tempo", "dashboard.section.tags": "Tag", "dashboard.section.categories": "Categorie", diff --git a/src/renderer/navigation/chatSession.ts b/src/renderer/navigation/chatSession.ts index 2047985..620140a 100644 --- a/src/renderer/navigation/chatSession.ts +++ b/src/renderer/navigation/chatSession.ts @@ -1,10 +1,12 @@ +import type { ProtocolResponseEnvelope } from '../types/electron'; + export interface ChatService { createConversation: (title?: string, model?: string) => Promise<{ id: string } | null | undefined>; sendMessage: ( conversationId: string, message: string, metadata?: SendMessageMetadata, - ) => Promise<{ success: boolean; message?: string; error?: string } | null | undefined>; + ) => Promise<{ success: boolean; message?: string; envelope?: ProtocolResponseEnvelope; protocolVersion?: '2.0'; traceId?: string; warnings?: string[]; error?: string } | null | undefined>; } export interface SendMessageMetadata { @@ -27,6 +29,10 @@ export interface SendConversationMessageInput { export interface SendConversationMessageResult { success: boolean; message: string; + envelope?: ProtocolResponseEnvelope; + protocolVersion?: '2.0'; + traceId?: string; + warnings?: string[]; error?: string; } @@ -69,5 +75,9 @@ export async function sendConversationMessage( return { success: true, message: result.message || '', + envelope: result.envelope, + protocolVersion: result.protocolVersion, + traceId: result.traceId, + warnings: result.warnings, }; } diff --git a/src/renderer/navigation/protocolNeedsInput.ts b/src/renderer/navigation/protocolNeedsInput.ts new file mode 100644 index 0000000..1aa9236 --- /dev/null +++ b/src/renderer/navigation/protocolNeedsInput.ts @@ -0,0 +1,38 @@ +import type { ProtocolNeedsInputField, ProtocolResponseEnvelope } from '../types/electron'; +import type { AssistantPanelElement } from './assistantPanelSpec'; + +function toFormField(field: ProtocolNeedsInputField): { + key: string; + label: string; + inputType: 'text' | 'textarea' | 'select' | 'checkbox' | 'date' | 'number'; + placeholder?: string; + defaultValue?: string | number | boolean; + options?: Array<{ label: string; value: string }>; + required?: boolean; +} { + return { + key: field.key, + label: field.label, + inputType: field.inputType, + placeholder: field.placeholder, + defaultValue: field.defaultValue, + options: field.options, + required: field.required, + }; +} + +export function toClarificationElements( + needsInput: ProtocolResponseEnvelope['needsInput'], +): AssistantPanelElement[] { + if (!needsInput.required || needsInput.fields.length === 0) { + return []; + } + + return [{ + type: 'form', + formId: 'agui-needs-input', + submitLabel: needsInput.fields[0].label, + action: 'submitNeedsInput', + fields: needsInput.fields.map(toFormField), + }]; +} diff --git a/src/renderer/python/pythonApiContractV1.ts b/src/renderer/python/pythonApiContractV1.ts index bc263f5..fdf3111 100644 --- a/src/renderer/python/pythonApiContractV1.ts +++ b/src/renderer/python/pythonApiContractV1.ts @@ -176,6 +176,7 @@ const METHODS_V1: PythonApiMethodContractV1[] = [ method('chat.validateApiKey', 'Validate chat API key and list available models.', [requiredString('apiKey')], '{ isValid: boolean; models: ChatModel[] }'), method('chat.setApiKey', 'Store chat API key.', [requiredString('apiKey')], '{ success: boolean; error?: string }'), method('chat.getApiKey', 'Get stored chat API key status.', [], 'ChatApiKeyStatus'), + method('chat.getProtocolHealth', 'Get AGUI protocol telemetry health snapshot.', [], 'ProtocolTelemetrySnapshot'), method('chat.getAvailableModels', 'Get available chat models and selected default.', [], '{ success: boolean; models?: ChatModel[]; selectedModel?: string; error?: string }'), method('chat.setDefaultModel', 'Set default chat model.', [requiredString('modelId')], '{ success: boolean; error?: string }'), method('chat.getSystemPrompt', 'Get configured system prompt.', [], '{ success: boolean; prompt?: string; error?: string }'), @@ -359,11 +360,25 @@ const DATA_STRUCTURES_V1: PythonApiDataStructureContractV1[] = [ { name: 'maskedKey', type: 'string', required: true, description: 'Masked key representation for UI display.' }, ], }, + { + name: 'ProtocolTelemetrySnapshot', + description: 'Aggregated protocol telemetry metrics for AGUI response health.', + fields: [ + { name: 'totalTurns', type: 'number', required: true, description: 'Total number of recorded assistant turns.' }, + { name: 'validEnvelopeTurns', type: 'number', required: true, description: 'Turns with schema-valid protocol envelopes.' }, + { name: 'repairAttempts', type: 'number', required: true, description: 'Number of response repair attempts.' }, + { name: 'fallbackTurns', type: 'number', required: true, description: 'Turns that used protocol fallback response.' }, + { name: 'blockedActionCount', type: 'number', required: true, description: 'Count of actions blocked by policy.' }, + { name: 'parseValidityRate', type: 'number', required: true, description: 'Ratio of valid envelopes to total turns.' }, + { name: 'repairRate', type: 'number', required: true, description: 'Ratio of repair attempts to total turns.' }, + { name: 'fallbackRate', type: 'number', required: true, description: 'Ratio of fallback turns to total turns.' }, + ], + }, ]; export const BDS_PYTHON_API_CONTRACT_V1: PythonApiContractV1 = { - version: '1.3.0', - generatedAt: '2026-02-24T00:00:00.000Z', + version: '1.4.0', + generatedAt: '2026-02-25T00:00:00.000Z', methods: METHODS_V1, dataStructures: DATA_STRUCTURES_V1, }; diff --git a/tests/engine/agentic/capabilities/registry.test.ts b/tests/engine/agentic/capabilities/registry.test.ts new file mode 100644 index 0000000..af1755d --- /dev/null +++ b/tests/engine/agentic/capabilities/registry.test.ts @@ -0,0 +1,32 @@ +import { describe, expect, it } from 'vitest'; +import { CapabilityRegistryService } from '../../../../src/main/agentic/capabilities/registry'; + +describe('CapabilityRegistryService', () => { + it('returns per-surface capability differences', () => { + const registry = new CapabilityRegistryService(); + + const tabCapabilities = registry.getSnapshot({ surface: 'tab' }); + const sidebarCapabilities = registry.getSnapshot({ surface: 'sidebar' }); + + expect(tabCapabilities.widgets).toContain('tabs'); + expect(sidebarCapabilities.widgets).toContain('tabs'); + expect(tabCapabilities.actions).toContain('toggleSidebar'); + expect(sidebarCapabilities.actions).toContain('toggleAssistantSidebar'); + expect(tabCapabilities.actions).not.toContain('toggleAssistantSidebar'); + }); + + it('omits disabled capabilities from active lists', () => { + const registry = new CapabilityRegistryService({ + disabledActions: ['openSettings'], + disabledWidgets: ['chart'], + disabledTools: ['view_image'], + }); + + const snapshot = registry.getSnapshot({ surface: 'tab' }); + + expect(snapshot.actions).not.toContain('openSettings'); + expect(snapshot.widgets).not.toContain('chart'); + expect(snapshot.tools).not.toContain('view_image'); + expect(snapshot.disabled).toEqual(expect.arrayContaining(['action:openSettings', 'widget:chart', 'tool:view_image'])); + }); +}); diff --git a/tests/engine/agentic/observability/protocolTelemetry.test.ts b/tests/engine/agentic/observability/protocolTelemetry.test.ts new file mode 100644 index 0000000..99bf897 --- /dev/null +++ b/tests/engine/agentic/observability/protocolTelemetry.test.ts @@ -0,0 +1,38 @@ +import { describe, expect, it } from 'vitest'; +import { ProtocolTelemetryService } from '../../../../src/main/agentic/observability/protocolTelemetry'; + +describe('ProtocolTelemetryService', () => { + it('tracks parse validity, repairs, fallback, and blocked actions', () => { + const telemetry = new ProtocolTelemetryService(); + + telemetry.recordTurn({ + validEnvelope: true, + repairAttempted: false, + fallbackUsed: false, + blockedActions: 0, + }); + + telemetry.recordTurn({ + validEnvelope: true, + repairAttempted: true, + fallbackUsed: false, + blockedActions: 1, + }); + + telemetry.recordTurn({ + validEnvelope: false, + repairAttempted: true, + fallbackUsed: true, + blockedActions: 2, + }); + + const snapshot = telemetry.getSnapshot(); + + expect(snapshot.totalTurns).toBe(3); + expect(snapshot.validEnvelopeTurns).toBe(2); + expect(snapshot.repairAttempts).toBe(2); + expect(snapshot.fallbackTurns).toBe(1); + expect(snapshot.blockedActionCount).toBe(3); + expect(snapshot.parseValidityRate).toBeCloseTo(2 / 3, 5); + }); +}); diff --git a/tests/engine/agentic/policy/actionPolicy.test.ts b/tests/engine/agentic/policy/actionPolicy.test.ts new file mode 100644 index 0000000..4014fc4 --- /dev/null +++ b/tests/engine/agentic/policy/actionPolicy.test.ts @@ -0,0 +1,21 @@ +import { describe, expect, it } from 'vitest'; +import { resolveActionPolicy } from '../../../../src/main/agentic/policy/actionPolicy'; + +describe('action policy', () => { + it('marks dangerous actions as requiring explicit confirmation', () => { + const policy = resolveActionPolicy('deletePost'); + expect(policy.level).toBe('danger'); + expect(policy.requiresConfirmation).toBe(true); + }); + + it('marks configurable but safe navigation actions as confirm', () => { + const policy = resolveActionPolicy('openSettings'); + expect(policy.level).toBe('confirm'); + expect(policy.requiresConfirmation).toBe(true); + }); + + it('defaults unknown actions to danger', () => { + const policy = resolveActionPolicy('unknownAction'); + expect(policy.level).toBe('danger'); + }); +}); diff --git a/tests/engine/agentic/protocol/responseBuilder.test.ts b/tests/engine/agentic/protocol/responseBuilder.test.ts new file mode 100644 index 0000000..bd02737 --- /dev/null +++ b/tests/engine/agentic/protocol/responseBuilder.test.ts @@ -0,0 +1,117 @@ +import { describe, expect, it } from 'vitest'; +import { ProtocolResponseBuilder } from '../../../../src/main/agentic/protocol/responseBuilder'; + +describe('ProtocolResponseBuilder', () => { + it('builds canonical envelope from mixed text + AGUI payload', () => { + const builder = new ProtocolResponseBuilder(); + + const raw = [ + 'I found weak months.', + '```json', + '{"specVersion":"1","elements":[{"type":"chart","chartType":"bar","series":[{"label":"Jan","value":10}]}]}', + '```', + ].join('\n'); + + const result = builder.build({ + rawAssistantOutput: raw, + surface: 'tab', + capabilities: { + widgets: ['chart'], + actions: ['openPost'], + tools: ['search_posts'], + }, + }); + + expect(result.envelope.ui?.elements).toHaveLength(1); + expect(result.envelope.assistantText).toContain('I found weak months'); + expect(result.envelope.protocolVersion).toBe('2.0'); + expect(result.repairAttempted).toBe(false); + }); + + it('repairs non-canonical envelope keys and validates output', () => { + const builder = new ProtocolResponseBuilder(); + + const raw = JSON.stringify({ + protocol_version: '2.0', + assistant_text: 'Need more details', + intent: 'ask_input', + needs_input: { + required: true, + fields: [{ key: 'date', label: 'Date', inputType: 'date' }], + }, + actions: [], + confidence: 0.8, + trace_id: 'trace-manual', + }); + + const result = builder.build({ + rawAssistantOutput: raw, + surface: 'sidebar', + capabilities: { + widgets: ['form'], + actions: ['openPost'], + tools: ['search_posts'], + }, + }); + + expect(result.repairAttempted).toBe(true); + expect(result.envelope.assistantText).toBe('Need more details'); + expect(result.envelope.needsInput.required).toBe(true); + expect(result.envelope.needsInput.fields).toHaveLength(1); + expect(result.validationError).toBeUndefined(); + }); + + it('falls back to safe summarize envelope when payload is invalid', () => { + const builder = new ProtocolResponseBuilder(); + + const raw = '{"specVersion":"9","elements":[]}'; + + const result = builder.build({ + rawAssistantOutput: raw, + surface: 'tab', + capabilities: { + widgets: ['chart'], + actions: ['openPost'], + tools: ['search_posts'], + }, + }); + + expect(result.envelope.intent).toBe('summarize'); + expect(result.envelope.ui).toBeUndefined(); + expect(result.envelope.assistantText).toContain('specVersion'); + expect(result.traceId.length).toBeGreaterThan(0); + }); + + it('blocks actions that are unavailable for the active surface capabilities', () => { + const builder = new ProtocolResponseBuilder(); + + const raw = JSON.stringify({ + protocolVersion: '2.0', + assistantText: 'Open settings?', + intent: 'propose_action', + needsInput: { required: false, fields: [] }, + actions: [{ + id: 'a1', + action: 'openSettings', + label: 'Open Settings', + policy: 'confirm', + requiresConfirmation: true, + }], + confidence: 0.7, + traceId: 'trace-abc', + }); + + const result = builder.build({ + rawAssistantOutput: raw, + surface: 'tab', + capabilities: { + widgets: ['chart'], + actions: ['openPost'], + tools: ['search_posts'], + }, + }); + + expect(result.envelope.actions).toHaveLength(0); + expect(result.warnings.some((warning) => warning.includes('openSettings'))).toBe(true); + }); +}); diff --git a/tests/engine/agentic/protocol/validator.test.ts b/tests/engine/agentic/protocol/validator.test.ts new file mode 100644 index 0000000..d769390 --- /dev/null +++ b/tests/engine/agentic/protocol/validator.test.ts @@ -0,0 +1,74 @@ +import { describe, expect, it } from 'vitest'; +import { + validateProtocolRequestEnvelope, + validateProtocolResponseEnvelope, + type ProtocolResponseEnvelope, +} from '../../../../src/main/agentic/protocol/validator'; + +describe('agentic protocol validator', () => { + it('validates canonical response envelope', () => { + const envelope: ProtocolResponseEnvelope = { + protocolVersion: '2.0', + assistantText: 'Done', + intent: 'summarize', + needsInput: { + required: false, + fields: [], + }, + actions: [], + confidence: 0.82, + traceId: 'trace-abc', + }; + + const result = validateProtocolResponseEnvelope(envelope); + expect(result.ok).toBe(true); + expect(result.error).toBeUndefined(); + }); + + it('rejects response envelope with unknown properties in strict mode', () => { + const result = validateProtocolResponseEnvelope({ + protocolVersion: '2.0', + assistantText: 'Done', + intent: 'summarize', + needsInput: { required: false, fields: [] }, + actions: [], + confidence: 0.8, + traceId: 'trace-abc', + extra: 'nope', + }); + + expect(result.ok).toBe(false); + expect(result.error?.code).toBe('AGUI_PROTOCOL_VALIDATION_ERROR'); + }); + + it('rejects needsInput.required=true without fields', () => { + const result = validateProtocolResponseEnvelope({ + protocolVersion: '2.0', + assistantText: 'Need details', + intent: 'ask_input', + needsInput: { required: true, fields: [] }, + actions: [], + confidence: 0.9, + traceId: 'trace-xyz', + }); + + expect(result.ok).toBe(false); + expect(result.error?.message).toContain('needsInput.fields'); + }); + + it('validates canonical request envelope with capabilities', () => { + const result = validateProtocolRequestEnvelope({ + protocolVersion: '2.0', + surface: 'tab', + messages: [{ role: 'user', content: 'Create a chart' }], + context: { projectId: 'project-1' }, + capabilities: { + widgets: ['chart', 'form'], + actions: ['openPost'], + tools: ['search_posts'], + }, + }); + + expect(result.ok).toBe(true); + }); +}); diff --git a/tests/engine/agentic/workflow/checkpointStore.test.ts b/tests/engine/agentic/workflow/checkpointStore.test.ts new file mode 100644 index 0000000..4081327 --- /dev/null +++ b/tests/engine/agentic/workflow/checkpointStore.test.ts @@ -0,0 +1,39 @@ +import { describe, expect, it } from 'vitest'; +import { + WorkflowCheckpointStore, + type WorkflowCheckpointSettingsAdapter, +} from '../../../../src/main/agentic/workflow/checkpointStore'; + +class InMemorySettingsAdapter implements WorkflowCheckpointSettingsAdapter { + private readonly store = new Map(); + + async getSetting(key: string): Promise { + return this.store.get(key) ?? null; + } + + async setSetting(key: string, value: string): Promise { + this.store.set(key, value); + } +} + +describe('WorkflowCheckpointStore', () => { + it('persists and reloads workflow checkpoints by conversation id', async () => { + const adapter = new InMemorySettingsAdapter(); + const store = new WorkflowCheckpointStore(adapter); + + await store.save({ + conversationId: 'conversation-1', + state: 'awaiting_input', + pendingFields: ['date'], + lastTraceId: 'trace-1', + updatedAt: new Date('2026-02-25T10:00:00.000Z').toISOString(), + }); + + const loaded = await store.load('conversation-1'); + + expect(loaded).not.toBeNull(); + expect(loaded?.state).toBe('awaiting_input'); + expect(loaded?.pendingFields).toEqual(['date']); + expect(loaded?.lastTraceId).toBe('trace-1'); + }); +}); diff --git a/tests/engine/agentic/workflow/stateMachine.test.ts b/tests/engine/agentic/workflow/stateMachine.test.ts new file mode 100644 index 0000000..2144d36 --- /dev/null +++ b/tests/engine/agentic/workflow/stateMachine.test.ts @@ -0,0 +1,38 @@ +import { describe, expect, it } from 'vitest'; +import { AgentTurnStateMachine } from '../../../../src/main/agentic/workflow/turnStateMachine'; + +describe('AgentTurnStateMachine', () => { + it('transitions to awaiting_input when envelope requests required input', () => { + const stateMachine = new AgentTurnStateMachine(); + + const next = stateMachine.transition({ + previousState: 'planning', + envelope: { + intent: 'ask_input', + needsInput: { + required: true, + fields: [{ key: 'date', label: 'Date', inputType: 'date' }], + }, + }, + }); + + expect(next).toBe('awaiting_input'); + }); + + it('transitions to completed when summarize intent has no required input', () => { + const stateMachine = new AgentTurnStateMachine(); + + const next = stateMachine.transition({ + previousState: 'observing', + envelope: { + intent: 'summarize', + needsInput: { + required: false, + fields: [], + }, + }, + }); + + expect(next).toBe('completed'); + }); +}); diff --git a/tests/renderer/components/EditorDashboardTimeline.test.tsx b/tests/renderer/components/EditorDashboardTimeline.test.tsx index 717bf9f..2a925a2 100644 --- a/tests/renderer/components/EditorDashboardTimeline.test.tsx +++ b/tests/renderer/components/EditorDashboardTimeline.test.tsx @@ -55,6 +55,18 @@ describe('Editor dashboard timeline', () => { ]); (window as any).electronAPI.posts.getTagsWithCounts = vi.fn().mockResolvedValue([]); (window as any).electronAPI.posts.getCategoriesWithCounts = vi.fn().mockResolvedValue([]); + (window as any).electronAPI.chat = { + getProtocolHealth: vi.fn().mockResolvedValue({ + totalTurns: 10, + validEnvelopeTurns: 9, + repairAttempts: 1, + fallbackTurns: 0, + blockedActionCount: 2, + parseValidityRate: 0.9, + repairRate: 0.1, + fallbackRate: 0, + }), + }; (window as any).electronAPI.tags = { getAll: vi.fn().mockResolvedValue([]), }; @@ -82,4 +94,17 @@ describe('Editor dashboard timeline', () => { expect(screen.getByText('2024')).toBeInTheDocument(); }); + + it('renders protocol telemetry stats in dashboard', async () => { + render(); + + await act(async () => { + await Promise.resolve(); + await Promise.resolve(); + await Promise.resolve(); + }); + + expect(screen.getByText('90%')).toBeInTheDocument(); + expect(screen.getByText('2 blocked actions')).toBeInTheDocument(); + }); }); \ No newline at end of file diff --git a/tests/renderer/navigation/protocolNeedsInput.test.ts b/tests/renderer/navigation/protocolNeedsInput.test.ts new file mode 100644 index 0000000..c32dac2 --- /dev/null +++ b/tests/renderer/navigation/protocolNeedsInput.test.ts @@ -0,0 +1,30 @@ +import { describe, expect, it } from 'vitest'; +import { toClarificationElements } from '../../../src/renderer/navigation/protocolNeedsInput'; + +describe('protocolNeedsInput', () => { + it('builds a clarification form element when required fields are provided', () => { + const elements = toClarificationElements({ + required: true, + fields: [ + { key: 'date', label: 'Date', inputType: 'date', required: true }, + { key: 'category', label: 'Category', inputType: 'select', options: [{ label: 'A', value: 'a' }] }, + ], + }); + + expect(elements).toHaveLength(1); + expect(elements[0]).toMatchObject({ + type: 'form', + formId: 'agui-needs-input', + action: 'submitNeedsInput', + }); + }); + + it('returns empty elements when input is not required', () => { + const elements = toClarificationElements({ + required: false, + fields: [], + }); + + expect(elements).toEqual([]); + }); +}); diff --git a/tests/renderer/python/pythonApiContractV1.test.ts b/tests/renderer/python/pythonApiContractV1.test.ts index 3517d95..b5eee7e 100644 --- a/tests/renderer/python/pythonApiContractV1.test.ts +++ b/tests/renderer/python/pythonApiContractV1.test.ts @@ -25,6 +25,7 @@ describe('pythonApiContractV1', () => { 'app.getSystemLanguage', 'chat.getConversations', 'chat.sendMessage', + 'chat.getProtocolHealth', ])); }); @@ -45,7 +46,7 @@ describe('pythonApiContractV1', () => { it('contains semantic version metadata for compatibility checks', () => { expect(BDS_PYTHON_API_CONTRACT_V1).toMatchObject({ - version: '1.3.0', + version: '1.4.0', generatedAt: expect.any(String), }); }); @@ -55,6 +56,7 @@ describe('pythonApiContractV1', () => { expect.objectContaining({ name: 'PostData' }), expect.objectContaining({ name: 'MediaData' }), expect.objectContaining({ name: 'ProjectData' }), + expect.objectContaining({ name: 'ProtocolTelemetrySnapshot' }), ])); }); }); diff --git a/tests/renderer/python/pythonApiInvokerV1.test.ts b/tests/renderer/python/pythonApiInvokerV1.test.ts index 4bc0bd0..85bb955 100644 --- a/tests/renderer/python/pythonApiInvokerV1.test.ts +++ b/tests/renderer/python/pythonApiInvokerV1.test.ts @@ -29,12 +29,16 @@ describe('invokePythonApiMethodV1', () => { const getProjectMetadata = vi.fn().mockResolvedValue({ name: 'My Project' }); const getAllProjects = vi.fn().mockResolvedValue([{ id: 'prj-1', name: 'Main' }]); const getAllPosts = vi.fn().mockResolvedValue({ items: [], hasMore: false, total: 0 }); + const getProtocolHealth = vi.fn().mockResolvedValue({ totalTurns: 1, parseValidityRate: 1 }); vi.stubGlobal('window', { electronAPI: { projects: { getAll: getAllProjects, }, + chat: { + getProtocolHealth, + }, posts: { search: searchPosts, getAll: getAllPosts, @@ -49,10 +53,12 @@ describe('invokePythonApiMethodV1', () => { await expect(invokePythonApiMethodV1('posts.getAll', { options: { limit: 10, offset: 5 } })).resolves.toEqual({ items: [], hasMore: false, total: 0 }); await expect(invokePythonApiMethodV1('posts.search', { query: 'hit' })).resolves.toEqual([{ id: 'p1', title: 'Hit' }]); await expect(invokePythonApiMethodV1('meta.getProjectMetadata', {})).resolves.toEqual({ name: 'My Project' }); + await expect(invokePythonApiMethodV1('chat.getProtocolHealth', {})).resolves.toEqual({ totalTurns: 1, parseValidityRate: 1 }); expect(getAllProjects).toHaveBeenCalledWith(); expect(getAllPosts).toHaveBeenCalledWith({ limit: 10, offset: 5 }); expect(searchPosts).toHaveBeenCalledWith('hit'); expect(getProjectMetadata).toHaveBeenCalledWith(); + expect(getProtocolHealth).toHaveBeenCalledWith(); }); it('rejects unknown methods and malformed args', async () => { @@ -66,6 +72,9 @@ describe('invokePythonApiMethodV1', () => { projects: { getAll: vi.fn(), }, + chat: { + getProtocolHealth: vi.fn(), + }, meta: { getProjectMetadata: vi.fn(), },