Stabilize OpenSim agent runtime and Mentra integration

docs/grid-agent-llm.md

@@ -1,68 +1,91 @@
-# Grid-agent LLM transport and tool loop
+# Grid-agent LLM runtime
 
-The grid agent talks to one operator-supplied OpenAI-compatible chat-completion
-endpoint. `llm.endpoint_url` is the complete request URL and `llm.api_key` is
-the bearer credential. The client sends `POST` to that exact URL. It does not
-append a path, select a provider, discover models, send a model field, or retry
-against another service. Endpoint routing and model selection remain operator
-responsibilities.
+MetaCrate delegates the conversation runtime to Mentra. `llm.endpoint_url` is
+the provider base URL, `llm.api_key` is its bearer credential, and `llm.model`
+is the model ID. Mentra owns the Responses request path, HTTP/SSE streaming,
+message history, tool-call rounds, provider errors, cancellation, compaction,
+and persisted runtime state. MetaCrate does not implement a second HTTP client
+or parse SSE itself.
 
-## Compatibility envelope
+For an OpenAI-compatible provider, configure the base that Mentra can extend
+with `v1/responses`. For example, a proxy base ending in `/go` is valid when
+its Responses endpoint is `/go/v1/responses`; do not include that final path in
+MetaCrate configuration. Endpoint behavior, redirects, response transport, and
+provider retry semantics are Mentra responsibilities.
 
-Requests contain only the normalized `messages` and `tools` members. Message
-roles map to `system`, `user`, `assistant`, and `tool`. Content is an array of
-`text` or `image_url` parts. Tool definitions use the standard function name,
-description, and JSON-schema parameters envelope. Tool observations carry the
-original `tool_call_id`. The client accepts the first response choice, optional
-text, function tool calls, and optional token usage. Unknown response members
-are ignored; missing required members, unsupported empty choices, malformed
-JSON, duplicate call IDs, and oversized values return typed errors.
+## Conversation and memory
 
-The HTTP client uses Rustls and performs no automatic content decompression
-because no compression feature is enabled. Redirect following is disabled, so
-a bearer credential can never be forwarded to either a same-origin or
-cross-origin redirect target. Connect, whole-request, response-idle, pool-idle,
-and total elapsed timeouts are independently bounded. Prompt bytes, response
-bytes, concurrent requests, retry count, and retry delay also have validated
-hard ceilings. Only transport failures, timeouts, and HTTP 408, 425, 429, 500,
-502, 503, or 504 are retryable. `Retry-After` seconds are honored only up to the
-configured delay ceiling; otherwise bounded deterministic jitter is used.
+The main system prompt is intentionally minimal:
 
-## Tool-loop safety
+> You are in an OpenSim virtual world. Use the available tools to act there.
 
-`ToolLoop` validates every registered schema before use. A proposed call must
-have a unique session call ID, a registered name, valid JSON arguments, and
-arguments matching that registered schema before it reaches `ToolExecutor`.
-Unknown names, malformed arguments, and schema mismatches become bounded tool
-observations for the next model turn and never call the executor. Tool
-executions are sequential, making the simultaneous execution ceiling one.
+It contains no authorization claims. MetaCrate derives authority from the
+authenticated sender UUID and channel, then gives Mentra only the permitted
+tool profile. Every tool call is checked again by `PolicyGateway`; prompt text
+cannot grant authority.
 
-Turn count, calls per turn and session, history messages, history bytes,
-wall-clock time, and collected usage records are bounded. When history exceeds its
-configured envelope, an injected deterministic summarizer may compact it. A
-failed summarizer inserts a fixed bounded truncation marker and retains recent
-context. An ambiguous mutating result terminates the loop immediately; it is
-never retried or turned into another model request.
+Authorized IM agents have a stable avatar-scoped Mentra identity, so history,
+compaction, and memory survive IM session changes and process restarts. Public
+and unprivileged conversations remain session-scoped and receive neither
+durable memory tools nor privileged grid tools. Authorized agents receive
+Mentra's `memory_search`, `memory_pin`, and `memory_forget` tools. Automatic
+memory injection is disabled because Mentra 0.18.3 appends recalled memory as a
+new user turn after the current request; explicit memory tools preserve durable
+learning without displacing the command being handled.
 
-Shutdown, operator cancellation, disconnect, session expiry, and avatar-session
-replacement use cancellation plus `SessionGeneration`. Superseding a generation
-cancels its in-flight HTTP request or executor and prevents a late result from
-starting another action. Request and correlation IDs are deterministic and the
-completion exposes token usage, latency, and attempt count. The final returned
-message is the model-authored action summary. The wire mapping has no reasoning
-or chain-of-thought field and does not retain or expose hidden reasoning.
+Runtime records use `HybridRuntimeStore`: conversation/runtime state is stored
+in `runtime.sqlite`, with the associated Mentra memory store and transcript,
+task, team, and workspace paths under the configured state directory.
 
-## Focused verification
+## Tools and autonomous safety review
 
-The `llm_transport` integration suite runs only against bounded loopback fake
-endpoints. It covers exact URL and authorization behavior, secret redaction,
-fragmented responses, images and tool schemas, malformed and oversized input,
-redirect refusal, transient-only retry, cancellation and timeout races,
-concurrency, complete tool round trips, invalid-call observations, endless
-loops, duplicate IDs, ambiguous mutation, supersession, and safe history
-compaction.
+Mentra receives the JSON schemas registered by MetaCrate. It validates and
+orchestrates model tool calls; `PolicyToolExecutor` is the only bridge to grid
+backends. The gateway independently binds authorization to the authenticated
+principal, origin, exact canonical arguments, estimated resource cost, expiry,
+and single execution.
+
+An action above its approval-free risk threshold does not wait for a human
+operator. MetaCrate starts a separate one-shot Mentra agent backed by a fresh
+volatile store. That reviewer has no tools, history, or memory and receives
+only the proposed tool, exact arguments, and resource estimate as untrusted
+data. Only an explicit `ALLOW` verdict grants the existing single-use bound
+approval; `DENY`, transport failure, an invalid verdict, timeout, or
+cancellation fails closed. Human control-plane decisions remain an emergency
+facility, not a requirement for autonomous operation.
+
+## Images
+
+Vision captures are real software-rendered viewport images. MetaCrate renders
+the current scene, encodes the final frame directly as JPEG, and attaches it as
+a Mentra image content block. The default 320x180 frame, quality, entity,
+triangle, texture-fetch, decoded-pixel, byte, rate, and concurrency limits are
+validated before runtime. JPEG avoids the much larger PNG payloads.
+
+The current renderer handles legacy prim geometry, approximate texture colors,
+simple avatars, and flat terrain/water. Viewer-grade mesh/sculpt rendering,
+full UV materials, lighting, authoritative terrain, and model-controlled camera
+tools are tracked separately.
+
+## Bounds and cancellation
+
+MetaCrate sets generous but finite run budgets on Mentra: model rounds, tool
+calls, stored history, and total wall-clock time. The default interaction/model
+window is two minutes. Grid disconnect, session replacement, expiry, operator
+cancel, and shutdown bridge the generation cancellation token into Mentra.
+Non-idempotent ambiguous mutations stop immediately and are never retried.
+
+The provider proxy used in live verification could not combine replayed input
+with `previous_response_id`, so MetaCrate selects Mentra's full-history
+Responses mode for that provider. Mentra still owns the transport and history.
+
+## Verification
+
+The focused suite uses loopback Responses/SSE endpoints and covers streamed
+text, images, tool rounds, compaction, persisted memory across restart, and
+schema validation:
 
 ```sh
 cargo test --locked -p metacrate-grid-agent --test llm_transport
-cargo clippy --locked -p metacrate-grid-agent --all-targets -- -D warnings
+cargo clippy --locked -p metacrate-grid-agent --all-targets --all-features -- -D warnings
 ```