92 lines
4.5 KiB
Markdown
92 lines
4.5 KiB
Markdown
# Grid-agent LLM runtime
|
|
|
|
MetaCrate delegates the conversation runtime to Mentra. `llm.endpoint_url` is
|
|
the provider base URL, `llm.api_key` is its bearer credential, and `llm.model`
|
|
is the model ID. Mentra owns the Responses request path, HTTP/SSE streaming,
|
|
message history, tool-call rounds, provider errors, cancellation, compaction,
|
|
and persisted runtime state. MetaCrate does not implement a second HTTP client
|
|
or parse SSE itself.
|
|
|
|
For an OpenAI-compatible provider, configure the base that Mentra can extend
|
|
with `v1/responses`. For example, a proxy base ending in `/go` is valid when
|
|
its Responses endpoint is `/go/v1/responses`; do not include that final path in
|
|
MetaCrate configuration. Endpoint behavior, redirects, response transport, and
|
|
provider retry semantics are Mentra responsibilities.
|
|
|
|
## Conversation and memory
|
|
|
|
The main system prompt is intentionally minimal:
|
|
|
|
> You are in an OpenSim virtual world. Use the available tools to act there.
|
|
|
|
It contains no authorization claims. MetaCrate derives authority from the
|
|
authenticated sender UUID and channel, then gives Mentra only the permitted
|
|
tool profile. Every tool call is checked again by `PolicyGateway`; prompt text
|
|
cannot grant authority.
|
|
|
|
Authorized IM agents have a stable avatar-scoped Mentra identity, so history,
|
|
compaction, and memory survive IM session changes and process restarts. Public
|
|
and unprivileged conversations remain session-scoped and receive neither
|
|
durable memory tools nor privileged grid tools. Authorized agents receive
|
|
Mentra's `memory_search`, `memory_pin`, and `memory_forget` tools. Automatic
|
|
memory injection is disabled because Mentra 0.18.3 appends recalled memory as a
|
|
new user turn after the current request; explicit memory tools preserve durable
|
|
learning without displacing the command being handled.
|
|
|
|
Runtime records use `HybridRuntimeStore`: conversation/runtime state is stored
|
|
in `runtime.sqlite`, with the associated Mentra memory store and transcript,
|
|
task, team, and workspace paths under the configured state directory.
|
|
|
|
## Tools and autonomous safety review
|
|
|
|
Mentra receives the JSON schemas registered by MetaCrate. It validates and
|
|
orchestrates model tool calls; `PolicyToolExecutor` is the only bridge to grid
|
|
backends. The gateway independently binds authorization to the authenticated
|
|
principal, origin, exact canonical arguments, estimated resource cost, expiry,
|
|
and single execution.
|
|
|
|
An action above its approval-free risk threshold does not wait for a human
|
|
operator. MetaCrate starts a separate one-shot Mentra agent backed by a fresh
|
|
volatile store. That reviewer has no tools, history, or memory and receives
|
|
only the proposed tool, exact arguments, and resource estimate as untrusted
|
|
data. Only an explicit `ALLOW` verdict grants the existing single-use bound
|
|
approval; `DENY`, transport failure, an invalid verdict, timeout, or
|
|
cancellation fails closed. Human control-plane decisions remain an emergency
|
|
facility, not a requirement for autonomous operation.
|
|
|
|
## Images
|
|
|
|
Vision captures are real software-rendered viewport images. MetaCrate renders
|
|
the current scene, encodes the final frame directly as JPEG, and attaches it as
|
|
a Mentra image content block. The default 320x180 frame, quality, entity,
|
|
triangle, texture-fetch, decoded-pixel, byte, rate, and concurrency limits are
|
|
validated before runtime. JPEG avoids the much larger PNG payloads.
|
|
|
|
The current renderer handles legacy prim geometry, approximate texture colors,
|
|
simple avatars, and flat terrain/water. Viewer-grade mesh/sculpt rendering,
|
|
full UV materials, lighting, authoritative terrain, and model-controlled camera
|
|
tools are tracked separately.
|
|
|
|
## Bounds and cancellation
|
|
|
|
MetaCrate sets generous but finite run budgets on Mentra: model rounds, tool
|
|
calls, stored history, and total wall-clock time. The default interaction/model
|
|
window is two minutes. Grid disconnect, session replacement, expiry, operator
|
|
cancel, and shutdown bridge the generation cancellation token into Mentra.
|
|
Non-idempotent ambiguous mutations stop immediately and are never retried.
|
|
|
|
The provider proxy used in live verification could not combine replayed input
|
|
with `previous_response_id`, so MetaCrate selects Mentra's full-history
|
|
Responses mode for that provider. Mentra still owns the transport and history.
|
|
|
|
## Verification
|
|
|
|
The focused suite uses loopback Responses/SSE endpoints and covers streamed
|
|
text, images, tool rounds, compaction, persisted memory across restart, and
|
|
schema validation:
|
|
|
|
```sh
|
|
cargo test --locked -p metacrate-grid-agent --test llm_transport
|
|
cargo clippy --locked -p metacrate-grid-agent --all-targets --all-features -- -D warnings
|
|
```
|