feat(grid-agent): add generic LLM tool loop (#119)
This commit is contained in:
68
docs/grid-agent-llm.md
Normal file
68
docs/grid-agent-llm.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# Grid-agent LLM transport and tool loop
|
||||
|
||||
The grid agent talks to one operator-supplied OpenAI-compatible chat-completion
|
||||
endpoint. `llm.endpoint_url` is the complete request URL and `llm.api_key` is
|
||||
the bearer credential. The client sends `POST` to that exact URL. It does not
|
||||
append a path, select a provider, discover models, send a model field, or retry
|
||||
against another service. Endpoint routing and model selection remain operator
|
||||
responsibilities.
|
||||
|
||||
## Compatibility envelope
|
||||
|
||||
Requests contain only the normalized `messages` and `tools` members. Message
|
||||
roles map to `system`, `user`, `assistant`, and `tool`. Content is an array of
|
||||
`text` or `image_url` parts. Tool definitions use the standard function name,
|
||||
description, and JSON-schema parameters envelope. Tool observations carry the
|
||||
original `tool_call_id`. The client accepts the first response choice, optional
|
||||
text, function tool calls, and optional token usage. Unknown response members
|
||||
are ignored; missing required members, unsupported empty choices, malformed
|
||||
JSON, duplicate call IDs, and oversized values return typed errors.
|
||||
|
||||
The HTTP client uses Rustls and performs no automatic content decompression
|
||||
because no compression feature is enabled. Redirect following is disabled, so
|
||||
a bearer credential can never be forwarded to either a same-origin or
|
||||
cross-origin redirect target. Connect, whole-request, response-idle, pool-idle,
|
||||
and total elapsed timeouts are independently bounded. Prompt bytes, response
|
||||
bytes, concurrent requests, retry count, and retry delay also have validated
|
||||
hard ceilings. Only transport failures, timeouts, and HTTP 408, 425, 429, 500,
|
||||
502, 503, or 504 are retryable. `Retry-After` seconds are honored only up to the
|
||||
configured delay ceiling; otherwise bounded deterministic jitter is used.
|
||||
|
||||
## Tool-loop safety
|
||||
|
||||
`ToolLoop` validates every registered schema before use. A proposed call must
|
||||
have a unique session call ID, a registered name, valid JSON arguments, and
|
||||
arguments matching that registered schema before it reaches `ToolExecutor`.
|
||||
Unknown names, malformed arguments, and schema mismatches become bounded tool
|
||||
observations for the next model turn and never call the executor. Tool
|
||||
executions are sequential, making the simultaneous execution ceiling one.
|
||||
|
||||
Turn count, calls per turn and session, history messages, history bytes,
|
||||
wall-clock time, and collected usage records are bounded. When history exceeds its
|
||||
configured envelope, an injected deterministic summarizer may compact it. A
|
||||
failed summarizer inserts a fixed bounded truncation marker and retains recent
|
||||
context. An ambiguous mutating result terminates the loop immediately; it is
|
||||
never retried or turned into another model request.
|
||||
|
||||
Shutdown, operator cancellation, disconnect, session expiry, and avatar-session
|
||||
replacement use cancellation plus `SessionGeneration`. Superseding a generation
|
||||
cancels its in-flight HTTP request or executor and prevents a late result from
|
||||
starting another action. Request and correlation IDs are deterministic and the
|
||||
completion exposes token usage, latency, and attempt count. The final returned
|
||||
message is the model-authored action summary. The wire mapping has no reasoning
|
||||
or chain-of-thought field and does not retain or expose hidden reasoning.
|
||||
|
||||
## Focused verification
|
||||
|
||||
The `llm_transport` integration suite runs only against bounded loopback fake
|
||||
endpoints. It covers exact URL and authorization behavior, secret redaction,
|
||||
fragmented responses, images and tool schemas, malformed and oversized input,
|
||||
redirect refusal, transient-only retry, cancellation and timeout races,
|
||||
concurrency, complete tool round trips, invalid-call observations, endless
|
||||
loops, duplicate IDs, ambiguous mutation, supersession, and safe history
|
||||
compaction.
|
||||
|
||||
```sh
|
||||
cargo test --locked -p metacrate-grid-agent --test llm_transport
|
||||
cargo clippy --locked -p metacrate-grid-agent --all-targets -- -D warnings
|
||||
```
|
||||
Reference in New Issue
Block a user