[Grid agent] Implement the generic OpenAI-compatible LLM transport and tool loop #119

New Issue

hugo · 2026-08-17T19:31:34Z

hugo commented

2026-08-17 19:31:34 +00:00

Objective

Implement one provider-neutral, pure-Rust HTTP client for an exact operator-supplied OpenAI-compatible endpoint and a bounded reasoning/tool-call loop that works with inexpensive or self-hosted models.

Endpoint contract

The only LLM connection configuration is endpoint_url plus api_key; post to the exact URL rather than appending provider paths. Send the key as bearer authorization and never log it.
Do not add a provider enum, predefined hosts, provider SDKs, model list/discovery, fallback vendors, or provider-specific request branches. The endpoint/gateway is responsible for model routing; document the minimal request/response compatibility envelope.
Use Rust HTTP/TLS and Serde dependencies with bounded connect, request, response, and idle timeouts; response decompression and redirect behavior must be explicit and safe. Refuse redirects that could forward authorization to another origin.
Support normalized text messages, system instructions, standard tool definitions/calls, and image content when supplied by the later vision issue. Preserve unknown harmless response fields and return typed errors for unsupported response shapes.

Tool loop and resilience

Validate tool arguments against the registered schema before policy evaluation or execution. Unknown tools and malformed arguments are observations returned to the model, never panics.
Bound prompt bytes, response bytes, turns, tool calls per turn/session, concurrent requests, retries, and total wall-clock time. Retry only transient transport/rate-limit/server failures with capped jittered backoff and Retry-After; never retry an ambiguous mutating tool execution.
Support cancellation from shutdown, operator control, session expiry, and disconnect. Late model results must not act on a superseded session.
Provide deterministic request IDs and correlation IDs, usage/latency metadata when returned, and concise model-authored action summaries. Do not request, persist, or expose hidden chain-of-thought.
Compact oversized history deterministically through bounded summaries; a failed summary must degrade to safe truncation.

Acceptance criteria

A fake HTTP endpoint verifies exact URL use, auth redaction, request schema, tool round trips, streaming/fragmentation if implemented, cancellation, timeouts, retry classification, and response limits.
Tests cover malformed JSON, HTML errors, oversized bodies, redirect credential leakage, duplicate tool IDs, unknown tools, endless tool loops, disconnect races, and ambiguous mutation failures.
The client remains responsive while several avatar sessions wait on slow inference; bounded concurrency prevents starvation and memory growth.
No provider-specific code or required model/provider setting is introduced.

Dependencies and exclusions

Depends on the architecture/configuration issue. Policy decisions and actual world tools belong to their dedicated issues; this issue provides the generic execution loop and test transport.

## Objective Implement one provider-neutral, pure-Rust HTTP client for an exact operator-supplied OpenAI-compatible endpoint and a bounded reasoning/tool-call loop that works with inexpensive or self-hosted models. ## Endpoint contract - The only LLM connection configuration is `endpoint_url` plus `api_key`; post to the exact URL rather than appending provider paths. Send the key as bearer authorization and never log it. - Do not add a provider enum, predefined hosts, provider SDKs, model list/discovery, fallback vendors, or provider-specific request branches. The endpoint/gateway is responsible for model routing; document the minimal request/response compatibility envelope. - Use Rust HTTP/TLS and Serde dependencies with bounded connect, request, response, and idle timeouts; response decompression and redirect behavior must be explicit and safe. Refuse redirects that could forward authorization to another origin. - Support normalized text messages, system instructions, standard tool definitions/calls, and image content when supplied by the later vision issue. Preserve unknown harmless response fields and return typed errors for unsupported response shapes. ## Tool loop and resilience - Validate tool arguments against the registered schema before policy evaluation or execution. Unknown tools and malformed arguments are observations returned to the model, never panics. - Bound prompt bytes, response bytes, turns, tool calls per turn/session, concurrent requests, retries, and total wall-clock time. Retry only transient transport/rate-limit/server failures with capped jittered backoff and `Retry-After`; never retry an ambiguous mutating tool execution. - Support cancellation from shutdown, operator control, session expiry, and disconnect. Late model results must not act on a superseded session. - Provide deterministic request IDs and correlation IDs, usage/latency metadata when returned, and concise model-authored action summaries. Do not request, persist, or expose hidden chain-of-thought. - Compact oversized history deterministically through bounded summaries; a failed summary must degrade to safe truncation. ## Acceptance criteria - [ ] A fake HTTP endpoint verifies exact URL use, auth redaction, request schema, tool round trips, streaming/fragmentation if implemented, cancellation, timeouts, retry classification, and response limits. - [ ] Tests cover malformed JSON, HTML errors, oversized bodies, redirect credential leakage, duplicate tool IDs, unknown tools, endless tool loops, disconnect races, and ambiguous mutation failures. - [ ] The client remains responsive while several avatar sessions wait on slow inference; bounded concurrency prevents starvation and memory growth. - [ ] No provider-specific code or required model/provider setting is introduced. ## Dependencies and exclusions Depends on the architecture/configuration issue. Policy decisions and actual world tools belong to their dedicated issues; this issue provides the generic execution loop and test transport.

hugo added this to the 14 - metacrate grid agent milestone 2026-08-17 19:31:34 +00:00

hugo added the enhancement label 2026-08-17 19:31:34 +00:00

hugo referenced this issue

2026-08-17 19:32:14 +00:00

implement an AI driven agent that runs in opensim #117

hugo referenced this issue from a commit

2026-08-17 20:46:04 +00:00

feat(grid-agent): add generic LLM tool loop (#119)

hugo commented

2026-08-17 20:46:27 +00:00

Implemented in commit a46bc42.

Implementation:

Added a provider-neutral Reqwest/Rustls client that posts to the exact configured endpoint, sends redacted bearer authorization, refuses redirects, disables automatic decompression features, and applies bounded connect/request/read-idle/pool/total timeouts.
Added normalized system/text/image/tool wire mappings, typed response errors, deterministic request and correlation IDs, usage/latency metadata, response-size enforcement, transient-only capped retries, Retry-After handling, and shared fair concurrency limits.
Added the bounded multi-turn ToolLoop with registered-schema validation before execution, safe observations for unknown/malformed calls, per-turn/session limits, session-wide call-ID deduplication, deterministic compaction/fallback truncation, ambiguous-mutation termination, and active cancellation/generation fencing for late model and executor results.
Documented the exact compatibility envelope and safety contract without adding provider/model configuration or world-policy behavior.

Issue-scoped verification:

cargo test --locked -p metacrate-grid-agent --all-targets: 30 passed, including 14 fake-endpoint adversarial cases.
cargo clippy --offline -p metacrate-grid-agent --all-targets -- -D warnings: passed.
rustdoc with -D warnings, rustfmt, git diff checks, dependency policy, cargo-deny, and cargo-audit: passed.
Linux target check passed. Windows and macOS cross-checks reached the shared Rustls AWS-LC dependency but this Linux host lacks the MinGW compiler and macOS cross C toolchain; no project-code portability error was reported. The source uses portable Tokio/Reqwest APIs and adds no platform-specific implementation.
cargo-machete reported only the pre-existing unrelated md-5 finding in libremetaverse-types; the new reqwest dependency is used.

No credentials or .env content were read or committed; all endpoint tests use bounded loopback fakes.

Implemented in commit a46bc42. Implementation: - Added a provider-neutral Reqwest/Rustls client that posts to the exact configured endpoint, sends redacted bearer authorization, refuses redirects, disables automatic decompression features, and applies bounded connect/request/read-idle/pool/total timeouts. - Added normalized system/text/image/tool wire mappings, typed response errors, deterministic request and correlation IDs, usage/latency metadata, response-size enforcement, transient-only capped retries, Retry-After handling, and shared fair concurrency limits. - Added the bounded multi-turn ToolLoop with registered-schema validation before execution, safe observations for unknown/malformed calls, per-turn/session limits, session-wide call-ID deduplication, deterministic compaction/fallback truncation, ambiguous-mutation termination, and active cancellation/generation fencing for late model and executor results. - Documented the exact compatibility envelope and safety contract without adding provider/model configuration or world-policy behavior. Issue-scoped verification: - cargo test --locked -p metacrate-grid-agent --all-targets: 30 passed, including 14 fake-endpoint adversarial cases. - cargo clippy --offline -p metacrate-grid-agent --all-targets -- -D warnings: passed. - rustdoc with -D warnings, rustfmt, git diff checks, dependency policy, cargo-deny, and cargo-audit: passed. - Linux target check passed. Windows and macOS cross-checks reached the shared Rustls AWS-LC dependency but this Linux host lacks the MinGW compiler and macOS cross C toolchain; no project-code portability error was reported. The source uses portable Tokio/Reqwest APIs and adds no platform-specific implementation. - cargo-machete reported only the pre-existing unrelated md-5 finding in libremetaverse-types; the new reqwest dependency is used. No credentials or .env content were read or committed; all endpoint tests use bounded loopback fakes.

hugo closed this issue

2026-08-17 20:46:27 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/MetaCrate#119