Package portable grid agent operation (#134)

2026-08-18 12:27:07 +02:00
parent fe8d0119ef
commit 00707fd8df
14 changed files with 693 additions and 9 deletions
--- a/docs/grid-agent-operations.md
+++ b/docs/grid-agent-operations.md
@@ -0,0 +1,262 @@
+# MetaCrate grid-agent operations
+
+This is the operator runbook for the portable `metacrate-grid-agent` binary.
+The binary has no dependency on systemd, a Windows service manager, a CLR,
+Python, Node.js, a provider SDK, or a shell at runtime. Linux service files and
+install helpers are packaging conveniences around the same foreground process
+and versioned control protocol.
+
+## Quick start and modes
+
+Build the credential-free binary and validate the example without contacting a
+grid or LLM:
+
+```sh
+cargo build --locked --release -p metacrate-grid-agent
+target/release/metacrate-grid-agent --config config/grid-agent.example.json --check-config
+target/release/metacrate-grid-agent --config config/grid-agent.example.json --run-once
+```
+
+`--check-config` parses files, resolves secret files/environment, validates all
+bounds and TLS files, then exits before constructing the grid or LLM clients.
+The example is fake/offline mode (`integrated=false`, `split=false`) and all
+credential-looking values are placeholders.
+
+For live foreground or an embedded TUI, build intentionally:
+
+```sh
+cargo build --locked --release -p metacrate-grid-agent --features live-grid
+target/release/metacrate-grid-agent --config /path/to/grid-agent.json
+target/release/metacrate-grid-agent --config /path/to/grid-agent.json --tui
+```
+
+Set `integrated=true` for the foreground and embedded-TUI commands. Set
+`split=true` for a headless service, then run the same binary as the TUI client:
+
+```sh
+metacrate-grid-agent --config /path/to/grid-agent.json
+metacrate-grid-agent --config /path/to/grid-agent.json --tui-client
+```
+
+The TUI client and service must use the same control address and operator token.
+`--tui-client` never needs the grid password or LLM key. `--run-once` performs
+one supervised readiness/login and clean logout cycle. `--print-paths` prints
+the platform defaults without loading configuration. If `--config` is omitted,
+the default file is used only when it already exists:
+
+| Platform | Configuration | Non-secret state |
+|---|---|---|
+| Linux/Unix | `$XDG_CONFIG_HOME/metacrate/grid-agent.json`, otherwise `$HOME/.config/metacrate/grid-agent.json` | `$XDG_DATA_HOME/metacrate/grid-agent`, otherwise `$HOME/.local/share/metacrate/grid-agent` |
+| macOS | `$HOME/Library/Application Support/MetaCrate/grid-agent.json` | `$HOME/Library/Application Support/MetaCrate/grid-agent` |
+| Windows | `%APPDATA%\MetaCrate\grid-agent.json` | `%LOCALAPPDATA%\MetaCrate\grid-agent` |
+
+`METACRATE_AGENT_STORAGE_PATH` overrides the state directory. Relative paths in
+a JSON document resolve relative to that document where specified; service
+deployments should use absolute paths.
+
+## Endpoint, grid identity, and authority
+
+`llm.endpoint_url` is the exact OpenAI-compatible chat-completions URL. It is
+not a base URL: MetaCrate does not append a path, discover models, select a
+provider, or rewrite query parameters. `llm.api_key` is sent as the bearer key
+only to that exact origin; redirects are refused. Prefer
+`METACRATE_AGENT_LLM_API_KEY_FILE` over inline JSON or direct environment text.
+
+Live modes require `grid.login_url`, `grid.avatar_name`, and a password from
+`METACRATE_AGENT_GRID_PASSWORD_FILE`. `authorized_avatar_uuids` contains exact
+grid UUIDs, never display names. Text claiming an authorized identity grants no
+authority. Public chat can request bounded informational work and safe public
+LSL delivery; movement, teleport, building, roaming changes, and administration
+remain policy-gated and require an authenticated authorized IM, operator action,
+or a narrowly bound scheduler grant as documented in the policy matrix.
+
+## Configuration contract and migration
+
+The JSON root uses `"schema_version": 1`. Omitting it is accepted as legacy
+version 1. Any other value fails before startup with a migration message. There
+is no automatic in-place migration: copy the file, update the copy using the
+release notes/example, validate the copy, then atomically select it. Unknown
+fields fail closed. Installers never overwrite or migrate operator files.
+
+Precedence, lowest to highest, is:
+
+1. bounded built-in defaults;
+2. JSON configuration;
+3. secret files named by JSON;
+4. environment-referenced secret files;
+5. direct environment values.
+
+Mode, endpoints, credentials, authorization UUIDs, TLS, storage, queue/resource
+limits, reconnect policy, behavior, and interaction settings are restart-only.
+Runtime control can pause/resume autonomy, toggle the roaming job, decide an
+existing approval, cancel an active action, expire/delete conversation state,
+inject an operator message, reconnect, or shut down; it does not silently
+rewrite the configuration. A future reloadable field must be explicitly added
+to the versioned control/config contract.
+
+## Secrets and privacy
+
+Threat model: grid residents, object/avatar/inventory metadata, capability
+replies, model output, public chat, IM text, and persisted files are untrusted.
+Remote peers may inject instructions, spoof names, replay approvals/call IDs,
+flood queues, delay or truncate replies, and attempt secret exfiltration. The
+service identity, local config/secret ACLs, authenticated UUID/control role,
+opaque policy authorization, generation fencing, bounded queues, and audit sink
+are trust boundaries. Host/root compromise, a malicious binary/dependency, and
+an operator deliberately approving a harmful action are outside the process
+sandbox and require OS/supply-chain/operational controls.
+
+Never place API keys, grid passwords, or control tokens in command arguments,
+unit files, wrapper XML, logs, crash-report commands, issue reports, or TUI
+screens. Secret wrappers redact `Debug` and `Display`, are not serializable, and
+normal observability stores pseudonymous correlation IDs, result codes, bounds,
+and hashes—not prompt bodies, credentials, visual pixels, or raw tool arguments.
+
+Secret files must be regular, non-symlink, bounded UTF-8 files containing one
+line. On Unix, startup rejects group/other permission bits; use mode `0600` or
+stricter and ownership by the service identity. On Windows, set an NTFS ACL that
+grants only the service identity and Administrators, for example with `icacls`;
+Rust's portable metadata API cannot prove arbitrary Windows ACL semantics, and
+FAT/network filesystems may not enforce them. Treat an unverifiable filesystem
+as unsuitable for unattended secrets.
+
+Rotate one credential at a time: write a new restricted file beside the old
+one, atomically replace or repoint the `_FILE` setting, run `--check-config`,
+then restart. Revoke the old credential only after readiness. Control observer
+and operator tokens must differ from each other and from grid/LLM credentials.
+
+## Linux systemd
+
+The hardened example is
+[`../packaging/metacrate-grid-agent/systemd/metacrate-grid-agent.service`](../packaging/metacrate-grid-agent/systemd/metacrate-grid-agent.service).
+Create the unprivileged `metacrate-agent` identity, copy (do not overwrite) an
+operator configuration to `/etc/metacrate/grid-agent.json`, create
+`/var/lib/metacrate/grid-agent`, and copy the environment-file example. The
+environment file contains only secret *paths*. Put the actual secret files in
+`/etc/metacrate/secrets`, owned by the service identity with mode `0600`.
+
+Install the unit, inspect the sandbox, validate, then start:
+
+```sh
+sudo systemd-analyze security packaging/metacrate-grid-agent/systemd/metacrate-grid-agent.service
+sudo -u metacrate-agent /usr/local/bin/metacrate-grid-agent --config /etc/metacrate/grid-agent.json --check-config
+sudo systemctl daemon-reload
+sudo systemctl enable --now metacrate-grid-agent.service
+```
+
+The unit uses `SIGINT`, matching foreground Ctrl-C and the owned graceful
+shutdown path. Do not replace it with `SIGKILL`. `ProtectSystem=strict` permits
+writes only under the state directory.
+
+## Windows service operation
+
+Run the same `.exe`, JSON, control TCP protocol, and shutdown path. Portable
+foreground operation in PowerShell is the baseline:
+
+```powershell
+& 'C:\Program Files\MetaCrate\metacrate-grid-agent.exe' --config 'C:\ProgramData\MetaCrate\grid-agent.json' --check-config
+& 'C:\Program Files\MetaCrate\metacrate-grid-agent.exe' --config 'C:\ProgramData\MetaCrate\grid-agent.json'
+```
+
+For unattended use, configure a maintained Windows service wrapper (for
+example WinSW) to launch exactly that command as a dedicated low-privilege
+account and to translate SCM Stop into Ctrl-C/console control before its timeout.
+Keep secrets in separate ACL-restricted files and expose only `_FILE` paths in
+the wrapper environment. Configure restart-on-failure, not unconditional rapid
+restart. Validate as the service identity before registration. The wrapper must
+not capture environment values or command output into a world-readable log.
+The supplied PowerShell installer replaces only the executable and never state.
+
+## Control, network, and health
+
+Split control defaults to loopback TCP. A non-loopback bind is rejected unless
+an explicit Rustls certificate/private key is configured. Use host firewalls to
+allow only operator networks, protect token files, and prefer loopback plus an
+authenticated tunnel. Observer tokens are read-only. Protocol framing, role
+permissions, cancellation, event gaps, and limits are in the control-plane doc.
+
+Readiness is stricter than a TCP connection: the runtime view must show
+`transport_connected=true` and `agent_ready=true`. The TUI Overview and
+observer Runtime request are health/readiness checks; process existence alone
+is not readiness. Alert on authentication-blocked, sustained degraded/backoff,
+queue saturation/dropped-event counters, audit failure, orphan builds, and a
+stale generation.
+
+## Retention, backup, upgrade, and rollback
+
+Conversation snapshots, landmark state, and audit journals are non-secret but
+privacy-sensitive. Stop or pause mutation, take a filesystem-consistent backup
+of the configured data directory, and encrypt/restrict the backup. Do not back
+up secrets with ordinary state. Journal rotation is bounded by the configured
+observability policy; ship rotated files to restricted storage before deletion
+when retention policy requires it. Replay is diagnostic and never executes an
+action.
+
+Release installation replaces only the binary. The POSIX and PowerShell helpers
+stage a temporary executable then move it into place; neither touches config,
+secrets, conversations, landmarks, or journals. For upgrade: back up state,
+install the new binary, validate the existing config, stop gracefully, start,
+and verify readiness. For rollback: stop gracefully, restore the prior binary,
+restore state only when the new version changed it incompatibly, validate, and
+start. Never run two service generations against one writable data directory.
+
+## Failure playbooks
+
+- Authentication blocked: pause retries, verify login URL/avatar and rotate the
+  password file; never paste it into logs. Force reconnect after correction.
+- LLM unavailable/rate limited: autonomy remains bounded; verify the exact URL,
+  firewall/DNS, and key. Multimodal rejection falls back to the textual scene
+  summary without resending the large image.
+- Maintenance/disconnect: allow generation fencing and bounded backoff. Stale
+  inference/mutation results are discarded; do not bypass reconnect controls.
+- Emergency: use operator Pause first, Cancel the exact active action when
+  appropriate, then Graceful Shutdown. Ctrl-C/SIGINT follows the same cleanup.
+- Orphan build: keep the reported object IDs, inspect ownership in-world, and
+  manually recover only those IDs. Never bulk-delete by name or proximity.
+- Corrupt persistence: preserve the quarantined file for restricted diagnosis;
+  the service recovers an older valid generation or starts fail-closed. Do not
+  hand-edit a live journal.
+- Full disk/audit backpressure: pause autonomy, free space according to retention
+  policy, and restart only after the audit path is writable. Policy fails closed
+  when required audit records cannot be accepted.
+
+## Resource defaults and unsupported operations
+
+The example records all current queue, message, conversation, tool, behavior,
+reconnect, and interaction defaults. Important defaults include 256 grid events,
+32 control commands, 512 observations, four concurrent inference requests,
+16 tool calls, 512 active senders/sessions, a 1 MiB transport body, and bounded
+10-second shutdown. Vision defaults to a 320x180 synthetic image with bounded
+entities, triangles, texture work, PNG bytes, time, and concurrency.
+
+Unsupported by design: arbitrary raw packets or agent-control flags, arbitrary
+shell/subprocess execution, provider SDK/model discovery, remote plaintext
+control, unauthenticated mutation, unrestricted walking/teleport/touch/follow,
+automatic config migration, persistence of viewport pixels, framebuffer/screen
+capture, and treating untrusted grid/LLM text as instructions or authority.
+
+## Release evidence
+
+Build the intentional graphs and retain the commands/output with the release:
+
+```sh
+cargo build --locked --release -p metacrate-grid-agent
+cargo build --locked --release -p metacrate-grid-agent --features live-grid
+cargo tree --locked -p metacrate-grid-agent --features live-grid -e normal,build
+cargo audit
+```
+
+Record the binary byte size (`stat -c %s` on Linux or `Get-Item ... .Length` on
+PowerShell) and the `cargo tree` inventory. Reject CLR/scripting/provider SDK,
+subprocess adapters, Skia/GPU, or undeclared native libraries on the agent path.
+The consolidated Gitea gate runs only on `ubuntu-latest`; Windows portability is
+proved by the existing cross-target compile/static gate rather than a Windows
+Gitea runner.
+
+The milestone baseline is recorded in
+[grid-agent-release-evidence.md](grid-agent-release-evidence.md).
+
+Related contracts: [architecture](grid-agent-architecture.md),
+[policy](grid-agent-policy.md), [session](grid-agent-session.md),
+[control](grid-agent-control-plane.md), [observability](grid-agent-observability.md),
+[TUI](grid-agent-tui.md), and [conversation storage](grid-agent-conversation.md).
--- a/docs/grid-agent-release-evidence.md
+++ b/docs/grid-agent-release-evidence.md
@@ -0,0 +1,34 @@
+# Grid-agent release evidence
+
+This checked-in measurement is the packaging baseline for milestone 14. It is
+not a downloadable artifact or a substitute for regenerating evidence for each
+release candidate.
+
+- Measured: 2026-08-18
+- Host: `aarch64-unknown-linux-gnu`
+- Rust: `rustc 1.97.1 (8bab26f4f 2026-07-14)`
+- Command: `cargo build --locked --release -p metacrate-grid-agent --features live-grid`
+- Profile: workspace `release` (`opt-level=1`, `debug=0`, incremental disabled,
+  256 codegen units to stay within the packaging memory ceiling)
+- Binary: `target/release/metacrate-grid-agent`
+- Size: 28,658,000 bytes
+- SHA-256: `c8ed694c6817ce9a897897f521e174fc8d90378ddf46565336c747e22d876dfc`
+- Normal/build dependency inventory: 192 unique Cargo package/version/path
+  identities from `cargo tree --locked -p metacrate-grid-agent --features
+  live-grid -e normal,build`
+- Dynamic ELF dependencies: the AArch64 loader, `libc`, `libm`, and `libgcc_s`
+  only. TLS's declared `aws-lc-sys` component is statically linked.
+
+The audited tree contains the MetaCrate agent, the original compatibility
+crates it consumes, the pure-Rust J2K/meshing/rendering path, Tokio, Reqwest,
+Rustls, AWS-LC, terminal/serialization/bounds utilities, and their build-time
+Rust tooling. It contains no CLR/Mono/.NET runtime, Python/Node/Lua scripting
+runtime, provider SDK, Skia/GPU stack, OpenJPEG feature, shell/subprocess
+adapter, or dynamically loaded undeclared application library.
+
+Regenerate for a release candidate with the commands in
+[grid-agent-operations.md](grid-agent-operations.md), retain the full unabridged
+`cargo tree` output as CI/release evidence, compare size intentionally, and run
+the dependency-policy and consolidated required gates. Binary hashes and sizes
+will change when source, compiler, target, profile, or dependency versions
+change.