[Grid agent] Add embodied attention, movement, and believable presence behavior #125

New Issue

hugo · 2026-08-17T19:31:35Z

hugo commented

2026-08-17 19:31:35 +00:00

Objective

Add an embodied behavior controller that makes the agent appear attentive and plausible in-world without granting the model unrestricted low-level movement or creating nuisance behavior.

Behavior model

Implement explicit modes such as offline, settling, available, engaged, executing, roaming, paused, and recovering. Mode changes are driven by grid readiness, conversations, authorized tasks, operator control, and scheduler events.
When responding to a nearby public speaker, resolve their current avatar position and smoothly turn the existing movement/camera toward them. Use a timeout and tolerate disappearance, invalid positions, or region changes.
Expose high-level bounded actions such as face avatar/point, look around, walk a short distance, stop, sit/stand, and return current pose. Translate them through existing AgentMovement; never expose arbitrary control flags or unbounded autopilot directly to the model.
Add human-readable pacing: configurable response delay ranges, short attention dwell, idle look/pose changes, typing/activity indication only where supported, and suppression while busy. Randomness must be seeded/injectable for tests.
Enforce parcel/region boundaries, distance and duration caps, collision/stuck timeout, cancellation, rate limits, and a global emergency stop. Do not follow residents, wander, fly, teleport, animate, or touch objects unless that behavior is explicitly enabled and policy-authorized.
Operator pause or a higher-priority authorized task preempts idle behavior. Reconnect and region crossing reset stale targets.

Acceptance criteria

Fake-grid packet tests verify turn/camera calculations, bounded motion, preemption, stop-on-timeout, region-change fencing, and zero movement while paused/offline.
Paused-time seeded tests make pacing and idle behavior deterministic.
The agent does not chatter or move continuously merely to appear alive; default idle behavior is low-frequency and safe.
Every behavior transition and action has an observable trigger, policy result, duration, and outcome.

Dependencies

Depends on lifecycle, messaging attention events, world perception, policy, and existing native movement APIs. Scheduled landmark roaming remains separate.

## Objective Add an embodied behavior controller that makes the agent appear attentive and plausible in-world without granting the model unrestricted low-level movement or creating nuisance behavior. ## Behavior model - Implement explicit modes such as offline, settling, available, engaged, executing, roaming, paused, and recovering. Mode changes are driven by grid readiness, conversations, authorized tasks, operator control, and scheduler events. - When responding to a nearby public speaker, resolve their current avatar position and smoothly turn the existing movement/camera toward them. Use a timeout and tolerate disappearance, invalid positions, or region changes. - Expose high-level bounded actions such as face avatar/point, look around, walk a short distance, stop, sit/stand, and return current pose. Translate them through existing `AgentMovement`; never expose arbitrary control flags or unbounded autopilot directly to the model. - Add human-readable pacing: configurable response delay ranges, short attention dwell, idle look/pose changes, typing/activity indication only where supported, and suppression while busy. Randomness must be seeded/injectable for tests. - Enforce parcel/region boundaries, distance and duration caps, collision/stuck timeout, cancellation, rate limits, and a global emergency stop. Do not follow residents, wander, fly, teleport, animate, or touch objects unless that behavior is explicitly enabled and policy-authorized. - Operator pause or a higher-priority authorized task preempts idle behavior. Reconnect and region crossing reset stale targets. ## Acceptance criteria - [ ] Fake-grid packet tests verify turn/camera calculations, bounded motion, preemption, stop-on-timeout, region-change fencing, and zero movement while paused/offline. - [ ] Paused-time seeded tests make pacing and idle behavior deterministic. - [ ] The agent does not chatter or move continuously merely to appear alive; default idle behavior is low-frequency and safe. - [ ] Every behavior transition and action has an observable trigger, policy result, duration, and outcome. ## Dependencies Depends on lifecycle, messaging attention events, world perception, policy, and existing native movement APIs. Scheduled landmark roaming remains separate.

hugo added this to the 14 - metacrate grid agent milestone 2026-08-17 19:31:35 +00:00

hugo added the enhancement label 2026-08-17 19:31:35 +00:00

hugo referenced this issue

2026-08-17 19:32:14 +00:00

implement an AI driven agent that runs in opensim #117

hugo referenced this issue from a commit

2026-08-18 00:26:32 +00:00

feat(grid-agent): add embodied behavior controller (#125)

hugo commented

2026-08-18 00:26:51 +00:00

Implemented in commit 058ed10.

What changed:

Added the generation-fenced embodied controller with offline, settling, available, engaged, executing, roaming, paused, and recovering modes.
Added bounded policy tools for face-avatar/point, one look shift, short walk, stop, sit, stand, and current pose; mutation tools are limited to authorized IM, local operator, and scheduler origins.
Public responses now use configurable seeded pacing before delivery, resolve the current nearby avatar, and turn camera/body in bounded 30-degree steps with attention dwell.
Short walks are distance/duration limited, verify the cached target parcel ID and region bounds, poll progress, detect stuck state, and always stop on arrival, timeout, cancellation, pause, emergency stop, disconnect, or region crossing.
Reused the shared native AgentMovement/AgentManager graph (turn_toward, local autopilot plus mandatory cancel, sit/stand) without exposing control flags or follow/wander/fly/teleport/animation/touch primitives.
Added lossless bounded transition/action observations with trigger, authorization class/ID, generation/region, duration, and outcome, plus validated configuration and operator documentation.

Verification:

cargo test -p metacrate-grid-agent --lib: 80 passed (including 9 deterministic fake-grid embodiment tests and public pre-delivery pacing).
dependency_policy: 5 passed; policy_gateway: 3 passed; session_supervisor: 1 passed.
cargo fmt --check, strict clippy -D warnings, rustdoc -D warnings, git diff --check, and cargo deny check all passed.
The live-grid adapter and binary passed strict feature-gated clippy against the API-faithful lightweight native fixture.

Environmental checks: the direct full generated libremetaverse build reached that crate but rustc was SIGKILLed at the 8 GiB container ceiling before compiling the agent; the Windows cross-check was blocked in aws-lc-sys because x86_64-w64-mingw32-gcc is not installed. The new runtime code contains no platform-specific API. No credentials or .env contents were read or committed.

Implemented in commit 058ed10. What changed: - Added the generation-fenced embodied controller with offline, settling, available, engaged, executing, roaming, paused, and recovering modes. - Added bounded policy tools for face-avatar/point, one look shift, short walk, stop, sit, stand, and current pose; mutation tools are limited to authorized IM, local operator, and scheduler origins. - Public responses now use configurable seeded pacing before delivery, resolve the current nearby avatar, and turn camera/body in bounded 30-degree steps with attention dwell. - Short walks are distance/duration limited, verify the cached target parcel ID and region bounds, poll progress, detect stuck state, and always stop on arrival, timeout, cancellation, pause, emergency stop, disconnect, or region crossing. - Reused the shared native AgentMovement/AgentManager graph (turn_toward, local autopilot plus mandatory cancel, sit/stand) without exposing control flags or follow/wander/fly/teleport/animation/touch primitives. - Added lossless bounded transition/action observations with trigger, authorization class/ID, generation/region, duration, and outcome, plus validated configuration and operator documentation. Verification: - cargo test -p metacrate-grid-agent --lib: 80 passed (including 9 deterministic fake-grid embodiment tests and public pre-delivery pacing). - dependency_policy: 5 passed; policy_gateway: 3 passed; session_supervisor: 1 passed. - cargo fmt --check, strict clippy -D warnings, rustdoc -D warnings, git diff --check, and cargo deny check all passed. - The live-grid adapter and binary passed strict feature-gated clippy against the API-faithful lightweight native fixture. Environmental checks: the direct full generated libremetaverse build reached that crate but rustc was SIGKILLed at the 8 GiB container ceiling before compiling the agent; the Windows cross-check was blocked in aws-lc-sys because x86_64-w64-mingw32-gcc is not installed. The new runtime code contains no platform-specific API. No credentials or .env contents were read or committed.

hugo closed this issue

2026-08-18 00:26:58 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/MetaCrate#125