Add headless wgpu vision and camera controls
Some checks failed
CI / rust-skia (Rust only) (push) Has been cancelled
CI / required (push) Has been cancelled

This commit is contained in:
2026-08-22 11:36:34 +02:00
parent 0dd2ca5824
commit c101007362
19 changed files with 1250 additions and 56 deletions

View File

@@ -226,8 +226,12 @@ The example records all current queue, message, conversation, tool, behavior,
reconnect, and interaction defaults. Important defaults include 256 grid events,
32 control commands, 512 observations, four concurrent inference requests,
16 tool calls, 512 active senders/sessions, a two-minute model window, and bounded
10-second shutdown. Vision defaults to a 320x180 software-rendered JPEG with bounded
entities, triangles, texture work, JPEG bytes, time, and concurrency.
10-second shutdown. Vision defaults to a 320x180 headless-wgpu JPEG with a
deterministic software fallback and bounded entities, triangles, texture work,
JPEG bytes, time, and concurrency. Authorized `behavior_camera_set` calls are
limited to 96 meters from the avatar by default, require a distinct target, and
accept a 20-120 degree vertical field of view; `behavior_camera_reset` restores
the avatar-facing 60-degree view.
Unsupported by design: arbitrary raw packets or agent-control flags, arbitrary
shell/subprocess execution, provider/model discovery outside Mentra, remote plaintext
@@ -248,7 +252,8 @@ cargo audit
Record the binary byte size (`stat -c %s` on Linux or `Get-Item ... .Length` on
PowerShell) and the `cargo tree` inventory. Reject CLR/scripting/provider SDK,
subprocess adapters, Skia/GPU, or undeclared native libraries on the agent path.
subprocess adapters, unreviewed graphics stacks, or undeclared native libraries
on the agent path.
The consolidated Gitea gate runs only on `ubuntu-latest`; Windows portability is
proved by the existing cross-target compile/static gate rather than a Windows
Gitea runner.