Add headless wgpu vision and camera controls
This commit is contained in:
@@ -56,16 +56,22 @@ facility, not a requirement for autonomous operation.
|
||||
|
||||
## Images
|
||||
|
||||
Vision captures are real software-rendered viewport images. MetaCrate renders
|
||||
the current scene, encodes the final frame directly as JPEG, and attaches it as
|
||||
a Mentra image content block. The default 320x180 frame, quality, entity,
|
||||
triangle, texture-fetch, decoded-pixel, byte, rate, and concurrency limits are
|
||||
validated before runtime. JPEG avoids the much larger PNG payloads.
|
||||
Vision captures are reconstructed viewport images rather than screenshots.
|
||||
The live service keeps one headless `wgpu` device, renders the current scene to
|
||||
an offscreen color/depth target, reads the frame back, encodes it directly as
|
||||
JPEG, and attaches it as a Mentra image content block. GPU setup and readback
|
||||
run outside the simulator event path. If no compatible adapter is available or
|
||||
a frame fails, the same scene is rendered by the deterministic software
|
||||
rasterizer. The default 320x180 frame, quality, entity, triangle, texture-fetch,
|
||||
decoded-pixel, byte, rate, and concurrency limits are validated before runtime.
|
||||
JPEG avoids the much larger PNG payloads.
|
||||
|
||||
The current renderer handles legacy prim geometry, approximate texture colors,
|
||||
simple avatars, and flat terrain/water. Viewer-grade mesh/sculpt rendering,
|
||||
full UV materials, lighting, authoritative terrain, and model-controlled camera
|
||||
tools are tracked separately.
|
||||
simple avatars, and flat terrain/water. `behavior_camera_set` gives an
|
||||
authorized model a bounded position, target, and vertical field of view;
|
||||
`behavior_camera_reset` restores the avatar-facing view. Viewer-grade
|
||||
mesh/sculpt rendering, full UV materials, lighting, authoritative terrain, and
|
||||
avatar/attachment fidelity remain tracked in the renderer issue.
|
||||
|
||||
## Bounds and cancellation
|
||||
|
||||
|
||||
Reference in New Issue
Block a user