Complete Bevy OpenSim scene rendering
Some checks failed
CI / rust-skia (Rust only) (push) Has been cancelled
CI / required (push) Has been cancelled

This commit is contained in:
2026-08-22 17:41:39 +02:00
parent f6f5abe464
commit 692894cac9
19 changed files with 6489 additions and 717 deletions

View File

@@ -57,21 +57,31 @@ facility, not a requirement for autonomous operation.
## Images
Vision captures are reconstructed viewport images rather than screenshots.
The live service keeps one headless `wgpu` device, renders the current scene to
an offscreen color/depth target, reads the frame back, encodes it directly as
JPEG, and attaches it as a Mentra image content block. GPU setup and readback
run outside the simulator event path. If no compatible adapter is available or
a frame fails, the same scene is rendered by the deterministic software
rasterizer. The default 320x180 frame, quality, entity, triangle, texture-fetch,
decoded-pixel, byte, rate, and concurrency limits are validated before runtime.
JPEG avoids the much larger PNG payloads.
`metacrate-rendering-wgpu` owns a dedicated-thread, headless Bevy/wgpu device,
renders the current scene to an offscreen color/depth target, reads the frame
back, encodes it directly as JPEG, and attaches it as a Mentra image content
block. GPU setup, scene conversion, and readback run outside the simulator event
path. If no compatible adapter is available or a frame fails, the deterministic
software rasterizer handles the same snapshot. The default 640x360 frame and
the entity, triangle, texture, JPEG, time, and concurrency safety ceilings are
validated before runtime. JPEG avoids the much larger PNG payloads.
The current renderer handles legacy prim geometry, approximate texture colors,
simple avatars, and flat terrain/water. `behavior_camera_set` gives an
authorized model a bounded position, target, and vertical field of view;
`behavior_camera_reset` restores the avatar-facing view. Viewer-grade
mesh/sculpt rendering, full UV materials, lighting, authoritative terrain, and
avatar/attachment fidelity remain tracked in the renderer issue.
The live source reconstructs authoritative terrain heights and detail textures,
legacy prims, sculpt maps, uploaded meshes, linksets, legacy Blinn-Phong and
PBR materials, independent UV transforms, water, and privacy-marked avatar
cards. It keeps bounded decoded-texture and geometry caches per region; Bevy
also retains unchanged GPU textures and meshes between frames. Textures are
decoded through the pure-Rust JPEG-2000 path, reduced to viewport-appropriate
resolution, and uploaded with filtered mip chains. The view radius is
configurable as `vision.max_distance_meters` and defaults to 64 meters. World
objects are selected by that distance; avatar attachments are rigged through
their linkset hierarchy and included only when the camera is focused on the
avatar, after world-object capacity is reserved.
`behavior_camera_set` gives an authorized model a bounded position, target, and
vertical field of view; `behavior_camera_reset` restores the avatar-facing
view. Full avatar body/attachment mesh fidelity and baking remain separately
tracked work.
## Bounds and cancellation