Complete Bevy OpenSim scene rendering
This commit is contained in:
@@ -57,21 +57,31 @@ facility, not a requirement for autonomous operation.
|
||||
## Images
|
||||
|
||||
Vision captures are reconstructed viewport images rather than screenshots.
|
||||
The live service keeps one headless `wgpu` device, renders the current scene to
|
||||
an offscreen color/depth target, reads the frame back, encodes it directly as
|
||||
JPEG, and attaches it as a Mentra image content block. GPU setup and readback
|
||||
run outside the simulator event path. If no compatible adapter is available or
|
||||
a frame fails, the same scene is rendered by the deterministic software
|
||||
rasterizer. The default 320x180 frame, quality, entity, triangle, texture-fetch,
|
||||
decoded-pixel, byte, rate, and concurrency limits are validated before runtime.
|
||||
JPEG avoids the much larger PNG payloads.
|
||||
`metacrate-rendering-wgpu` owns a dedicated-thread, headless Bevy/wgpu device,
|
||||
renders the current scene to an offscreen color/depth target, reads the frame
|
||||
back, encodes it directly as JPEG, and attaches it as a Mentra image content
|
||||
block. GPU setup, scene conversion, and readback run outside the simulator event
|
||||
path. If no compatible adapter is available or a frame fails, the deterministic
|
||||
software rasterizer handles the same snapshot. The default 640x360 frame and
|
||||
the entity, triangle, texture, JPEG, time, and concurrency safety ceilings are
|
||||
validated before runtime. JPEG avoids the much larger PNG payloads.
|
||||
|
||||
The current renderer handles legacy prim geometry, approximate texture colors,
|
||||
simple avatars, and flat terrain/water. `behavior_camera_set` gives an
|
||||
authorized model a bounded position, target, and vertical field of view;
|
||||
`behavior_camera_reset` restores the avatar-facing view. Viewer-grade
|
||||
mesh/sculpt rendering, full UV materials, lighting, authoritative terrain, and
|
||||
avatar/attachment fidelity remain tracked in the renderer issue.
|
||||
The live source reconstructs authoritative terrain heights and detail textures,
|
||||
legacy prims, sculpt maps, uploaded meshes, linksets, legacy Blinn-Phong and
|
||||
PBR materials, independent UV transforms, water, and privacy-marked avatar
|
||||
cards. It keeps bounded decoded-texture and geometry caches per region; Bevy
|
||||
also retains unchanged GPU textures and meshes between frames. Textures are
|
||||
decoded through the pure-Rust JPEG-2000 path, reduced to viewport-appropriate
|
||||
resolution, and uploaded with filtered mip chains. The view radius is
|
||||
configurable as `vision.max_distance_meters` and defaults to 64 meters. World
|
||||
objects are selected by that distance; avatar attachments are rigged through
|
||||
their linkset hierarchy and included only when the camera is focused on the
|
||||
avatar, after world-object capacity is reserved.
|
||||
|
||||
`behavior_camera_set` gives an authorized model a bounded position, target, and
|
||||
vertical field of view; `behavior_camera_reset` restores the avatar-facing
|
||||
view. Full avatar body/attachment mesh fidelity and baking remain separately
|
||||
tracked work.
|
||||
|
||||
## Bounds and cancellation
|
||||
|
||||
|
||||
@@ -226,9 +226,10 @@ The example records all current queue, message, conversation, tool, behavior,
|
||||
reconnect, and interaction defaults. Important defaults include 256 grid events,
|
||||
32 control commands, 512 observations, four concurrent inference requests,
|
||||
16 tool calls, 512 active senders/sessions, a two-minute model window, and bounded
|
||||
10-second shutdown. Vision defaults to a 320x180 headless-wgpu JPEG with a
|
||||
deterministic software fallback and bounded entities, triangles, texture work,
|
||||
JPEG bytes, time, and concurrency. Authorized `behavior_camera_set` calls are
|
||||
10-second shutdown. Vision defaults to a 640x360 headless Bevy/wgpu JPEG, a
|
||||
64-meter configurable view radius, a persistent 512 MiB decoded-texture cache,
|
||||
and a deterministic software fallback with bounded entities, triangles, asset
|
||||
work, JPEG bytes, time, and concurrency. Authorized `behavior_camera_set` calls are
|
||||
limited to 96 meters from the avatar by default, require a distinct target, and
|
||||
accept a 20-120 degree vertical field of view; `behavior_camera_reset` restores
|
||||
the avatar-facing 60-degree view.
|
||||
|
||||
Reference in New Issue
Block a user