MetaCrate/README.md

MetaCrate

MetaCrate is a test-driven, clean native Rust reimplementation of LibreMetaverse. LibreMetaverse is a behavioral and API reference only: the finished project must not load, host, bind to, invoke, or ship the .NET implementation. The current stage is a compiling structural shell: crate boundaries mirror the .NET library projects, public C# type names have Rust declarations, every upstream NUnit invocation has a traceable catalog entry, and every sample/tool project has a Rust binary target. Types, StructuredData, Imaging, PrimMesher, both rendering adapters, and the main assembly's packet/message/asset/primitive wire-data, core runtime/networking, avatar-facing manager, world/social/service manager, RLV, LSL tools, Utilities, Vivox, and WebRTC slices now have complete callable, failure-only signatures. The independent downstream fixture compiles every cataloged type and member with zero exclusions, and all 1,289 NUnit invocations now have reviewed Rust parity cases. Production behavior implementation is the next stage.

A case counts as translated only when an explicit Rust test body calls the mapped API and retains the upstream identity and body hash. The parity ledger now contains zero pending or unreviewed cases.

The source snapshot, compatibility rules, dependency research, and ordered implementation plan are in RUSTREWRITE.md.

The authoritative compiled .NET surface is checked in as api/public-api.json. Every type and member has a reviewed Rust destination in api/RUST-TYPES.tsv and api/RUST-MAPPING.tsv; api/README.md documents deterministic regeneration and validation. All API crates expose the same typed Error for fallible unimplemented members; infallible shims use libremetaverse_types::unimplemented_api! so placeholders cannot look like successful behavior.

cargo build --workspace
cargo test --workspace --no-run
python3 tools/generate_rust_mapping.py --check
python3 tools/generate_api_shims.py --check
python3 tools/check_test_parity.py
python3 tools/audit_red_suite.py

tests/upstream-tests.json is the machine-readable NUnit parity catalog. Translated tests live in hand-written Rust files with the parity-case marker documented in tests/PARITY.md; python3 tools/generate_surface.py --check verifies the catalog against the pinned adjacent LibreMetaverse checkout without overwriting those files.

Running cargo test --workspace is intentionally red during the shim stage.

Native implementation progress

Milestone 04 is complete. libremetaverse-types now provides native UUID, incremental CRC-32, little-endian floating-point byte helpers, Vector2/Vector3/Vector3d/Vector4 arithmetic and serialization, Color4 color conversion, Ray storage, Quaternion rotation/interpolation/serialization, and Matrix4 construction, transforms, inversion, and decomposition. UUID protocol bytes use explicit network order while compat::Guid conversions preserve the mixed-endian .NET byte-array layout. Vector, color, and normalized quaternion byte encodings are explicitly little-endian and do not depend on host architecture. Matrix4 preserves LibreMetaverse's row-major storage, row-vector transform direction, and translation in the fourth row; quaternion components are stored as (x, y, z, w) and products use the reference Hamilton-product direction. All Types enum discriminants retain their C# widths, extensible protocol flags preserve unknown bits, and the native conversion layer covers numeric and floating-point byte order, UTF-8 and hex text, hashes and PBKDF2, timestamps, IP addresses, region handles, enum metadata, and cross-platform OS detection. The Types collections, including duplicate-preserving multi-value and synchronized double-key maps, bounded LRU/MRU caches, deterministic absolute and sliding expiration, and hierarchical token-bucket throttling are native Rust as well. Cache and bucket tests use injected monotonic clocks, and concurrency invariants run without sleep-based timing. All 45 Types and all 942 mapped members now resolve to native implementations; no generated Types shims remain. Milestone 05 now has a native, format-neutral OSD value model with structural equality and hashing, permissive LibreMetaverse conversion defaults, explicit integer/date byte order, synchronized snapshot-based arrays and maps, and bounded parser dispatch. OSD map order is intentionally unspecified at the public model boundary; encoders must choose and document any stable ordering they require. Untrusted dispatch limits input bytes, nesting depth, decoded nodes, and aggregate binary allocation before returning a value. The native Binary LLSD codec covers every reference marker, both accepted headers, exact numeric and date byte order, seekable stream overloads, and position-bearing errors for malformed or truncated input. Its parser and encoder enforce the same byte, depth, node, and allocation bounds; map keys are encoded in sorted order to make output stable despite the model's intentionally unordered maps. The bounded Notation LLSD codec preserves UTF-16 length-prefix semantics, the reference escape rules and accepted scalar spellings, base64, base16 and length-prefixed binary forms, and both compact and formatted output. Malformed notation reports UTF-16 offsets with parser context, while input, output, nesting, node, and aggregate allocation limits apply to untrusted text. The native XML LLSD codec covers the wrapper and inner-element APIs, all scalar and container elements, namespace-local names, XML declarations, comments, CDATA, entity decoding, and the reference's nonstandard Linden processing instruction handling. Serialization is compact and deterministic for maps. DTD declarations and named entity expansion are disabled; input, output, nesting, node, and aggregate allocation limits are enforced with byte-positioned parse errors. JSON conversion is explicit rather than a derived Serde mapping: integral width coercion, empty strings, binary arrays, typed values, and the reference's default omission rules retain their C# behavior. Its reader rejects duplicate properties and enforces byte, depth, node, and allocation limits while compact output uses System.Text.Json-compatible escaping and stable map-key ordering. The private OSD Protobuf schema is implemented directly with fixed field tags, ZigZag int32, little-endian IEEE-754 fixed64 values, 16-byte UUIDs, bounded length-delimited containers, and wire-type-aware unknown-field skipping. The schema remains an internal compatibility format; native encoders emit stable sorted maps and accept the exact optional LLSD Protobuf header. Cross-format conversion follows the reference's format-specific boundaries:

Format	Round-trip behavior
Binary LLSD	All regular OSD variants are lossless; raw OSDLlsdXml is unsupported.
Notation LLSD	All regular OSD variants are lossless; raw OSDLlsdXml is unsupported.
XML LLSD	Regular variants are lossless; raw OSDLlsdXml is injected as an element and therefore decodes as that element's ordinary OSD value.
JSON OSD	Booleans, integers, reals, nonempty strings, maps, and arrays retain structure; UUID, date, and URI become strings, binary becomes an integer array, and undefined, empty strings, and raw XML become JSON null.
Protobuf OSD	Regular variants are lossless within the reference schema's date range and whole-second precision; raw OSDLlsdXml maps to undefined.

Checked-in malformed corpora and bounded deterministic mutations exercise all five parsers without an external fuzzing runtime. They cover truncation, tags, lengths, payloads, delimiters, nesting, entity rejection, contextual errors, and the parser resource limits. Every encoder sorts map keys where necessary, so serialization is deterministic across platforms. Milestone 06 starts with a native ManagedImage: top-left-origin, byte-per-sample planar storage has an explicit width-byte row stride, with gray stored in the red plane and optional color, alpha, and bump planes. Checked constructors and resizers cap decoded images at 4096 by 4096 worth of pixels, codec adapters cap buffered encoded input at 64 MiB, and canonical interleaving accepts explicit row strides without exposing codec-library types through the core abstraction. The native TGA/Pfim-compatible path decodes uncompressed and RLE true-color, grayscale, and indexed TGA data with all descriptor orientations, plus bounded legacy and DX10 DDS packed luminance/RGB(A) layouts and BC1 through BC7 blocks. TGA output preserves the reference header, BGR(A) order, descriptor choice, alpha-only expansion, and deterministic trailing padding. File and stream inputs are capped before decoding, and all dimensions, packed rows, palettes, packets, masks, and block indices use checked arithmetic with positioned parse errors. Legacy and DX10 BC1 through BC5 decoding is built in. The default dds-bc67 feature adds BC6H/BC7 decoding through the safe, pure-Rust bcdec_rs crate; disabling default features removes that optional dependency and makes those two formats return a typed unsupported-format error. The controlled audit aggregates every expected failure by standardized C# member ID and rejects unrelated fixture, assertion, compile, or symbol errors.