MetaCrate/README.md

# MetaCrate

MetaCrate is a test-driven, clean native Rust reimplementation of
[LibreMetaverse](https://github.com/cinderblocks/libremetaverse). LibreMetaverse
is a behavioral and API reference only: the finished project must not load,
host, bind to, invoke, or ship the .NET implementation. The current
stage is a compiling structural shell: crate boundaries mirror the .NET library
projects, public C# type names have Rust declarations, every upstream NUnit
invocation has a traceable catalog entry, and every sample/tool project has a
Rust binary target. Types, StructuredData, Imaging, PrimMesher, both rendering adapters, and
the main assembly's packet/message/asset/primitive wire-data, core
runtime/networking, avatar-facing manager, world/social/service manager, RLV,
LSL tools, Utilities, Vivox, and WebRTC slices now have complete callable,
failure-only signatures. The independent downstream fixture compiles every
cataloged type and member with zero exclusions, and all 1,289 NUnit invocations
now have reviewed Rust parity cases. Production behavior implementation is the
next stage.

A case counts as translated only when an explicit Rust test body calls the
mapped API and retains the upstream identity and body hash. The parity ledger
now contains zero pending or unreviewed cases.

The source snapshot, compatibility rules, dependency research, and ordered
implementation plan are in [RUSTREWRITE.md](RUSTREWRITE.md).

The authoritative compiled .NET surface is checked in as
[`api/public-api.json`](api/public-api.json). Every type and member has a
reviewed Rust destination in [`api/RUST-TYPES.tsv`](api/RUST-TYPES.tsv) and
[`api/RUST-MAPPING.tsv`](api/RUST-MAPPING.tsv); [`api/README.md`](api/README.md)
documents deterministic regeneration and validation.
All API crates expose the same typed `Error` for fallible unimplemented members;
infallible shims use `libremetaverse_types::unimplemented_api!` so placeholders
cannot look like successful behavior.

```sh
cargo build --workspace
cargo test --workspace --no-run
python3 tools/generate_rust_mapping.py --check
python3 tools/generate_api_shims.py --check
python3 tools/check_test_parity.py
python3 tools/audit_red_suite.py
```

`tests/upstream-tests.json` is the machine-readable NUnit parity catalog.
Translated tests live in hand-written Rust files with the `parity-case` marker
documented in `tests/PARITY.md`; `python3 tools/generate_surface.py --check`
verifies the catalog against the pinned adjacent LibreMetaverse checkout without
overwriting those files.

Running `cargo test --workspace` is intentionally red during the shim stage.

## Native implementation progress

Milestone 04 is complete. `libremetaverse-types` now provides
native UUID, incremental CRC-32, little-endian floating-point byte helpers,
Vector2/Vector3/Vector3d/Vector4 arithmetic and serialization, Color4 color
conversion, Ray storage, Quaternion rotation/interpolation/serialization, and
Matrix4 construction, transforms, inversion, and decomposition. UUID protocol
bytes use explicit network order while `compat::Guid` conversions preserve the
mixed-endian .NET byte-array layout. Vector, color, and normalized quaternion
byte encodings are explicitly little-endian and do not depend on host
architecture. Matrix4 preserves LibreMetaverse's row-major storage, row-vector
transform direction, and translation in the fourth row; quaternion components
are stored as `(x, y, z, w)` and products use the reference Hamilton-product
direction. All Types enum discriminants retain their C# widths, extensible
protocol flags preserve unknown bits, and the native conversion layer covers
numeric and floating-point byte order, UTF-8
and hex text, hashes and PBKDF2, timestamps, IP addresses, region handles, enum
metadata, and cross-platform OS detection. The Types collections,
including duplicate-preserving multi-value and synchronized double-key maps,
bounded LRU/MRU caches, deterministic absolute and sliding expiration, and
hierarchical token-bucket throttling are native Rust as well. Cache and bucket
tests use injected monotonic clocks, and concurrency invariants run without
sleep-based timing. All 45 Types and all 942 mapped members now resolve to
native implementations; no generated Types shims remain.
Milestone 05 now has a native, format-neutral OSD value model with structural
equality and hashing, permissive `LibreMetaverse` conversion defaults, explicit
integer/date byte order, synchronized snapshot-based arrays and maps, and
bounded parser dispatch. OSD map order is intentionally unspecified at the
public model boundary; encoders must choose and document any stable ordering
they require. Untrusted dispatch limits input bytes, nesting depth, decoded
nodes, and aggregate binary allocation before returning a value. The native
Binary LLSD codec covers every reference marker, both accepted headers, exact
numeric and date byte order, seekable stream overloads, and position-bearing
errors for malformed or truncated input. Its parser and encoder enforce the
same byte, depth, node, and allocation bounds; map keys are encoded in sorted
order to make output stable despite the model's intentionally unordered maps.
The bounded Notation LLSD codec preserves UTF-16 length-prefix semantics, the
reference escape rules and accepted scalar spellings, base64, base16 and
length-prefixed binary forms, and both compact and formatted output. Malformed
notation reports UTF-16 offsets with parser context, while input, output,
nesting, node, and aggregate allocation limits apply to untrusted text.
The native XML LLSD codec covers the wrapper and inner-element APIs, all scalar
and container elements, namespace-local names, XML declarations, comments,
CDATA, entity decoding, and the reference's nonstandard Linden processing
instruction handling. Serialization is compact and deterministic for maps.
DTD declarations and named entity expansion are disabled; input, output,
nesting, node, and aggregate allocation limits are enforced with byte-positioned
parse errors.
JSON conversion is explicit rather than a derived Serde mapping: integral
width coercion, empty strings, binary arrays, typed values, and the reference's
default omission rules retain their C# behavior. Its reader rejects duplicate
properties and enforces byte, depth, node, and allocation limits while compact
output uses System.Text.Json-compatible escaping and stable map-key ordering.
The private OSD Protobuf schema is implemented directly with fixed field tags,
ZigZag `int32`, little-endian IEEE-754 fixed64 values, 16-byte UUIDs, bounded
length-delimited containers, and wire-type-aware unknown-field skipping. The
schema remains an internal compatibility format; native encoders emit stable
sorted maps and accept the exact optional LLSD Protobuf header.
Cross-format conversion follows the reference's format-specific boundaries:

| Format | Round-trip behavior |
| --- | --- |
| Binary LLSD | All regular OSD variants are lossless; raw `OSDLlsdXml` is unsupported. |
| Notation LLSD | All regular OSD variants are lossless; raw `OSDLlsdXml` is unsupported. |
| XML LLSD | Regular variants are lossless; raw `OSDLlsdXml` is injected as an element and therefore decodes as that element's ordinary OSD value. |
| JSON OSD | Booleans, integers, reals, nonempty strings, maps, and arrays retain structure; UUID, date, and URI become strings, binary becomes an integer array, and undefined, empty strings, and raw XML become JSON `null`. |
| Protobuf OSD | Regular variants are lossless within the reference schema's date range and whole-second precision; raw `OSDLlsdXml` maps to undefined. |

Checked-in malformed corpora and bounded deterministic mutations exercise all
five parsers without an external fuzzing runtime. They cover truncation, tags,
lengths, payloads, delimiters, nesting, entity rejection, contextual errors,
and the parser resource limits. Every encoder sorts map keys where necessary,
so serialization is deterministic across platforms.
Milestone 06 starts with a native `ManagedImage`: top-left-origin,
byte-per-sample planar storage has an explicit width-byte row stride, with gray
stored in the red plane and optional color, alpha, and bump planes. Checked
constructors and resizers cap decoded images at 4096 by 4096 worth of pixels,
codec adapters cap buffered encoded input at 64 MiB, and canonical
interleaving accepts explicit row strides without exposing codec-library types
through the core abstraction.
The native TGA/Pfim-compatible path decodes uncompressed and RLE true-color,
grayscale, and indexed TGA data with all descriptor orientations, plus bounded
legacy and DX10 DDS packed luminance/RGB(A) layouts and BC1 through BC7 blocks.
TGA output preserves the reference header, BGR(A) order, descriptor choice,
alpha-only expansion, and deterministic trailing padding. File and stream
inputs are capped before decoding, and all dimensions, packed rows, palettes,
packets, masks, and block indices use checked arithmetic with positioned parse
errors.
Legacy and DX10 BC1 through BC5 decoding is built in. The default `dds-bc67`
feature adds BC6H/BC7 decoding through the safe, pure-Rust `bcdec_rs` crate;
disabling default features removes that optional dependency and makes those
two formats return a typed unsupported-format error.
The controlled audit aggregates every expected failure by standardized C#
member ID and rejects unrelated fixture, assertion, compile, or symbol errors.