51 KiB
LibreMetaverse to Rust rewrite plan
1. Scope and source of truth
MetaCrate is a new native Rust implementation of the public API and observable behavior of LibreMetaverse. LibreMetaverse is reference material, not a runtime dependency or implementation component. The source snapshot used to create this shell is:
- repository: https://github.com/cinderblocks/libremetaverse
- branch:
master - commit:
2aa70bb68513b39795da5d13c88f31b86e85a3ba - commit date: 2026-08-07
- local checkout state during inventory: clean
- license base:
libremetaverse/LICENSE.txt, BSD 3-Clause
Generated inventories are intentionally checked in:
api/TYPES.tsvmaps every discovered public C# type declaration to a Rust crate/module/type and source line.api/SURFACE.tsvrecords every public declaration line in the 13 library projects, including a SHA-256 digest.api/public-api.jsonis the authoritative compiled-metadata catalog for all 13 library assemblies after all six source generators run. It also records every external type referenced by a public signature.api/RUST-TYPES.tsvresolves all 3,066 public and 142 external signature types, plus two support traits referenced by public inheritance metadata.api/RUST-MAPPING.tsvrecords the reviewed Rust destination and complete signature decision for all 30,789 public members.tests/upstream-tests.jsonrecords every NUnit invocation and the SHA-256 of its C# method body.tests/PARITY.mdmaps every NUnit invocation to its Rust test name.programs/upstream-programs.jsonrecords every source file and digest for the nine sample/tool projects.
Run python3 tools/generate_surface.py after deliberately changing the pinned
upstream snapshot, then run python3 tools/extract_public_api.py to rebuild the
authoritative metadata catalog twice and require byte-identical output. The
source generator rejects a test count other than 1,289 so upstream drift cannot
silently remove tests. Update both commit constants and review all ledger diffs
in the same change. The earlier 1,295 inventory included five commented-out
[Test] attributes and double-counted the parameterless [Test] marker on one
parameterized [TestCase] method; those are not NUnit invocations.
Current shell status
The current workspace is a structural baseline, not a working client:
- all 13 public .NET library projects have corresponding Rust crates;
- 1,705 source-discovered public C# type declarations produce compiling Rust type/trait shims;
- 12,195 public declaration lines are retained in the API ledger;
- compiled metadata records 3,066 public types and 30,789 public members with matching independent ECMA-335 counts, with all 142 external signature types resolved to cross-platform Rust or project-owned boundary types;
- the completed Types, StructuredData, Imaging, PrimMesher, rendering, main wire/data, core runtime/networking, avatar-facing manager, and world/social/service manager, RLV, LSL tools, Utilities, Vivox, and WebRTC slices expose all 3,066 final Rust-facing types and all 30,789 callable members with real fields/constants/enum values and standardized failing bodies;
- all 1,289 real NUnit
[Test]/[TestCase]invocations have stable catalog entries; all 1,266 non-live, non-benchmark cases are translated, all 19 live-grid cases and four benchmarks are reviewed, and none remain pending; - all nine sample/tool projects have compiling Rust binary targets;
- the 128 TestClient command source files are retained as a command inventory.
No generated type shells remain pending: all cataloged members are callable,
and the standalone downstream fixture names or invokes all 3,066 types and
30,789 members with zero exclusions. api/SURFACE.tsv is still an inventory
aid, not proof of API coverage: it records declaration lines but does not fully
parse multiline signatures. Likewise, a pending catalog entry is not a
semantic translation. The full public API signature shim
gate is complete; semantic test translation is the next stage.
The avatar-facing slice preserves the C# agent, movement/camera, appearance and
outfit-policy, inventory/AIS/store, asset/cache, avatar, and animesh concepts.
Long-running mapped calls retain an explicit cross-platform cancellation token
and return the shared Error::NotImplemented(NotImplemented { csharp_member })
error; events return a typed subscription guard so callback lifetime is visible
to Rust callers.
The world-facing slice preserves the C# object, parcel, terrain, grid, environment, estate, directory, friend, group, sound, interest-list, marketplace, and reporting concepts. Server-facing async operations retain explicit cancellation and shared typed failures; request/reply events expose typed payloads and subscription guards, while catalog constants and flag values remain real.
The extension slice preserves the C# RLV restrictions, callbacks and services; LSL lexer/parser/generator and diagnostic types; Utilities helpers; Vivox protocol/session/manager types; and WebRTC signaling/device/session concepts. Blocking compatibility surfaces stay explicit, stored callbacks and events use typed ownership, and parser/voice bodies fail before performing parsing, networking, device access, or proprietary/native work.
No functional porting starts until the semantic test-suite gate is complete across the entire workspace, rather than one crate at a time:
- every public C# type and member has a reviewed, callable Rust signature with an intentionally failing body;
- every C# test is translated semantically and compiles while calling that Rust API;
- only then is behavior implemented, one crate at a time, until those already translated tests turn green.
Full-rewrite constraint
The final project must be entirely implemented in new Rust code. The following are prohibited in production libraries, examples, tools, tests presented as Rust behavior, and release artifacts:
- loading or hosting the CLR/.NET runtime;
- P/Invoke, COM, C ABI, UniFFI, generated bindings, or any other foreign-function bridge to LibreMetaverse assemblies;
- invoking
dotnet, LibreMetaverse programs, or helper services as a subprocess to implement Rust API behavior; - embedding, redistributing, dynamically loading, or downloading compiled LibreMetaverse assemblies;
- forwarding calls over IPC/RPC to a running C# implementation;
- retaining a C# fallback path for APIs not yet implemented in Rust;
- treating an interoperability comparison harness as the implementation.
During development, the pinned C# source and compiled metadata may be inspected to inventory public signatures, understand algorithms and state transitions, and create golden inputs/outputs. The original C# tests and programs may be run separately as an oracle to produce or compare fixtures. Those development tools must remain outside the Rust runtime graph, must not be required to build or use published Rust crates, and must never satisfy a Rust compatibility test by executing the C# implementation.
Third-party native libraries such as OpenJPEG, Skia, libopus, operating-system APIs, and ordinary Rust crates are allowed where documented and license- compatible; the prohibition is specifically against reusing LibreMetaverse or another port of its implementation behind a foreign interface. All LibreMetaverse-specific protocol logic, models, state machines, managers, serialization rules, and public behavior must be reimplemented in Rust.
The final release gate must verify that published crate metadata and built
artifacts contain no dependency on dotnet, Mono, CLR hosting APIs, C#
assemblies, or LibreMetaverse bridge libraries. A clean machine with only the
documented Rust/native prerequisites must be able to build, test, document, and
run the Rust implementation.
2. Workspace design
| Rust crate | C# project | C# files | C# lines | Public declaration rows | Responsibility |
|---|---|---|---|---|---|
libremetaverse-types |
LibreMetaverse.Types |
22 | 9,292 | 703 | UUIDs, math types, enums, caches and small collections |
libremetaverse-structured-data |
LibreMetaverse.StructuredData |
10 | 4,069 | 303 | OSD value model and LLSD encodings |
libremetaverse-imaging |
LibreMetaverse.Imaging.Abstractions |
2 | 454 | 18 | codec traits and managed image representation |
libremetaverse-imaging-skia |
LibreMetaverse.Imaging.Skia |
1 | 369 | 3 | optional Skia codec adapter |
libremetaverse-prim-mesher |
LibreMetaverse.PrimMesher |
5 | 3,477 | 222 | prim/sculpt mesh generation |
libremetaverse |
LibreMetaverse |
210 | 138,916 | 8,982 | client, protocol, managers, assets and rendering model |
libremetaverse-rendering-simple |
LibreMetaverse.Rendering.Simple |
1 | 161 | 6 | simple renderer |
libremetaverse-rendering-mesh-foundry |
LibreMetaverse.Rendering.MeshFoundry |
1 | 852 | 14 | full mesh renderer |
libremetaverse-lsl-tools |
LibreMetaverse.LslTools |
175 | 27,856 | 962 | LSL lexer/parser generator and generated grammar |
libremetaverse-rlv |
LibreMetaverse.RLV |
32 | 6,296 | 262 | RLV parsing, restrictions, locks and callbacks |
libremetaverse-utilities |
LibreMetaverse.Utilities |
1 | 289 | 12 | high-level convenience helpers |
libremetaverse-voice-vivox |
LibreMetaverse.Voice.Vivox |
12 | 4,544 | 440 | Vivox control protocol adapter |
libremetaverse-voice-webrtc |
LibreMetaverse.Voice.WebRTC |
11 | 7,023 | 268 | WebRTC voice adapter |
libremetaverse is the umbrella client crate. It re-exports the foundational
crates under types, structured_data, and imaging_abstractions. Rendering,
voice, RLV, LSL tools, and native Skia remain separate so users do not pay for
large or platform-specific dependency trees unless requested.
The workspace uses Rust 2024 with MSRV 1.96. Generated code is committed so ordinary consumers do not need Python, .NET, Roslyn, or the upstream checkout.
3. Public API translation policy
Compatibility means the same concepts, wire behavior, state transitions, errors, and event payloads. It does not mean mechanically reproducing C# syntax. Apply these rules consistently:
| C# surface | Rust surface |
|---|---|
LibreMetaverse.Foo.Bar |
the owning crate plus foo::Bar; omit the repeated assembly namespace |
| PascalCase methods | snake_case; add a deprecated PascalCase forwarding method only when migration evidence justifies it |
Task<T> / Task |
async fn -> Result<T, Error> / async fn -> Result<(), Error> |
Async method suffix |
omit it when the Rust method is async; document the C# source name |
CancellationToken |
tokio_util::sync::CancellationToken at long-lived operation boundaries |
| nullable reference | Option<T>; never use a sentinel UUID/string unless the protocol defines one |
| exception hierarchy | small non-exhaustive thiserror enums per crate; retain source error and protocol context |
| public mutable property | private field plus validated getter/setter, or a public field only for inert value records |
| overloads | one method with typed parameter structs; use separate descriptive names when operations differ semantically |
| interface | object-safe trait when dynamic dispatch is required, otherwise a generic trait bound |
| delegate | Fn, FnMut, or FnOnce bounds; boxed callback only when callbacks are stored |
| .NET event | typed subscription returning a guard, or a tokio::sync::broadcast receiver for asynchronous fan-out |
IDisposable |
ownership and Drop; add explicit close/shutdown().await when failure or async work must be observed |
byte[] |
borrowed &[u8] for parsing, bytes::Bytes for shared network payloads, Vec<u8> for owned mutation |
IReadOnlyList<T> |
&[T] or Arc<[T]>; return iterators when callers do not need storage |
Dictionary |
HashMap; use BTreeMap only when deterministic ordering is observable |
| immutable collections | owned Vec/HashMap or Arc<[T]>; do not add a persistent-collection crate without a measured need |
| monitor/reader-writer lock | std::sync first; use parking_lot only after profiling or poisoning-policy review |
| generated flags enum | bitflags; preserve unknown bits with from_bits_retain |
| C# enum on the wire | #[repr(...)] plus checked conversion; include Unknown(raw) when the protocol is extensible |
UUID |
a compatibility newtype around uuid::Uuid, with explicit big/little-endian wire helpers |
| vectors/quaternions/matrices | compatibility structs with LibreMetaverse field/order semantics; use glam internally after parity tests prove conversions |
| dependency injection container | normal constructors and explicit trait parameters; add no service-locator equivalent |
The checked-in mapping ledgers make those rules concrete:
- properties become getter/setter methods; indexers become
item/set_itemmethods with their index parameters; - events become typed subscriptions returning a guard; stored delegates become
Fncallbacks, while CLRBeginInvoke/EndInvokerows are retained as intentional differences; inborrows immutably,refborrows mutably, andoutis a mutable output; optional parameters useOption<T>rather than Rust default arguments;- nullable values use
Option<T>at the exact metadata node, including nested generic arguments; - inheritance maps to traits and composition rather than class inheritance; generic parameters retain their names and cataloged constraints as explicit Rust bounds recorded in the mapping ledgers;
- operators receive stable trait-style names; overloads receive parameter-based descriptive suffixes and a stable ID suffix only when necessary;
[Flags]enums usebitflagswith unknown bits retained; wire enums use their exact width/discriminants and anUnknown(raw)form only when extensible;- all third-party types exposed by C# signatures map through project-owned cross-platform boundary types. No mapping may name a macOS-only API unless equivalent Linux and Windows implementations are provided.
Run python3 tools/generate_rust_mapping.py --check after any catalog or mapping
rule change. It regenerates in memory and rejects duplicate source IDs or Rust
destinations, stale/missing members, unresolved referenced types, invalid
statuses, missing assembly representatives, and platform-specific targets.
Public structs and enums that come from the protocol must be #[non_exhaustive]
when servers may add values. Do not derive Serialize/Deserialize as a
substitute for the specified wire encoding: LLUDP, LLSD, MessagePack, asset
formats, and login XML each require their own compatibility tests.
3.1 Full signature-shim gate
The first handover task after this structural shell is a complete callable API shim. Do not start real behavior while completing it.
Build the authoritative API catalog during development from compiled .NET
metadata after running the original source generators, not from the line-
oriented regex ledger alone. This extraction is an inventory tool only and must
not become a build-time or runtime dependency of the Rust crates.
Build one supported target framework into a temporary output directory and use
System.Reflection.Metadata or Roslyn symbols to emit machine-readable records
for every public/nested-public:
- class, struct, record, interface, enum and delegate, including generic parameters and constraints;
- constructor and method, including overloads, ref/out/in parameters, optional values and return type;
- property/indexer and getter/setter visibility;
- field and constant, including value and numeric width;
- event and delegate payload;
- base type, implemented interface and relevant public attribute.
api/public-api.json is that catalog. Its external_types section records each
non-LibreMetaverse type used by a public signature, its defining assembly, and
every owning LibreMetaverse documentation ID; it intentionally does not copy
the external assembly's member surface. Create api/RUST-MAPPING.tsv with one
row per C# member and resolve every external entry to Rust core/std, an adopted
crate, or a native MetaCrate replacement: stable C# documentation ID, C#
signature, Rust crate/path, Rust signature, mapping decision, and status. A
coverage checker must fail for missing members, duplicate Rust destinations,
unresolved external types, or stale source IDs. api/SURFACE.tsv remains useful
for source navigation but is not the coverage authority.
Each mapped Rust item must actually compile and be callable:
- methods, constructors and property accessors have their final reviewed Rust names, parameters, ownership/borrowing, asyncness, and return types;
- overloads use stable descriptive Rust names or typed argument structs, with the mapping recorded explicitly;
- enums contain their real variants/discriminants and flags contain their real bits, because tests and downstream code must compile against them;
- event payloads, delegates and interfaces have their real fields/trait methods;
- generic bounds and thread-safety guarantees are deliberate, not erased to an untyped placeholder.
The body is the only part allowed to be fake. Fallible Rust APIs return a shared
typed Error::NotImplemented(NotImplemented { csharp_member }) error. Infallible constructors,
accessors, operators and trait methods call a shared unimplemented_api!
placeholder that panics with the stable C# member ID. Constants and enum values
must have their real values. Never return plausible defaults such as false,
zero, an empty collection, or a nil UUID: those could make translated tests pass
for the wrong reason.
The failure contract lives in libremetaverse-types and every API crate exposes
it as crate::Error:
- synchronous fallible members return
Err(Error::NotImplemented(NotImplemented::new(CSHARP_ID))); async members do the same when first polled and start no work first; - a stream-producing operation returns the typed error before exposing a
stream; an intrinsically infallible stream factory uses
unimplemented_api!instead of returning an empty stream; - callback registration and stored-trait construction fail before retaining the callback/object; generated callbacks are never silent no-ops;
- infallible constructors, accessors, operators, and trait methods call
unimplemented_api!(CSHARP_ID), whose panic marker is distinct from every typed domain error; - constants, enum discriminants, flags, and inert public fields use their real
catalog values/layout. Generated shims do not derive
Defaultunless the reviewed API explicitly defines that behavior.
python3 tools/generate_rust_mapping.py --check scans every generated shim and
rejects Default derives, plausible false/zero/empty/nil returns, and any
function body that lacks the standardized typed error or panic mechanism.
The signature gate is complete only when:
- the metadata catalog reports 100% public type/member mapping coverage for all 13 library assemblies, including generated APIs;
- all crates build and
cargo docresolves every public signature; - a downstream compile fixture can name/call every mapped Rust item;
- there are no erased
ShimValue, dynamic argument bags, invented variadics, or member signatures present only as ledger text; - implementation bodies contain only the standardized failure placeholder, constants, enum discriminants, and inert data-layout boilerplate.
3.2 Full semantic test-suite gate
After the full signature shim compiles, translate the entire C# test suite while leaving all library bodies unimplemented. Every one of the 1,289 NUnit invocations must become a Rust test with the same inputs, setup, operation, observations, assertions, tolerances and expected error/event behavior. Shared C# test helpers become shared Rust test helpers; embedded fixtures and literal payloads are copied with license/source attribution and byte hashes.
The parity harness is in place: every invocation has a stable source/case ID,
parameter identity, body hash, category, fixture dependency list, Rust location,
and semantic-review status. Reviewed tests are identified by parity-case
markers in hand-written Rust files, so regeneration records unresolved cases
without creating fake executable tests and fails on body drift. The checked-in
audit reports pending, translated, ignored-live, benchmark, drifted, missing,
duplicate, stale, and unreviewed cases. The initial handover contained 1,289
unreviewed cases; the current ledger contains all 1,266 ordinary cases as
translated, 19 reviewed live-grid cases, four benchmark-reviewed cases, and no
pending or unreviewed cases.
The Rust tests must call the public APIs rather than internal replacements. Where the C# tests call internal members through friend-assembly access, record that fact and place equivalent Rust unit tests inside the owning crate without making the member public solely for testing.
The tests must compile even though running them is red. An expected C# exception
maps to a specific Rust Err variant, never to should_panic around a public
API call; this ensures an unimplemented_api! panic remains a failure rather
than accidentally satisfying the test. Only tests whose C# purpose is genuinely
to verify a thrown runtime invariant may use catch_unwind/should_panic, and
they must distinguish the intended invariant from the placeholder panic.
The test-suite gate is complete only when:
- no generated placeholder or dispatcher body can count as reviewed;
- the parity catalog reports exactly one reviewed Rust case for every upstream
[Test]/[TestCase], with the matching C# body hash; cargo test --workspace --no-runcompiles all tests against public signatures;- a controlled test run fails at standardized unimplemented member IDs rather than from missing symbols, compile errors, malformed fixtures, or weakened assertions;
- live-server/device tests are faithfully translated and are conditionally ignored only while their documented credentials are absent;
- no production method contains real behavior beyond what is required to make signatures and constants compile.
3.3 Fixed controlled-red baseline
Milestone 03 closes against the pinned 1,289-invocation catalog, not the older
1,295 source-text estimate. The six-case difference is intentional: five
attributes were commented out and one parameterless [Test] marker duplicated
a parameterized [TestCase] method. Reintroducing those entries would create
tests that NUnit never runs.
tests/red-suite-baseline.json fixes the machine-checked ledger at 1,266
ordinary translations, 19 live-grid translations, four benchmarks, and zero
pending, drifted, duplicate, missing, or unreviewed cases. Run the controlled
audit with:
python3 tools/audit_red_suite.py
With all three live-grid credentials present, the baseline executes every case:
22 tests pass, 1,287 fail at 131 standardized C# member IDs, and none are
ignored. Twenty passes are gate/support tests. The only two passing parity cases
verify the required BAKED_TEXTURE_COUNT constant and composable validation
flags; they do not represent implemented production behavior. Without complete
credentials, the same audit conditionally ignores exactly the 19 live-grid
cases and requires the remaining 1,268 parity failures to retain standardized
member IDs. Production crates remain the failure-only milestone-02 shims.
4. Validated Rust dependency map
Versions below were queried from crates.io on 2026-08-08 with cargo search
and, for risky/native crates, cargo info. They are candidates, not blanket
dependencies. Add one to a crate only when its first implementation uses it,
then commit Cargo.lock for application/test reproducibility. Recheck versions,
MSRV, features, and licenses at adoption time.
| .NET/NuGet responsibility | Rust candidate and validated version | Adoption notes |
|---|---|---|
| async runtime, channels, timers | tokio 1.53.1, tokio-util 0.7.19, futures 0.3.33 |
One Tokio runtime owned by the application; libraries never create nested runtimes. |
System.Net.Http / WinHTTP |
reqwest 0.13.4 |
MSRV 1.85. Use rustls/default TLS deliberately, connection pooling, streaming bodies, and injected client/timeouts. |
| JSON and general data mapping | serde 1.0.229, serde_json 1.0.151, serde_bytes 0.11.19 |
Use only where the C# implementation is schema-driven JSON. Hand-write LLSD encodings. |
| MessagePack 3.1.8 | rmp-serde 1.3.1, rmpv 1.3.1 |
Golden byte vectors must prove integer widths, maps, extensions, and field ordering. |
| UUID | uuid 1.24.0 |
Wrap it; do not expose dependency-specific endian assumptions as protocol behavior. |
| byte buffers | bytes 1.12.1, bytemuck 1.25.2 |
Prefer explicit endian reads. bytemuck is only for proven POD layouts, never untrusted variable packets. |
| vector/matrix math | glam 0.33.3 |
Keep compatibility newtypes to control component order, precision, equality, and serialization. |
| flags | bitflags 2.13.1 |
Retain unknown bits from newer grids. |
| errors | thiserror 2.0.20 |
Application binaries may use richer reporting; public libraries return typed errors. |
| ZLogger / Microsoft logging | tracing 0.1.44, tracing-subscriber 0.3.23 |
Libraries emit spans/events; binaries choose formatting and filters. Never log credentials or capability tokens. |
| rate limiting | governor 0.10.4 |
Candidate for caps categories. First reproduce burst/refill/cancellation behavior with deterministic clock tests. |
| compression and tar archives | flate2 1.1.9, tar 0.4.46 |
Enforce decompressed-size, path traversal, and entry-count limits on untrusted OAR data. |
| XML and URL | quick-xml 0.41.0, url 2.5.8, base64 0.23.1 |
Streaming XML for LLSD/login; retain exact URL escaping behavior with fixtures. |
| CoreJ2K 2.3.3.91 | system OpenJPEG 2.5.4+ through the private libremetaverse-openjpeg adapter |
Opt-in jpeg2000 feature; checked-in minimal bindings avoid a libclang build dependency. Bounded memory streams and header validation isolate native code. Golden tests cover channel order, alpha, 16-bit precision, dimensions, discard levels, lossless/lossy modes, and malformed input. |
| SkiaSharp 4.150.1 | skia-safe 0.99.0 |
Adopted behind the opt-in skia feature. Target-specific official binary-cache feature sets provide BMP/GIF/ICO/JPEG/PNG/WBMP/WebP decoding on macOS, Linux, and Windows; checked owned buffers keep Skia types out of the core image API. |
| Pfim 0.11.4 | image 0.25.10, ddsfile 0.6.0 |
image covers TGA and common DDS decoding; ddsfile exposes DDS container details. Golden files decide whether both are needed. |
| OggVorbisEncoder 1.2.2 | vorbis_rs 0.5.6 |
BSD-3-Clause, MSRV 1.82, backed by C libraries. Feature-gate native audio encoding. |
| SIPSorcery 8.0.23 | webrtc 0.20.0, cpal 0.18.1, opus 0.3.1 |
Validate SDP, ICE, data-channel framing, audio formats, device hotplug, and native libopus deployment separately. Do not claim parity from successful compilation. |
| LSL generated parser | lalrpop 0.23.1 |
Candidate only. Preserve grammar conflicts, recovery, token positions, and diagnostics before replacing the generated parser. |
| NUnit/Moq | built-in test harness, mockall 0.15.0, proptest 1.11.0 |
Prefer fakes and deterministic protocol fixtures; use mocks only for interaction contracts. Add properties after direct parity cases exist. |
| NUnit benchmarks | criterion 0.8.2 |
Port benchmark-category methods to benches/; never make timing thresholds correctness tests. |
| sample command lines | clap 4.6.6 |
Add when implementing the first real program CLI; keep credentials in arguments/env only long enough to authenticate. |
System.*, BCL compatibility packages, immutable collections, DI abstractions,
and object pooling usually map to the Rust standard library and ownership. Do
not select a crate merely because NuGet used one. In particular, begin with
Mutex<HashMap<...>> rather than a concurrent-map dependency and add a pool
only after allocations appear in a profile.
5. Module-by-module implementation guidance
5.1 libremetaverse-types
This is the first implementation crate because nearly every protocol and test depends on it.
- UUID: wrap
uuid::Uuid; implement zero/random/parse/format, byte-array constructors, CRC/combine helpers, ordering, hashing, and every endian path fromUUID.cs. Keep theUUIDname as a public compatibility type. - Vector2/3/3d/4, Quaternion, Matrix4, Ray, Color4: preserve constructors,
constants, component order, normalization of zero values, quaternion
multiplication direction, approximate-comparison helpers, and binary/string
formats. Use
glaminternally only after conversion tests pass. - Enums and primitive enums: translate exact numeric discriminants and flag
widths. Use
bitflagswhere C# uses[Flags]; retain unknown values/bits. - CRC32 and conversions: port byte-for-byte with fixed golden vectors and explicit endianness. Avoid platform-sized integers on the wire.
- CacheDictionary, ExpiringCache, TokenBucket: inject a clock in tests;
preserve expiry/burst semantics. Start with
HashMapplus one lock. - DoubleDictionary and MultiValueDictionary: expose iterators and borrowing rather than cloning collections. Preserve duplicate and replacement rules.
Completion gate: every Types-related parity test is a semantic Rust test, all format/byte fixtures match C#, and no generated shim remains for a Types API.
5.2 libremetaverse-structured-data
- OSD value model: implement a non-exhaustive enum for undefined, boolean,
integer, real, string, UUID, date, URI, binary, array, and map. Preserve C#
conversion/default behavior deliberately; do not let
serde_json::Valuedefine the public model. - OSDArray/OSDMap: wrap
Vec<OSD>and an order-appropriate map. Verify whether serialized map ordering is observable in each format before choosingHashMapversusBTreeMap/insertion order. - Binary LLSD: streaming parser over
&[u8]/Read; checked lengths, recursion limits, exact endian rules, and no panics on malformed input. - Notation LLSD: preserve escaping, length-prefixed strings/binary, numbers, dates, URIs, whitespace, and error positions.
- XML LLSD: use
quick-xml; reject/entity-limit hostile input and retain exact element mapping. - JSON LLSD: use
serde_jsonbehind explicit OSD conversions so UUID/date/ URI/binary tagging remains compatible. - Protobuf OSD: first identify whether this is a stable external format. If stable, preserve tags with generated Rust types; otherwise keep it internal.
Completion gate: parse and emit C# golden bytes/text for every format, including malformed, deep, empty, Unicode, NaN/infinity, and cross-format cases.
5.3 libremetaverse-imaging
- Translate
ManagedImageas an owned image buffer with explicit dimensions, channel flags/layout, stride, and checked size arithmetic. - Translate
ITextureCodecto an object-safeTextureCodectrait only if runtime codec selection is needed. Codec methods return typed decode/encode errors and accept limits. - Keep public APIs independent of Skia/OpenJPEG types so pure-Rust or platform codecs can be substituted.
Completion gate: channel interleave/deinterleave, resize, alpha/bump behavior, and invalid-dimension tests match the original.
5.4 libremetaverse-imaging-skia
Implement SkiaTextureCodec as an optional adapter with skia-safe. Convert at
the boundary to/from ManagedImage; never expose skia_safe types in the core
trait. CI must cover macOS, Linux, and Windows or explicitly document unsupported
targets. Validate native binary provenance and packaging.
5.5 libremetaverse-prim-mesher
- Port
Coord,Quat, faces, paths, profiles, viewer faces, and indexing as value types with checked indices. - Port linear/circular extrusion before sculpt meshing. Preserve winding, normals, UVs, face numbering, hollow/profile cuts, twists, tapers, and shears.
- Port
SculptMapdecoding throughlibremetaverse-imaging, thenSculptMesh. - Make OBJ export a formatting layer over mesh data; deterministic output is a testable contract.
Completion gate: all 48 PrimMesher rendering-test invocations and all MeshFoundry tests are semantic ports, face indices stay in range, and C# versus Rust mesh fixtures compare within the original tolerances.
5.6 libremetaverse protocol and client core
Implement the central crate in dependency slices, not file order.
Wire protocol and generated packets
- Port
BitPack, zero coding, packet headers, ACK handling, sequence numbers, fragmentation, throttles, andUDPPacketBufferfirst. - Replace the six Roslyn source generators with a Rust
codegentool/build step reading the samedata/andlinden/inputs. Commit generated packet, visual-parameter, skeleton, tree, genepool, and attention Rust sources. - The generator must produce deterministic output; CI regenerates to a temporary directory and fails on diff. Generated packet tests compare exact bytes and IDs against C# fixtures.
- Parsers operate on borrowed slices where practical and reject truncated, oversized, or unknown blocks without unchecked indexing.
Networking and capabilities
UDPBase,NetworkManager,Simulator,ProtocolManager, andUdpThrottleuse Tokio UDP tasks with explicit ownership and cancellation. One task owns socket writes; decoded events cross bounded channels to prevent unbounded memory growth.Caps,HttpCapsClient, event queue, caps-to-packet, and rate limiting use an injectedreqwest::Client. Preserve redirects, content types, retry policy, cancellation, progress reporting, and non-HTTP location rejection.- Login/XML-RPC, seed capabilities, region crossing, reconnect, logout, and timeout state transitions must be modeled explicitly and tested with local fake servers before live-grid use.
- Capability URLs/tokens and login credentials are secrets: redact them from
Debugand logs.
Grid client and service composition
GridClientowns managers and shared runtime state. Construct it with aGridClientBuilderonly for real optional policies/codecs; required services remain constructor arguments.- Replace
IGridClientand service-collection extensions with focused traits at test seams. Avoid a general service locator. Settings/*become typed configuration structs grouped like the C# settings, with validated durations/sizes and defaults proven by tests.Loggermaps totracing;UtilizationStatisticsexports snapshots without making a metrics backend mandatory.
Agent, movement, chat, effects, teleport and money
- Port
AgentManagerpartial files as modules underagent, but expose one coherentAgentManagerAPI. Keep camera/movement state separate from network commands. - Event-argument classes become event structs. Subscriptions must be removable and must not retain the whole client after drop.
- Teleport and region crossing are explicit state machines with timeout, cancellation, progress, failure reason, and simulator handoff tests.
- Money and permissions paths use fixed integer widths and never log sensitive transaction descriptions without opt-in.
Inventory
- Port inventory enums/base records/nodes/store before
InventoryManager. Preserve link resolution, folder/item distinctions, ownership and permissions. - Separate local tree/cache mutation from UDP/capability transport. This makes
InventoryAISClientindependently testable with recorded LLSD fixtures. - Async fetch, give, rez, task inventory, current outfit folder, and callbacks
must have cancellation and deduplication tests. Do not hold locks across
.await. - Persisted cache format needs versioning, atomic replacement, corruption handling, and bounded deserialization.
Assets, appearance, avatar and animesh
- Port the base
Assetmodel and individual asset types as typed wrappers over validated bytes. Decode lazily where the C# behavior permits. AssetManager,AssetCache,DownloadManager, andTexturePipelineshare a deduplicating request layer with bounded concurrency and cancellation.- Port wearables, baking, current-outfit policies, texture compositing, avatar
definitions, skeleton, visual params, animations, gestures, and animesh in
that order. Copy and retain the upstream Linden assets only after reviewing
their separate
cc-by-sa-3.0.txtobligations. - Animation interpolation, joint transforms, skinning, attachment rigs, and physics must use the original test tolerances and matrix multiplication order.
Objects, parcels, terrain, environment and rendering model
- Port
Primitive, texture entry, media, particles, permissions, materials, and simulator collections beforeObjectManager. - Object updates need an explicit decode/apply split; preserve terse/compressed/ full update semantics, stale-update behavior, and parent-child linking.
- Parcel, estate, grid, terrain, sound, environment, interest-list, directory, friends, groups, marketplace, and experience managers each get a transport- independent state/model layer plus network handlers.
- Terrain codecs and GLTF/material handling use golden fixtures and size limits. GLTF JSON may use Serde, but binary buffers/accessors require checked offset arithmetic.
Archives and import/export
OarFile, tar reader/writer, region settings, and asset archiver usetarandflate2with path normalization, extraction-root enforcement, and size/count limits.- Collada/OBJ/GLTF import/export keeps format DTOs separate from world objects. Deterministic output and round-trip fixtures are required; live upload is a later layer.
Threading and utilities
- Replace custom events/semaphores/read-write locks with Tokio or
std::syncprimitives where semantics match. Port custom optimistic/spin locks only if a benchmark and correctness test show they are still required. - Replace array pools with
Vec/Bytesreuse first. Add pooling only after profiling and with a bounded pool. - Observable/event dictionaries expose change streams without executing user callbacks while holding internal locks.
Completion gate for the core crate: no public core shim remains; all offline parity tests pass; malformed-input fuzz targets do not panic; fake-server login, caps, UDP, reconnect, and cancellation scenarios pass on all supported OSes.
5.7 Rendering crates
libremetaverse-rendering-simple ports SimpleRenderer first as the reference
implementation of the core rendering trait. It should favor clarity and serve
as a fixture generator.
libremetaverse-rendering-mesh-foundry then ports the complete MeshFoundry
pipeline using types, OSD, core assets, PrimMesher, and imaging. Preserve face
grouping, materials, transforms, skin weights, normals/tangents, texture
coordinates, and output ordering. Do not merge the two crates: users should be
able to choose the small renderer without the full mesh pipeline.
5.8 libremetaverse-lsl-tools
- Treat
Tools/as the parser-generator implementation andYYClass/as generated grammar output. Do not line-by-line port all generated classes. - First translate lexer tokens, source locations, errors, comments, precedence, grammar productions, and recovery behavior into compatibility tests.
- Re-express the grammar in LALRPOP only if it reproduces accepted/rejected programs and diagnostic locations. Otherwise port the existing table machine with generated Rust tables.
- Generated tables are committed and deterministically reproducible. Keep public
token/parser entry points recognizable for migration, while Rust iterators and
Resultreplace C# enumerators/exceptions.
Completion gate: all grammar fixtures and error-recovery cases match, and the generated code is warning-free without 150 hand-maintained class shims.
5.9 libremetaverse-rlv
- Port message/command parsing and enums first, preserving case, separators, attachment point aliases, option handling, and invalid-command errors.
- Port blacklist/provider/callback interfaces as narrow traits. Return typed actions from pure parsing/decision code; execute viewer actions through an injected callback adapter.
- Port restriction state, camera settings, shared folders, inventory maps, locked folders, attachment requests, permissions, and service orchestration.
- Use immutable snapshots or short locks for restriction reads; callbacks run after locks are released.
Completion gate: every RLV command/query/restriction/exception test case is a semantic Rust test and all original expected error distinctions remain visible.
5.10 libremetaverse-utilities
Port the convenience functions only after their owning lower-level APIs exist. Prefer free functions or extension traits with explicit client references. Do not use this crate to bypass ownership, cancellation, or error handling in the core crate.
5.11 Voice crates
For libremetaverse-voice-vivox:
- port XML/control definitions, account/session/participant models, TCP pipe, connector, gateway, and manager as a protocol adapter;
- keep blocking compatibility APIs as thin wrappers in the program layer, not library methods that create runtimes;
- Vivox SDK/server availability and licensing are external prerequisites. The crate can be protocol-complete without shipping proprietary native binaries.
For libremetaverse-voice-webrtc:
- port signaling messages and data-channel framing before peer/media handling;
- adapt
webrtc,cpal, andopusbehind local traits so protocol tests do not require audio hardware; - make device selection, sample format/rate/channel conversion, mute, reconnect, ICE/SDP negotiation, and teardown explicit state transitions;
- CI uses virtual/fake audio; real-device and live voice validation is a manual gate on macOS, Linux, and Windows.
Completion gate: recorded signaling/data-channel fixtures pass offline, fake audio round-trips have bounded latency/loss behavior, teardown leaves no tasks, and opt-in live validation succeeds without logging tokens.
6. Test-first migration workflow
The compatibility suite is deliberately red in the shell stage. First finish the complete signature gate in section 3.1 across all crates. Then translate the complete suite in section 3.2 across all crates. Only after both global gates may implementation proceed in dependency order.
For each pending catalog case:
- Open the exact C# source/line and verify its method-body SHA-256 still matches
tests/upstream-tests.json. - Identify setup, action, assertions, parameterized cases, fixtures, categories, and intended exception/event/timeout behavior.
- Write an explicit Rust
#[test]body in a hand-written test file, add itsparity-casemarker, and call the mapped library API directly. - Port literal bytes, UUIDs, timestamps, tolerances, culture assumptions, and ordering exactly. Do not weaken an assertion merely because the shim differs.
- If the needed API is absent, stop and treat that as a signature-coverage defect: add the missing catalog/mapping entry and final failing signature, then rerun the 100% API coverage gate.
- Keep the test red at the standardized unimplemented member boundary; do not implement behavior during the test-translation stage.
- Mark the JSON entry with a future
status: semanticfield only after review; enhance the generator/checker to preserve those reviewed entries rather than overwrite them.
Test translation rules:
- NUnit
[TestCase]invocations remain separate Rust tests so failures identify the exact case. Assert.Throwsmaps to matching a specific Rust error variant, not any error.- floating assertions retain the original tolerance and NaN behavior.
- async tests use paused/injected time where possible; no arbitrary sleeps.
- Moq interaction tests prefer a small fake recording calls; use Mockall only when a trait contract has many independent expectations.
RequiresLiveServertests useGRID_USER,GRID_PASSWORD, andGRID_LOGIN_URLfrom the process environment or the ignored workspace.env; they are ignored when any value is absent and run automatically when all three are present..env.exampledocuments the non-secret shape.- benchmark-category methods move to Criterion and are removed from correctness pass/fail counts.
- malformed wire/asset tests become fuzz seeds after direct parity is passing.
Required gates:
cargo fmt --all -- --check
python3 tools/generate_rust_mapping.py --check
python3 tools/generate_api_shims.py --check
python3 tools/check_api_coverage.py
cargo check --workspace --all-targets
cargo check --locked --manifest-path tests/api-compile/Cargo.toml
cargo test --workspace --no-run
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo doc --workspace --no-deps
During Stage 0, pending cases exist only in the parity catalog. During the full test-suite gate, the catalog's pending count is the explicit remaining-work count and CI prohibits increases. At the end of that gate, tests still fail, but only because callable production signatures report standardized not-implemented member IDs. During implementation, that failure inventory then shrinks as behavior turns the fixed suite green.
7. Program and live-grid validation plan
The programs package contains these binary targets:
| Rust binary | C# project | Port purpose |
|---|---|---|
simple-bot |
SimpleBot |
login, IM/chat events, movement and animations |
prim-inspector |
PrimInspector |
object discovery/properties and transforms |
inventory-explorer |
InventoryExplorer |
inventory fetch, search, stats and export |
packet-dump |
PacketDump |
raw/decoded packet subscription and logging |
irc-gateway |
IRCGateway |
concurrent external chat bridge |
test-client |
TestClient |
full interactive client and all 128 command targets |
vivox-test |
VivoxTest |
Vivox account/session/participant validation |
webrtc-test |
WebRtcTest |
WebRTC signaling, devices and audio validation |
osd-inspector |
OSDInspector |
offline LLSD inspect/convert/validate/primitive round trip |
Port osd-inspector first because it is offline and validates StructuredData.
Then port simple-bot, packet-dump, prim-inspector, and
inventory-explorer as vertical grid slices. Port TestClient commands by folder
(system/login first, then communication, inventory, objects, movement, land,
groups, friends, directory, appearance, voice, stats) while keeping
commands::TEST_CLIENT_COMMANDS as the completeness inventory. Voice programs
come last because they need native/device and service prerequisites.
Live validation must use a dedicated test account and an explicitly supplied grid URI. Secrets are read from environment variables or the OS secret store, never source/config committed to Git. A live smoke run records sanitized:
- server/grid/version and Rust commit;
- login and initial simulator/capabilities completion;
- IM/chat send/receive;
- movement/teleport/region crossing;
- inventory fetch and one reversible test-folder operation;
- object discovery/property request;
- asset/texture download and decode;
- clean logout and task/socket teardown.
Operations that spend currency, modify estate/parcel state, upload assets, delete inventory, or affect other users require separate opt-in flags and are not part of the default smoke test.
8. Ordered delivery milestones
- Structural inventory shell (current Stage 0): crates, type/declaration ledgers, failing test parity entries, program targets, license, and this plan compile. This is not the callable API or semantic test suite.
- Full public API signature shim: extract authoritative compiled metadata, map 100% of public members to reviewed Rust signatures, and implement only standardized failing bodies. All signatures and downstream compile fixtures build before test translation begins.
- Full semantic test translation: replace all 1,289 pending cases with exact reviewed Rust equivalents using the public shim. The entire suite compiles and fails only at standardized unimplemented member boundaries.
- Types implementation: UUID, math, enums, conversions, collections, cache/token bucket; turn only the already-existing Types tests green.
- StructuredData implementation: OSD plus binary/notation/XML/JSON/protobuf compatibility; turn only the already-existing StructuredData tests green.
- Imaging and PrimMesher implementation: managed images, codec trait, TGA/J2K adapter, extrusion/sculpt/OBJ and rendering tests.
- Wire codegen: deterministic Rust packet/visual/skeleton/tree/genepool/ attention generation and golden packet bytes.
- Networking/client lifecycle: UDP, caps, login, simulators, settings, cancellation, reconnect and logout against fake servers.
- World vertical slices: agent, inventory, assets/appearance, objects, parcels/terrain/environment, social/directory/groups, marketplace/estate.
- Rendering, RLV, LSL tools: implement against their already-translated suites and add public migration docs.
- Programs/live grid: offline inspector, four core examples, TestClient, then voice adapters and opt-in live/device validation.
- Hardening/release: fuzz untrusted parsers, cross-platform native CI, MSRV, docs/examples, semver/API audit, license/asset audit, performance comparison against the separately run pinned C# implementation, and an artifact/dependency audit proving there is no CLR, .NET assembly, subprocess, RPC, or LibreMetaverse foreign-interface dependency.
Each milestone ends with zero unreviewed generated shims in its owned modules, no weakened parity assertions, formatted/clippy-clean code, public Rust docs that cite the corresponding C# concept, and a refreshed completeness report from the checked-in ledgers.