Files
MetaCrate/RUSTREWRITE.md

49 KiB

LibreMetaverse to Rust rewrite plan

1. Scope and source of truth

MetaCrate is a new native Rust implementation of the public API and observable behavior of LibreMetaverse. LibreMetaverse is reference material, not a runtime dependency or implementation component. The source snapshot used to create this shell is:

  • repository: https://github.com/cinderblocks/libremetaverse
  • branch: master
  • commit: 2aa70bb68513b39795da5d13c88f31b86e85a3ba
  • commit date: 2026-08-07
  • local checkout state during inventory: clean
  • license base: libremetaverse/LICENSE.txt, BSD 3-Clause

Generated inventories are intentionally checked in:

  • api/TYPES.tsv maps every discovered public C# type declaration to a Rust crate/module/type and source line.
  • api/SURFACE.tsv records every public declaration line in the 13 library projects, including a SHA-256 digest.
  • api/public-api.json is the authoritative compiled-metadata catalog for all 13 library assemblies after all six source generators run. It also records every external type referenced by a public signature.
  • api/RUST-TYPES.tsv resolves all 3,066 public and 142 external signature types, plus two support traits referenced by public inheritance metadata.
  • api/RUST-MAPPING.tsv records the reviewed Rust destination and complete signature decision for all 30,789 public members.
  • tests/upstream-tests.json records every NUnit invocation and the SHA-256 of its C# method body.
  • tests/PARITY.md maps every NUnit invocation to its Rust test name.
  • programs/upstream-programs.json records every source file and digest for the nine sample/tool projects.

Run python3 tools/generate_surface.py after deliberately changing the pinned upstream snapshot, then run python3 tools/extract_public_api.py to rebuild the authoritative metadata catalog twice and require byte-identical output. The source generator rejects a test count other than 1,289 so upstream drift cannot silently remove tests. Update both commit constants and review all ledger diffs in the same change. The earlier 1,295 inventory included five commented-out [Test] attributes and double-counted the parameterless [Test] marker on one parameterized [TestCase] method; those are not NUnit invocations.

Current shell status

The current workspace is a structural baseline, not a working client:

  • all 13 public .NET library projects have corresponding Rust crates;
  • 1,705 source-discovered public C# type declarations produce compiling Rust type/trait shims;
  • 12,195 public declaration lines are retained in the API ledger;
  • compiled metadata records 3,066 public types and 30,789 public members with matching independent ECMA-335 counts, with all 142 external signature types resolved to cross-platform Rust or project-owned boundary types;
  • the completed Types, StructuredData, Imaging, PrimMesher, rendering, main wire/data, core runtime/networking, avatar-facing manager, and world/social/service manager, RLV, LSL tools, Utilities, Vivox, and WebRTC slices expose all 3,066 final Rust-facing types and all 30,789 callable members with real fields/constants/enum values and standardized failing bodies;
  • all 1,289 real NUnit [Test]/[TestCase] invocations have stable catalog entries; 145 are translated, four benchmarks are reviewed, and 1,140 remain pending without executable placeholder tests;
  • all nine sample/tool projects have compiling Rust binary targets;
  • the 128 TestClient command source files are retained as a command inventory.

No generated type shells remain pending: all cataloged members are callable, and the standalone downstream fixture names or invokes all 3,066 types and 30,789 members with zero exclusions. api/SURFACE.tsv is still an inventory aid, not proof of API coverage: it records declaration lines but does not fully parse multiline signatures. Likewise, a pending catalog entry is not a semantic translation. The full public API signature shim gate is complete; semantic test translation is the next stage.

The avatar-facing slice preserves the C# agent, movement/camera, appearance and outfit-policy, inventory/AIS/store, asset/cache, avatar, and animesh concepts. Long-running mapped calls retain an explicit cross-platform cancellation token and return the shared Error::NotImplemented(NotImplemented { csharp_member }) error; events return a typed subscription guard so callback lifetime is visible to Rust callers.

The world-facing slice preserves the C# object, parcel, terrain, grid, environment, estate, directory, friend, group, sound, interest-list, marketplace, and reporting concepts. Server-facing async operations retain explicit cancellation and shared typed failures; request/reply events expose typed payloads and subscription guards, while catalog constants and flag values remain real.

The extension slice preserves the C# RLV restrictions, callbacks and services; LSL lexer/parser/generator and diagnostic types; Utilities helpers; Vivox protocol/session/manager types; and WebRTC signaling/device/session concepts. Blocking compatibility surfaces stay explicit, stored callbacks and events use typed ownership, and parser/voice bodies fail before performing parsing, networking, device access, or proprietary/native work.

No functional porting starts until the semantic test-suite gate is complete across the entire workspace, rather than one crate at a time:

  1. every public C# type and member has a reviewed, callable Rust signature with an intentionally failing body;
  2. every C# test is translated semantically and compiles while calling that Rust API;
  3. only then is behavior implemented, one crate at a time, until those already translated tests turn green.

Full-rewrite constraint

The final project must be entirely implemented in new Rust code. The following are prohibited in production libraries, examples, tools, tests presented as Rust behavior, and release artifacts:

  • loading or hosting the CLR/.NET runtime;
  • P/Invoke, COM, C ABI, UniFFI, generated bindings, or any other foreign-function bridge to LibreMetaverse assemblies;
  • invoking dotnet, LibreMetaverse programs, or helper services as a subprocess to implement Rust API behavior;
  • embedding, redistributing, dynamically loading, or downloading compiled LibreMetaverse assemblies;
  • forwarding calls over IPC/RPC to a running C# implementation;
  • retaining a C# fallback path for APIs not yet implemented in Rust;
  • treating an interoperability comparison harness as the implementation.

During development, the pinned C# source and compiled metadata may be inspected to inventory public signatures, understand algorithms and state transitions, and create golden inputs/outputs. The original C# tests and programs may be run separately as an oracle to produce or compare fixtures. Those development tools must remain outside the Rust runtime graph, must not be required to build or use published Rust crates, and must never satisfy a Rust compatibility test by executing the C# implementation.

Third-party native libraries such as OpenJPEG, Skia, libopus, operating-system APIs, and ordinary Rust crates are allowed where documented and license- compatible; the prohibition is specifically against reusing LibreMetaverse or another port of its implementation behind a foreign interface. All LibreMetaverse-specific protocol logic, models, state machines, managers, serialization rules, and public behavior must be reimplemented in Rust.

The final release gate must verify that published crate metadata and built artifacts contain no dependency on dotnet, Mono, CLR hosting APIs, C# assemblies, or LibreMetaverse bridge libraries. A clean machine with only the documented Rust/native prerequisites must be able to build, test, document, and run the Rust implementation.

2. Workspace design

Rust crate C# project C# files C# lines Public declaration rows Responsibility
libremetaverse-types LibreMetaverse.Types 22 9,292 703 UUIDs, math types, enums, caches and small collections
libremetaverse-structured-data LibreMetaverse.StructuredData 10 4,069 303 OSD value model and LLSD encodings
libremetaverse-imaging LibreMetaverse.Imaging.Abstractions 2 454 18 codec traits and managed image representation
libremetaverse-imaging-skia LibreMetaverse.Imaging.Skia 1 369 3 optional Skia codec adapter
libremetaverse-prim-mesher LibreMetaverse.PrimMesher 5 3,477 222 prim/sculpt mesh generation
libremetaverse LibreMetaverse 210 138,916 8,982 client, protocol, managers, assets and rendering model
libremetaverse-rendering-simple LibreMetaverse.Rendering.Simple 1 161 6 simple renderer
libremetaverse-rendering-mesh-foundry LibreMetaverse.Rendering.MeshFoundry 1 852 14 full mesh renderer
libremetaverse-lsl-tools LibreMetaverse.LslTools 175 27,856 962 LSL lexer/parser generator and generated grammar
libremetaverse-rlv LibreMetaverse.RLV 32 6,296 262 RLV parsing, restrictions, locks and callbacks
libremetaverse-utilities LibreMetaverse.Utilities 1 289 12 high-level convenience helpers
libremetaverse-voice-vivox LibreMetaverse.Voice.Vivox 12 4,544 440 Vivox control protocol adapter
libremetaverse-voice-webrtc LibreMetaverse.Voice.WebRTC 11 7,023 268 WebRTC voice adapter

libremetaverse is the umbrella client crate. It re-exports the foundational crates under types, structured_data, and imaging_abstractions. Rendering, voice, RLV, LSL tools, and native Skia remain separate so users do not pay for large or platform-specific dependency trees unless requested.

The workspace uses Rust 2024 with MSRV 1.85. Generated code is committed so ordinary consumers do not need Python, .NET, Roslyn, or the upstream checkout.

3. Public API translation policy

Compatibility means the same concepts, wire behavior, state transitions, errors, and event payloads. It does not mean mechanically reproducing C# syntax. Apply these rules consistently:

C# surface Rust surface
LibreMetaverse.Foo.Bar the owning crate plus foo::Bar; omit the repeated assembly namespace
PascalCase methods snake_case; add a deprecated PascalCase forwarding method only when migration evidence justifies it
Task<T> / Task async fn -> Result<T, Error> / async fn -> Result<(), Error>
Async method suffix omit it when the Rust method is async; document the C# source name
CancellationToken tokio_util::sync::CancellationToken at long-lived operation boundaries
nullable reference Option<T>; never use a sentinel UUID/string unless the protocol defines one
exception hierarchy small non-exhaustive thiserror enums per crate; retain source error and protocol context
public mutable property private field plus validated getter/setter, or a public field only for inert value records
overloads one method with typed parameter structs; use separate descriptive names when operations differ semantically
interface object-safe trait when dynamic dispatch is required, otherwise a generic trait bound
delegate Fn, FnMut, or FnOnce bounds; boxed callback only when callbacks are stored
.NET event typed subscription returning a guard, or a tokio::sync::broadcast receiver for asynchronous fan-out
IDisposable ownership and Drop; add explicit close/shutdown().await when failure or async work must be observed
byte[] borrowed &[u8] for parsing, bytes::Bytes for shared network payloads, Vec<u8> for owned mutation
IReadOnlyList<T> &[T] or Arc<[T]>; return iterators when callers do not need storage
Dictionary HashMap; use BTreeMap only when deterministic ordering is observable
immutable collections owned Vec/HashMap or Arc<[T]>; do not add a persistent-collection crate without a measured need
monitor/reader-writer lock std::sync first; use parking_lot only after profiling or poisoning-policy review
generated flags enum bitflags; preserve unknown bits with from_bits_retain
C# enum on the wire #[repr(...)] plus checked conversion; include Unknown(raw) when the protocol is extensible
UUID a compatibility newtype around uuid::Uuid, with explicit big/little-endian wire helpers
vectors/quaternions/matrices compatibility structs with LibreMetaverse field/order semantics; use glam internally after parity tests prove conversions
dependency injection container normal constructors and explicit trait parameters; add no service-locator equivalent

The checked-in mapping ledgers make those rules concrete:

  • properties become getter/setter methods; indexers become item/set_item methods with their index parameters;
  • events become typed subscriptions returning a guard; stored delegates become Fn callbacks, while CLR BeginInvoke/EndInvoke rows are retained as intentional differences;
  • in borrows immutably, ref borrows mutably, and out is a mutable output; optional parameters use Option<T> rather than Rust default arguments;
  • nullable values use Option<T> at the exact metadata node, including nested generic arguments;
  • inheritance maps to traits and composition rather than class inheritance; generic parameters retain their names and cataloged constraints as explicit Rust bounds recorded in the mapping ledgers;
  • operators receive stable trait-style names; overloads receive parameter-based descriptive suffixes and a stable ID suffix only when necessary;
  • [Flags] enums use bitflags with unknown bits retained; wire enums use their exact width/discriminants and an Unknown(raw) form only when extensible;
  • all third-party types exposed by C# signatures map through project-owned cross-platform boundary types. No mapping may name a macOS-only API unless equivalent Linux and Windows implementations are provided.

Run python3 tools/generate_rust_mapping.py --check after any catalog or mapping rule change. It regenerates in memory and rejects duplicate source IDs or Rust destinations, stale/missing members, unresolved referenced types, invalid statuses, missing assembly representatives, and platform-specific targets.

Public structs and enums that come from the protocol must be #[non_exhaustive] when servers may add values. Do not derive Serialize/Deserialize as a substitute for the specified wire encoding: LLUDP, LLSD, MessagePack, asset formats, and login XML each require their own compatibility tests.

3.1 Full signature-shim gate

The first handover task after this structural shell is a complete callable API shim. Do not start real behavior while completing it.

Build the authoritative API catalog during development from compiled .NET metadata after running the original source generators, not from the line- oriented regex ledger alone. This extraction is an inventory tool only and must not become a build-time or runtime dependency of the Rust crates. Build one supported target framework into a temporary output directory and use System.Reflection.Metadata or Roslyn symbols to emit machine-readable records for every public/nested-public:

  • class, struct, record, interface, enum and delegate, including generic parameters and constraints;
  • constructor and method, including overloads, ref/out/in parameters, optional values and return type;
  • property/indexer and getter/setter visibility;
  • field and constant, including value and numeric width;
  • event and delegate payload;
  • base type, implemented interface and relevant public attribute.

api/public-api.json is that catalog. Its external_types section records each non-LibreMetaverse type used by a public signature, its defining assembly, and every owning LibreMetaverse documentation ID; it intentionally does not copy the external assembly's member surface. Create api/RUST-MAPPING.tsv with one row per C# member and resolve every external entry to Rust core/std, an adopted crate, or a native MetaCrate replacement: stable C# documentation ID, C# signature, Rust crate/path, Rust signature, mapping decision, and status. A coverage checker must fail for missing members, duplicate Rust destinations, unresolved external types, or stale source IDs. api/SURFACE.tsv remains useful for source navigation but is not the coverage authority.

Each mapped Rust item must actually compile and be callable:

  • methods, constructors and property accessors have their final reviewed Rust names, parameters, ownership/borrowing, asyncness, and return types;
  • overloads use stable descriptive Rust names or typed argument structs, with the mapping recorded explicitly;
  • enums contain their real variants/discriminants and flags contain their real bits, because tests and downstream code must compile against them;
  • event payloads, delegates and interfaces have their real fields/trait methods;
  • generic bounds and thread-safety guarantees are deliberate, not erased to an untyped placeholder.

The body is the only part allowed to be fake. Fallible Rust APIs return a shared typed Error::NotImplemented(NotImplemented { csharp_member }) error. Infallible constructors, accessors, operators and trait methods call a shared unimplemented_api! placeholder that panics with the stable C# member ID. Constants and enum values must have their real values. Never return plausible defaults such as false, zero, an empty collection, or a nil UUID: those could make translated tests pass for the wrong reason.

The failure contract lives in libremetaverse-types and every API crate exposes it as crate::Error:

  • synchronous fallible members return Err(Error::NotImplemented(NotImplemented::new(CSHARP_ID))); async members do the same when first polled and start no work first;
  • a stream-producing operation returns the typed error before exposing a stream; an intrinsically infallible stream factory uses unimplemented_api! instead of returning an empty stream;
  • callback registration and stored-trait construction fail before retaining the callback/object; generated callbacks are never silent no-ops;
  • infallible constructors, accessors, operators, and trait methods call unimplemented_api!(CSHARP_ID), whose panic marker is distinct from every typed domain error;
  • constants, enum discriminants, flags, and inert public fields use their real catalog values/layout. Generated shims do not derive Default unless the reviewed API explicitly defines that behavior.

python3 tools/generate_rust_mapping.py --check scans every generated shim and rejects Default derives, plausible false/zero/empty/nil returns, and any function body that lacks the standardized typed error or panic mechanism.

The signature gate is complete only when:

  • the metadata catalog reports 100% public type/member mapping coverage for all 13 library assemblies, including generated APIs;
  • all crates build and cargo doc resolves every public signature;
  • a downstream compile fixture can name/call every mapped Rust item;
  • there are no erased ShimValue, dynamic argument bags, invented variadics, or member signatures present only as ledger text;
  • implementation bodies contain only the standardized failure placeholder, constants, enum discriminants, and inert data-layout boilerplate.

3.2 Full semantic test-suite gate

After the full signature shim compiles, translate the entire C# test suite while leaving all library bodies unimplemented. Every one of the 1,289 NUnit invocations must become a Rust test with the same inputs, setup, operation, observations, assertions, tolerances and expected error/event behavior. Shared C# test helpers become shared Rust test helpers; embedded fixtures and literal payloads are copied with license/source attribution and byte hashes.

The parity harness is in place: every invocation has a stable source/case ID, parameter identity, body hash, category, fixture dependency list, Rust location, and semantic-review status. Reviewed tests are identified by parity-case markers in hand-written Rust files, so regeneration records unresolved cases without creating fake executable tests and fails on body drift. The checked-in audit reports pending, translated, ignored-live, benchmark, drifted, missing, duplicate, stale, and unreviewed cases. The initial handover contained 1,289 unreviewed cases; the current ledger contains 149 translated, three reviewed live-grid cases, four benchmark-reviewed, and 1,133 pending cases.

The Rust tests must call the public APIs rather than internal replacements. Where the C# tests call internal members through friend-assembly access, record that fact and place equivalent Rust unit tests inside the owning crate without making the member public solely for testing.

The tests must compile even though running them is red. An expected C# exception maps to a specific Rust Err variant, never to should_panic around a public API call; this ensures an unimplemented_api! panic remains a failure rather than accidentally satisfying the test. Only tests whose C# purpose is genuinely to verify a thrown runtime invariant may use catch_unwind/should_panic, and they must distinguish the intended invariant from the placeholder panic.

The test-suite gate is complete only when:

  • no generated placeholder or dispatcher body can count as reviewed;
  • the parity catalog reports exactly one reviewed Rust case for every upstream [Test]/[TestCase], with the matching C# body hash;
  • cargo test --workspace --no-run compiles all tests against public signatures;
  • a controlled test run fails at standardized unimplemented member IDs rather than from missing symbols, compile errors, malformed fixtures, or weakened assertions;
  • live-server/device tests are faithfully translated and are conditionally ignored only while their documented credentials are absent;
  • no production method contains real behavior beyond what is required to make signatures and constants compile.

4. Validated Rust dependency map

Versions below were queried from crates.io on 2026-08-08 with cargo search and, for risky/native crates, cargo info. They are candidates, not blanket dependencies. Add one to a crate only when its first implementation uses it, then commit Cargo.lock for application/test reproducibility. Recheck versions, MSRV, features, and licenses at adoption time.

.NET/NuGet responsibility Rust candidate and validated version Adoption notes
async runtime, channels, timers tokio 1.53.1, tokio-util 0.7.19, futures 0.3.33 One Tokio runtime owned by the application; libraries never create nested runtimes.
System.Net.Http / WinHTTP reqwest 0.13.4 MSRV 1.85. Use rustls/default TLS deliberately, connection pooling, streaming bodies, and injected client/timeouts.
JSON and general data mapping serde 1.0.229, serde_json 1.0.151, serde_bytes 0.11.19 Use only where the C# implementation is schema-driven JSON. Hand-write LLSD encodings.
MessagePack 3.1.8 rmp-serde 1.3.1, rmpv 1.3.1 Golden byte vectors must prove integer widths, maps, extensions, and field ordering.
UUID uuid 1.24.0 Wrap it; do not expose dependency-specific endian assumptions as protocol behavior.
byte buffers bytes 1.12.1, bytemuck 1.25.2 Prefer explicit endian reads. bytemuck is only for proven POD layouts, never untrusted variable packets.
vector/matrix math glam 0.33.3 Keep compatibility newtypes to control component order, precision, equality, and serialization.
flags bitflags 2.13.1 Retain unknown bits from newer grids.
errors thiserror 2.0.20 Application binaries may use richer reporting; public libraries return typed errors.
ZLogger / Microsoft logging tracing 0.1.44, tracing-subscriber 0.3.23 Libraries emit spans/events; binaries choose formatting and filters. Never log credentials or capability tokens.
rate limiting governor 0.10.4 Candidate for caps categories. First reproduce burst/refill/cancellation behavior with deterministic clock tests.
compression and tar archives flate2 1.1.9, tar 0.4.46 Enforce decompressed-size, path traversal, and entry-count limits on untrusted OAR data.
XML and URL quick-xml 0.41.0, url 2.5.8, base64 0.23.1 Streaming XML for LLSD/login; retain exact URL escaping behavior with fixtures.
CoreJ2K 2.3.3.91 jpeg2k 0.10.1 Defaults to OpenJPEG/native bindings. Keep behind a codec trait and feature; test channel order, alpha, dimensions, discard levels, and malformed input.
SkiaSharp 4.150.1 skia-safe 0.99.0 MSRV 1.85 and native/binary-cache build. Optional adapter only; core image APIs must not leak Skia types.
Pfim 0.11.4 image 0.25.10, ddsfile 0.6.0 image covers TGA and common DDS decoding; ddsfile exposes DDS container details. Golden files decide whether both are needed.
OggVorbisEncoder 1.2.2 vorbis_rs 0.5.6 BSD-3-Clause, MSRV 1.82, backed by C libraries. Feature-gate native audio encoding.
SIPSorcery 8.0.23 webrtc 0.20.0, cpal 0.18.1, opus 0.3.1 Validate SDP, ICE, data-channel framing, audio formats, device hotplug, and native libopus deployment separately. Do not claim parity from successful compilation.
LSL generated parser lalrpop 0.23.1 Candidate only. Preserve grammar conflicts, recovery, token positions, and diagnostics before replacing the generated parser.
NUnit/Moq built-in test harness, mockall 0.15.0, proptest 1.11.0 Prefer fakes and deterministic protocol fixtures; use mocks only for interaction contracts. Add properties after direct parity cases exist.
NUnit benchmarks criterion 0.8.2 Port benchmark-category methods to benches/; never make timing thresholds correctness tests.
sample command lines clap 4.6.6 Add when implementing the first real program CLI; keep credentials in arguments/env only long enough to authenticate.

System.*, BCL compatibility packages, immutable collections, DI abstractions, and object pooling usually map to the Rust standard library and ownership. Do not select a crate merely because NuGet used one. In particular, begin with Mutex<HashMap<...>> rather than a concurrent-map dependency and add a pool only after allocations appear in a profile.

5. Module-by-module implementation guidance

5.1 libremetaverse-types

This is the first implementation crate because nearly every protocol and test depends on it.

  • UUID: wrap uuid::Uuid; implement zero/random/parse/format, byte-array constructors, CRC/combine helpers, ordering, hashing, and every endian path from UUID.cs. Keep the UUID name as a public compatibility type.
  • Vector2/3/3d/4, Quaternion, Matrix4, Ray, Color4: preserve constructors, constants, component order, normalization of zero values, quaternion multiplication direction, approximate-comparison helpers, and binary/string formats. Use glam internally only after conversion tests pass.
  • Enums and primitive enums: translate exact numeric discriminants and flag widths. Use bitflags where C# uses [Flags]; retain unknown values/bits.
  • CRC32 and conversions: port byte-for-byte with fixed golden vectors and explicit endianness. Avoid platform-sized integers on the wire.
  • CacheDictionary, ExpiringCache, TokenBucket: inject a clock in tests; preserve expiry/burst semantics. Start with HashMap plus one lock.
  • DoubleDictionary and MultiValueDictionary: expose iterators and borrowing rather than cloning collections. Preserve duplicate and replacement rules.

Completion gate: every Types-related parity test is a semantic Rust test, all format/byte fixtures match C#, and no generated shim remains for a Types API.

5.2 libremetaverse-structured-data

  • OSD value model: implement a non-exhaustive enum for undefined, boolean, integer, real, string, UUID, date, URI, binary, array, and map. Preserve C# conversion/default behavior deliberately; do not let serde_json::Value define the public model.
  • OSDArray/OSDMap: wrap Vec<OSD> and an order-appropriate map. Verify whether serialized map ordering is observable in each format before choosing HashMap versus BTreeMap/insertion order.
  • Binary LLSD: streaming parser over &[u8]/Read; checked lengths, recursion limits, exact endian rules, and no panics on malformed input.
  • Notation LLSD: preserve escaping, length-prefixed strings/binary, numbers, dates, URIs, whitespace, and error positions.
  • XML LLSD: use quick-xml; reject/entity-limit hostile input and retain exact element mapping.
  • JSON LLSD: use serde_json behind explicit OSD conversions so UUID/date/ URI/binary tagging remains compatible.
  • Protobuf OSD: first identify whether this is a stable external format. If stable, preserve tags with generated Rust types; otherwise keep it internal.

Completion gate: parse and emit C# golden bytes/text for every format, including malformed, deep, empty, Unicode, NaN/infinity, and cross-format cases.

5.3 libremetaverse-imaging

  • Translate ManagedImage as an owned image buffer with explicit dimensions, channel flags/layout, stride, and checked size arithmetic.
  • Translate ITextureCodec to an object-safe TextureCodec trait only if runtime codec selection is needed. Codec methods return typed decode/encode errors and accept limits.
  • Keep public APIs independent of Skia/OpenJPEG types so pure-Rust or platform codecs can be substituted.

Completion gate: channel interleave/deinterleave, resize, alpha/bump behavior, and invalid-dimension tests match the original.

5.4 libremetaverse-imaging-skia

Implement SkiaTextureCodec as an optional adapter with skia-safe. Convert at the boundary to/from ManagedImage; never expose skia_safe types in the core trait. CI must cover macOS, Linux, and Windows or explicitly document unsupported targets. Validate native binary provenance and packaging.

5.5 libremetaverse-prim-mesher

  • Port Coord, Quat, faces, paths, profiles, viewer faces, and indexing as value types with checked indices.
  • Port linear/circular extrusion before sculpt meshing. Preserve winding, normals, UVs, face numbering, hollow/profile cuts, twists, tapers, and shears.
  • Port SculptMap decoding through libremetaverse-imaging, then SculptMesh.
  • Make OBJ export a formatting layer over mesh data; deterministic output is a testable contract.

Completion gate: all 48 PrimMesher rendering-test invocations and all MeshFoundry tests are semantic ports, face indices stay in range, and C# versus Rust mesh fixtures compare within the original tolerances.

5.6 libremetaverse protocol and client core

Implement the central crate in dependency slices, not file order.

Wire protocol and generated packets

  • Port BitPack, zero coding, packet headers, ACK handling, sequence numbers, fragmentation, throttles, and UDPPacketBuffer first.
  • Replace the six Roslyn source generators with a Rust codegen tool/build step reading the same data/ and linden/ inputs. Commit generated packet, visual-parameter, skeleton, tree, genepool, and attention Rust sources.
  • The generator must produce deterministic output; CI regenerates to a temporary directory and fails on diff. Generated packet tests compare exact bytes and IDs against C# fixtures.
  • Parsers operate on borrowed slices where practical and reject truncated, oversized, or unknown blocks without unchecked indexing.

Networking and capabilities

  • UDPBase, NetworkManager, Simulator, ProtocolManager, and UdpThrottle use Tokio UDP tasks with explicit ownership and cancellation. One task owns socket writes; decoded events cross bounded channels to prevent unbounded memory growth.
  • Caps, HttpCapsClient, event queue, caps-to-packet, and rate limiting use an injected reqwest::Client. Preserve redirects, content types, retry policy, cancellation, progress reporting, and non-HTTP location rejection.
  • Login/XML-RPC, seed capabilities, region crossing, reconnect, logout, and timeout state transitions must be modeled explicitly and tested with local fake servers before live-grid use.
  • Capability URLs/tokens and login credentials are secrets: redact them from Debug and logs.

Grid client and service composition

  • GridClient owns managers and shared runtime state. Construct it with a GridClientBuilder only for real optional policies/codecs; required services remain constructor arguments.
  • Replace IGridClient and service-collection extensions with focused traits at test seams. Avoid a general service locator.
  • Settings/* become typed configuration structs grouped like the C# settings, with validated durations/sizes and defaults proven by tests.
  • Logger maps to tracing; UtilizationStatistics exports snapshots without making a metrics backend mandatory.

Agent, movement, chat, effects, teleport and money

  • Port AgentManager partial files as modules under agent, but expose one coherent AgentManager API. Keep camera/movement state separate from network commands.
  • Event-argument classes become event structs. Subscriptions must be removable and must not retain the whole client after drop.
  • Teleport and region crossing are explicit state machines with timeout, cancellation, progress, failure reason, and simulator handoff tests.
  • Money and permissions paths use fixed integer widths and never log sensitive transaction descriptions without opt-in.

Inventory

  • Port inventory enums/base records/nodes/store before InventoryManager. Preserve link resolution, folder/item distinctions, ownership and permissions.
  • Separate local tree/cache mutation from UDP/capability transport. This makes InventoryAISClient independently testable with recorded LLSD fixtures.
  • Async fetch, give, rez, task inventory, current outfit folder, and callbacks must have cancellation and deduplication tests. Do not hold locks across .await.
  • Persisted cache format needs versioning, atomic replacement, corruption handling, and bounded deserialization.

Assets, appearance, avatar and animesh

  • Port the base Asset model and individual asset types as typed wrappers over validated bytes. Decode lazily where the C# behavior permits.
  • AssetManager, AssetCache, DownloadManager, and TexturePipeline share a deduplicating request layer with bounded concurrency and cancellation.
  • Port wearables, baking, current-outfit policies, texture compositing, avatar definitions, skeleton, visual params, animations, gestures, and animesh in that order. Copy and retain the upstream Linden assets only after reviewing their separate cc-by-sa-3.0.txt obligations.
  • Animation interpolation, joint transforms, skinning, attachment rigs, and physics must use the original test tolerances and matrix multiplication order.

Objects, parcels, terrain, environment and rendering model

  • Port Primitive, texture entry, media, particles, permissions, materials, and simulator collections before ObjectManager.
  • Object updates need an explicit decode/apply split; preserve terse/compressed/ full update semantics, stale-update behavior, and parent-child linking.
  • Parcel, estate, grid, terrain, sound, environment, interest-list, directory, friends, groups, marketplace, and experience managers each get a transport- independent state/model layer plus network handlers.
  • Terrain codecs and GLTF/material handling use golden fixtures and size limits. GLTF JSON may use Serde, but binary buffers/accessors require checked offset arithmetic.

Archives and import/export

  • OarFile, tar reader/writer, region settings, and asset archiver use tar and flate2 with path normalization, extraction-root enforcement, and size/count limits.
  • Collada/OBJ/GLTF import/export keeps format DTOs separate from world objects. Deterministic output and round-trip fixtures are required; live upload is a later layer.

Threading and utilities

  • Replace custom events/semaphores/read-write locks with Tokio or std::sync primitives where semantics match. Port custom optimistic/spin locks only if a benchmark and correctness test show they are still required.
  • Replace array pools with Vec/Bytes reuse first. Add pooling only after profiling and with a bounded pool.
  • Observable/event dictionaries expose change streams without executing user callbacks while holding internal locks.

Completion gate for the core crate: no public core shim remains; all offline parity tests pass; malformed-input fuzz targets do not panic; fake-server login, caps, UDP, reconnect, and cancellation scenarios pass on all supported OSes.

5.7 Rendering crates

libremetaverse-rendering-simple ports SimpleRenderer first as the reference implementation of the core rendering trait. It should favor clarity and serve as a fixture generator.

libremetaverse-rendering-mesh-foundry then ports the complete MeshFoundry pipeline using types, OSD, core assets, PrimMesher, and imaging. Preserve face grouping, materials, transforms, skin weights, normals/tangents, texture coordinates, and output ordering. Do not merge the two crates: users should be able to choose the small renderer without the full mesh pipeline.

5.8 libremetaverse-lsl-tools

  • Treat Tools/ as the parser-generator implementation and YYClass/ as generated grammar output. Do not line-by-line port all generated classes.
  • First translate lexer tokens, source locations, errors, comments, precedence, grammar productions, and recovery behavior into compatibility tests.
  • Re-express the grammar in LALRPOP only if it reproduces accepted/rejected programs and diagnostic locations. Otherwise port the existing table machine with generated Rust tables.
  • Generated tables are committed and deterministically reproducible. Keep public token/parser entry points recognizable for migration, while Rust iterators and Result replace C# enumerators/exceptions.

Completion gate: all grammar fixtures and error-recovery cases match, and the generated code is warning-free without 150 hand-maintained class shims.

5.9 libremetaverse-rlv

  • Port message/command parsing and enums first, preserving case, separators, attachment point aliases, option handling, and invalid-command errors.
  • Port blacklist/provider/callback interfaces as narrow traits. Return typed actions from pure parsing/decision code; execute viewer actions through an injected callback adapter.
  • Port restriction state, camera settings, shared folders, inventory maps, locked folders, attachment requests, permissions, and service orchestration.
  • Use immutable snapshots or short locks for restriction reads; callbacks run after locks are released.

Completion gate: every RLV command/query/restriction/exception test case is a semantic Rust test and all original expected error distinctions remain visible.

5.10 libremetaverse-utilities

Port the convenience functions only after their owning lower-level APIs exist. Prefer free functions or extension traits with explicit client references. Do not use this crate to bypass ownership, cancellation, or error handling in the core crate.

5.11 Voice crates

For libremetaverse-voice-vivox:

  • port XML/control definitions, account/session/participant models, TCP pipe, connector, gateway, and manager as a protocol adapter;
  • keep blocking compatibility APIs as thin wrappers in the program layer, not library methods that create runtimes;
  • Vivox SDK/server availability and licensing are external prerequisites. The crate can be protocol-complete without shipping proprietary native binaries.

For libremetaverse-voice-webrtc:

  • port signaling messages and data-channel framing before peer/media handling;
  • adapt webrtc, cpal, and opus behind local traits so protocol tests do not require audio hardware;
  • make device selection, sample format/rate/channel conversion, mute, reconnect, ICE/SDP negotiation, and teardown explicit state transitions;
  • CI uses virtual/fake audio; real-device and live voice validation is a manual gate on macOS, Linux, and Windows.

Completion gate: recorded signaling/data-channel fixtures pass offline, fake audio round-trips have bounded latency/loss behavior, teardown leaves no tasks, and opt-in live validation succeeds without logging tokens.

6. Test-first migration workflow

The compatibility suite is deliberately red in the shell stage. First finish the complete signature gate in section 3.1 across all crates. Then translate the complete suite in section 3.2 across all crates. Only after both global gates may implementation proceed in dependency order.

For each pending catalog case:

  1. Open the exact C# source/line and verify its method-body SHA-256 still matches tests/upstream-tests.json.
  2. Identify setup, action, assertions, parameterized cases, fixtures, categories, and intended exception/event/timeout behavior.
  3. Write an explicit Rust #[test] body in a hand-written test file, add its parity-case marker, and call the mapped library API directly.
  4. Port literal bytes, UUIDs, timestamps, tolerances, culture assumptions, and ordering exactly. Do not weaken an assertion merely because the shim differs.
  5. If the needed API is absent, stop and treat that as a signature-coverage defect: add the missing catalog/mapping entry and final failing signature, then rerun the 100% API coverage gate.
  6. Keep the test red at the standardized unimplemented member boundary; do not implement behavior during the test-translation stage.
  7. Mark the JSON entry with a future status: semantic field only after review; enhance the generator/checker to preserve those reviewed entries rather than overwrite them.

Test translation rules:

  • NUnit [TestCase] invocations remain separate Rust tests so failures identify the exact case.
  • Assert.Throws maps to matching a specific Rust error variant, not any error.
  • floating assertions retain the original tolerance and NaN behavior.
  • async tests use paused/injected time where possible; no arbitrary sleeps.
  • Moq interaction tests prefer a small fake recording calls; use Mockall only when a trait contract has many independent expectations.
  • RequiresLiveServer tests use GRID_USER, GRID_PASSWORD, and GRID_LOGIN_URL from the process environment or the ignored workspace .env; they are ignored when any value is absent and run automatically when all three are present. .env.example documents the non-secret shape.
  • benchmark-category methods move to Criterion and are removed from correctness pass/fail counts.
  • malformed wire/asset tests become fuzz seeds after direct parity is passing.

Required gates:

cargo fmt --all -- --check
python3 tools/generate_rust_mapping.py --check
python3 tools/generate_api_shims.py --check
python3 tools/check_api_coverage.py
cargo check --workspace --all-targets
cargo check --locked --manifest-path tests/api-compile/Cargo.toml
cargo test --workspace --no-run
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo doc --workspace --no-deps

During Stage 0, pending cases exist only in the parity catalog. During the full test-suite gate, the catalog's pending count is the explicit remaining-work count and CI prohibits increases. At the end of that gate, tests still fail, but only because callable production signatures report standardized not-implemented member IDs. During implementation, that failure inventory then shrinks as behavior turns the fixed suite green.

7. Program and live-grid validation plan

The programs package contains these binary targets:

Rust binary C# project Port purpose
simple-bot SimpleBot login, IM/chat events, movement and animations
prim-inspector PrimInspector object discovery/properties and transforms
inventory-explorer InventoryExplorer inventory fetch, search, stats and export
packet-dump PacketDump raw/decoded packet subscription and logging
irc-gateway IRCGateway concurrent external chat bridge
test-client TestClient full interactive client and all 128 command targets
vivox-test VivoxTest Vivox account/session/participant validation
webrtc-test WebRtcTest WebRTC signaling, devices and audio validation
osd-inspector OSDInspector offline LLSD inspect/convert/validate/primitive round trip

Port osd-inspector first because it is offline and validates StructuredData. Then port simple-bot, packet-dump, prim-inspector, and inventory-explorer as vertical grid slices. Port TestClient commands by folder (system/login first, then communication, inventory, objects, movement, land, groups, friends, directory, appearance, voice, stats) while keeping commands::TEST_CLIENT_COMMANDS as the completeness inventory. Voice programs come last because they need native/device and service prerequisites.

Live validation must use a dedicated test account and an explicitly supplied grid URI. Secrets are read from environment variables or the OS secret store, never source/config committed to Git. A live smoke run records sanitized:

  • server/grid/version and Rust commit;
  • login and initial simulator/capabilities completion;
  • IM/chat send/receive;
  • movement/teleport/region crossing;
  • inventory fetch and one reversible test-folder operation;
  • object discovery/property request;
  • asset/texture download and decode;
  • clean logout and task/socket teardown.

Operations that spend currency, modify estate/parcel state, upload assets, delete inventory, or affect other users require separate opt-in flags and are not part of the default smoke test.

8. Ordered delivery milestones

  1. Structural inventory shell (current Stage 0): crates, type/declaration ledgers, failing test parity entries, program targets, license, and this plan compile. This is not the callable API or semantic test suite.
  2. Full public API signature shim: extract authoritative compiled metadata, map 100% of public members to reviewed Rust signatures, and implement only standardized failing bodies. All signatures and downstream compile fixtures build before test translation begins.
  3. Full semantic test translation: replace all 1,289 pending cases with exact reviewed Rust equivalents using the public shim. The entire suite compiles and fails only at standardized unimplemented member boundaries.
  4. Types implementation: UUID, math, enums, conversions, collections, cache/token bucket; turn only the already-existing Types tests green.
  5. StructuredData implementation: OSD plus binary/notation/XML/JSON/protobuf compatibility; turn only the already-existing StructuredData tests green.
  6. Imaging and PrimMesher implementation: managed images, codec trait, TGA/J2K adapter, extrusion/sculpt/OBJ and rendering tests.
  7. Wire codegen: deterministic Rust packet/visual/skeleton/tree/genepool/ attention generation and golden packet bytes.
  8. Networking/client lifecycle: UDP, caps, login, simulators, settings, cancellation, reconnect and logout against fake servers.
  9. World vertical slices: agent, inventory, assets/appearance, objects, parcels/terrain/environment, social/directory/groups, marketplace/estate.
  10. Rendering, RLV, LSL tools: implement against their already-translated suites and add public migration docs.
  11. Programs/live grid: offline inspector, four core examples, TestClient, then voice adapters and opt-in live/device validation.
  12. Hardening/release: fuzz untrusted parsers, cross-platform native CI, MSRV, docs/examples, semver/API audit, license/asset audit, performance comparison against the separately run pinned C# implementation, and an artifact/dependency audit proving there is no CLR, .NET assembly, subprocess, RPC, or LibreMetaverse foreign-interface dependency.

Each milestone ends with zero unreviewed generated shims in its owned modules, no weakened parity assertions, formatted/clippy-clean code, public Rust docs that cite the corresponding C# concept, and a refreshed completeness report from the checked-in ledgers.