887 lines
54 KiB
Markdown
887 lines
54 KiB
Markdown
# LibreMetaverse to Rust rewrite plan
|
|
|
|
## 1. Scope and source of truth
|
|
|
|
MetaCrate is a new native Rust implementation of the public API and observable
|
|
behavior of LibreMetaverse. LibreMetaverse is reference material, not a runtime
|
|
dependency or implementation component. The source snapshot used to create this
|
|
shell is:
|
|
|
|
- repository: <https://github.com/cinderblocks/libremetaverse>
|
|
- branch: `master`
|
|
- commit: `2aa70bb68513b39795da5d13c88f31b86e85a3ba`
|
|
- commit date: 2026-08-07
|
|
- local checkout state during inventory: clean
|
|
- license base: `libremetaverse/LICENSE.txt`, BSD 3-Clause
|
|
|
|
Generated inventories are intentionally checked in:
|
|
|
|
- `api/TYPES.tsv` maps every discovered public C# type declaration to a Rust
|
|
crate/module/type and source line.
|
|
- `api/SURFACE.tsv` records every public declaration line in the 13 library
|
|
projects, including a SHA-256 digest.
|
|
- `api/public-api.json` is the authoritative compiled-metadata catalog for all
|
|
13 library assemblies after all six source generators run. It also records
|
|
every external type referenced by a public signature.
|
|
- `api/RUST-TYPES.tsv` resolves all 3,066 public and 142 external signature
|
|
types, plus two support traits referenced by public inheritance metadata.
|
|
- `api/RUST-MAPPING.tsv` records the reviewed Rust destination and complete
|
|
signature decision for all 30,789 public members.
|
|
- `tests/upstream-tests.json` records every NUnit invocation and the SHA-256 of
|
|
its C# method body.
|
|
- `tests/PARITY.md` maps every NUnit invocation to its Rust test name.
|
|
- `programs/upstream-programs.json` records every source file and digest for the
|
|
nine sample/tool projects.
|
|
|
|
Run `python3 tools/generate_surface.py` after deliberately changing the pinned
|
|
upstream snapshot, then run `python3 tools/extract_public_api.py` to rebuild the
|
|
authoritative metadata catalog twice and require byte-identical output. The
|
|
source generator rejects a test count other than 1,289 so upstream drift cannot
|
|
silently remove tests. Update both commit constants and review all ledger diffs
|
|
in the same change. The earlier 1,295 inventory included five commented-out
|
|
`[Test]` attributes and double-counted the parameterless `[Test]` marker on one
|
|
parameterized `[TestCase]` method; those are not NUnit invocations.
|
|
|
|
### Current shell status
|
|
|
|
The current workspace is a structural baseline, not a working client:
|
|
|
|
- all 13 public .NET library projects have corresponding Rust crates;
|
|
- 1,705 source-discovered public C# type declarations produce compiling Rust
|
|
type/trait shims;
|
|
- 12,195 public declaration lines are retained in the API ledger;
|
|
- compiled metadata records 3,066 public types and 30,789 public members with
|
|
matching independent ECMA-335 counts, with all 142 external signature types
|
|
resolved to cross-platform Rust or project-owned boundary types;
|
|
- the completed Types, StructuredData, Imaging, PrimMesher, rendering, main
|
|
wire/data, core runtime/networking, avatar-facing manager, and
|
|
world/social/service manager, RLV, LSL tools, Utilities, Vivox, and WebRTC
|
|
slices expose all 3,066 final Rust-facing types and all 30,789 callable
|
|
members with real fields/constants/enum values and standardized failing
|
|
bodies;
|
|
- all 1,289 real NUnit `[Test]`/`[TestCase]` invocations have stable catalog
|
|
entries; all 1,266 non-live, non-benchmark cases are translated, all 19
|
|
live-grid cases and four benchmarks are reviewed, and none remain pending;
|
|
- all nine sample/tool projects have compiling Rust binary targets;
|
|
- the 128 TestClient command source files are retained as a command inventory.
|
|
|
|
No generated type shells remain pending: all cataloged members are callable,
|
|
and the standalone downstream fixture names or invokes all 3,066 types and
|
|
30,789 members with zero exclusions. `api/SURFACE.tsv` is still an inventory
|
|
aid, not proof of API coverage: it records declaration lines but does not fully
|
|
parse multiline signatures. Likewise, a pending catalog entry is not a
|
|
semantic translation. **The full public API signature shim
|
|
gate is complete; semantic test translation is the next stage.**
|
|
|
|
The avatar-facing slice preserves the C# agent, movement/camera, appearance and
|
|
outfit-policy, inventory/AIS/store, asset/cache, avatar, and animesh concepts.
|
|
Long-running mapped calls retain an explicit cross-platform cancellation token
|
|
and return the shared `Error::NotImplemented(NotImplemented { csharp_member })`
|
|
error; events return a typed subscription guard so callback lifetime is visible
|
|
to Rust callers.
|
|
|
|
The world-facing slice preserves the C# object, parcel, terrain, grid,
|
|
environment, estate, directory, friend, group, sound, interest-list,
|
|
marketplace, and reporting concepts. Server-facing async operations retain
|
|
explicit cancellation and shared typed failures; request/reply events expose
|
|
typed payloads and subscription guards, while catalog constants and flag values
|
|
remain real.
|
|
|
|
The extension slice preserves the C# RLV restrictions, callbacks and services;
|
|
LSL lexer/parser/generator and diagnostic types; Utilities helpers; Vivox
|
|
protocol/session/manager types; and WebRTC signaling/device/session concepts.
|
|
Blocking compatibility surfaces stay explicit, stored callbacks and events use
|
|
typed ownership, and parser/voice bodies fail before performing parsing,
|
|
networking, device access, or proprietary/native work.
|
|
|
|
No functional porting starts until the semantic test-suite gate is complete
|
|
across the entire workspace, rather than one crate at a time:
|
|
|
|
1. every public C# type and member has a reviewed, callable Rust signature with
|
|
an intentionally failing body;
|
|
2. every C# test is translated semantically and compiles while calling that
|
|
Rust API;
|
|
3. only then is behavior implemented, one crate at a time, until those already
|
|
translated tests turn green.
|
|
|
|
### Full-rewrite constraint
|
|
|
|
The final project must be entirely implemented in new Rust code. The following
|
|
are prohibited in production libraries, examples, tools, tests presented as
|
|
Rust behavior, and release artifacts:
|
|
|
|
- loading or hosting the CLR/.NET runtime;
|
|
- P/Invoke, COM, C ABI, UniFFI, generated bindings, or any other foreign-function
|
|
bridge to LibreMetaverse assemblies;
|
|
- invoking `dotnet`, LibreMetaverse programs, or helper services as a subprocess
|
|
to implement Rust API behavior;
|
|
- embedding, redistributing, dynamically loading, or downloading compiled
|
|
LibreMetaverse assemblies;
|
|
- forwarding calls over IPC/RPC to a running C# implementation;
|
|
- retaining a C# fallback path for APIs not yet implemented in Rust;
|
|
- treating an interoperability comparison harness as the implementation.
|
|
|
|
During development, the pinned C# source and compiled metadata may be inspected
|
|
to inventory public signatures, understand algorithms and state transitions,
|
|
and create golden inputs/outputs. The original C# tests and programs may be run
|
|
separately as an oracle to produce or compare fixtures. Those development tools
|
|
must remain outside the Rust runtime graph, must not be required to build or use
|
|
published Rust crates, and must never satisfy a Rust compatibility test by
|
|
executing the C# implementation.
|
|
|
|
Third-party native libraries such as OpenJPEG, Skia, libopus, operating-system
|
|
APIs, and ordinary Rust crates are allowed where documented and license-
|
|
compatible; the prohibition is specifically against reusing LibreMetaverse or
|
|
another port of its implementation behind a foreign interface. All
|
|
LibreMetaverse-specific protocol logic, models, state machines, managers,
|
|
serialization rules, and public behavior must be reimplemented in Rust.
|
|
|
|
The final release gate must verify that published crate metadata and built
|
|
artifacts contain no dependency on `dotnet`, Mono, CLR hosting APIs, C#
|
|
assemblies, or LibreMetaverse bridge libraries. A clean machine with only the
|
|
documented Rust/native prerequisites must be able to build, test, document, and
|
|
run the Rust implementation.
|
|
|
|
## 2. Workspace design
|
|
|
|
| Rust crate | C# project | C# files | C# lines | Public declaration rows | Responsibility |
|
|
|---|---|---:|---:|---:|---|
|
|
| `libremetaverse-types` | `LibreMetaverse.Types` | 22 | 9,292 | 703 | UUIDs, math types, enums, caches and small collections |
|
|
| `libremetaverse-structured-data` | `LibreMetaverse.StructuredData` | 10 | 4,069 | 303 | OSD value model and LLSD encodings |
|
|
| `libremetaverse-imaging` | `LibreMetaverse.Imaging.Abstractions` | 2 | 454 | 18 | codec traits and managed image representation |
|
|
| `libremetaverse-imaging-skia` | `LibreMetaverse.Imaging.Skia` | 1 | 369 | 3 | optional Skia codec adapter |
|
|
| `libremetaverse-prim-mesher` | `LibreMetaverse.PrimMesher` | 5 | 3,477 | 222 | prim/sculpt mesh generation |
|
|
| `libremetaverse` | `LibreMetaverse` | 210 | 138,916 | 8,982 | client, protocol, managers, assets and rendering model |
|
|
| `libremetaverse-rendering-simple` | `LibreMetaverse.Rendering.Simple` | 1 | 161 | 6 | simple renderer |
|
|
| `libremetaverse-rendering-mesh-foundry` | `LibreMetaverse.Rendering.MeshFoundry` | 1 | 852 | 14 | full mesh renderer |
|
|
| `libremetaverse-lsl-tools` | `LibreMetaverse.LslTools` | 175 | 27,856 | 962 | LSL lexer/parser generator and generated grammar |
|
|
| `libremetaverse-rlv` | `LibreMetaverse.RLV` | 32 | 6,296 | 262 | RLV parsing, restrictions, locks and callbacks |
|
|
| `libremetaverse-utilities` | `LibreMetaverse.Utilities` | 1 | 289 | 12 | high-level convenience helpers |
|
|
| `libremetaverse-voice-vivox` | `LibreMetaverse.Voice.Vivox` | 12 | 4,544 | 440 | Vivox control protocol adapter |
|
|
| `libremetaverse-voice-webrtc` | `LibreMetaverse.Voice.WebRTC` | 11 | 7,023 | 268 | WebRTC voice adapter |
|
|
|
|
`libremetaverse` is the umbrella client crate. It re-exports the foundational
|
|
crates under `types`, `structured_data`, and `imaging_abstractions`. Rendering,
|
|
voice, RLV, LSL tools, and native Skia remain separate so users do not pay for
|
|
large or platform-specific dependency trees unless requested.
|
|
|
|
The workspace uses Rust 2024 with Rust 1.97.1. Generated code is committed so
|
|
ordinary consumers do not need Python, .NET, Roslyn, or the upstream checkout.
|
|
|
|
## 3. Public API translation policy
|
|
|
|
Compatibility means the same concepts, wire behavior, state transitions,
|
|
errors, and event payloads. It does not mean mechanically reproducing C# syntax.
|
|
Apply these rules consistently:
|
|
|
|
| C# surface | Rust surface |
|
|
|---|---|
|
|
| `LibreMetaverse.Foo.Bar` | the owning crate plus `foo::Bar`; omit the repeated assembly namespace |
|
|
| PascalCase methods | `snake_case`; add a deprecated PascalCase forwarding method only when migration evidence justifies it |
|
|
| `Task<T>` / `Task` | `async fn -> Result<T, Error>` / `async fn -> Result<(), Error>` |
|
|
| `Async` method suffix | omit it when the Rust method is `async`; document the C# source name |
|
|
| `CancellationToken` | `tokio_util::sync::CancellationToken` at long-lived operation boundaries |
|
|
| nullable reference | `Option<T>`; never use a sentinel UUID/string unless the protocol defines one |
|
|
| exception hierarchy | small non-exhaustive `thiserror` enums per crate; retain source error and protocol context |
|
|
| public mutable property | private field plus validated getter/setter, or a public field only for inert value records |
|
|
| overloads | one method with typed parameter structs; use separate descriptive names when operations differ semantically |
|
|
| interface | object-safe trait when dynamic dispatch is required, otherwise a generic trait bound |
|
|
| delegate | `Fn`, `FnMut`, or `FnOnce` bounds; boxed callback only when callbacks are stored |
|
|
| .NET event | typed subscription returning a guard, or a `tokio::sync::broadcast` receiver for asynchronous fan-out |
|
|
| `IDisposable` | ownership and `Drop`; add explicit `close`/`shutdown().await` when failure or async work must be observed |
|
|
| `byte[]` | borrowed `&[u8]` for parsing, `bytes::Bytes` for shared network payloads, `Vec<u8>` for owned mutation |
|
|
| `IReadOnlyList<T>` | `&[T]` or `Arc<[T]>`; return iterators when callers do not need storage |
|
|
| `Dictionary` | `HashMap`; use `BTreeMap` only when deterministic ordering is observable |
|
|
| immutable collections | owned `Vec`/`HashMap` or `Arc<[T]>`; do not add a persistent-collection crate without a measured need |
|
|
| monitor/reader-writer lock | `std::sync` first; use `parking_lot` only after profiling or poisoning-policy review |
|
|
| generated flags enum | `bitflags`; preserve unknown bits with `from_bits_retain` |
|
|
| C# enum on the wire | `#[repr(...)]` plus checked conversion; include `Unknown(raw)` when the protocol is extensible |
|
|
| `UUID` | a compatibility newtype around `uuid::Uuid`, with explicit big/little-endian wire helpers |
|
|
| vectors/quaternions/matrices | compatibility structs with LibreMetaverse field/order semantics; use `glam` internally after parity tests prove conversions |
|
|
| dependency injection container | normal constructors and explicit trait parameters; add no service-locator equivalent |
|
|
|
|
The checked-in mapping ledgers make those rules concrete:
|
|
|
|
- properties become getter/setter methods; indexers become `item`/`set_item`
|
|
methods with their index parameters;
|
|
- events become typed subscriptions returning a guard; stored delegates become
|
|
`Fn` callbacks, while CLR `BeginInvoke`/`EndInvoke` rows are retained as
|
|
intentional differences;
|
|
- `in` borrows immutably, `ref` borrows mutably, and `out` is a mutable output;
|
|
optional parameters use `Option<T>` rather than Rust default arguments;
|
|
- nullable values use `Option<T>` at the exact metadata node, including nested
|
|
generic arguments;
|
|
- inheritance maps to traits and composition rather than class inheritance;
|
|
generic parameters retain their names and cataloged constraints as explicit
|
|
Rust bounds recorded in the mapping ledgers;
|
|
- operators receive stable trait-style names; overloads receive parameter-based
|
|
descriptive suffixes and a stable ID suffix only when necessary;
|
|
- `[Flags]` enums use `bitflags` with unknown bits retained; wire enums use their
|
|
exact width/discriminants and an `Unknown(raw)` form only when extensible;
|
|
- all third-party types exposed by C# signatures map through project-owned
|
|
cross-platform boundary types. No mapping may name a macOS-only API unless
|
|
equivalent Linux and Windows implementations are provided.
|
|
|
|
Run `python3 tools/generate_rust_mapping.py --check` after any catalog or mapping
|
|
rule change. It regenerates in memory and rejects duplicate source IDs or Rust
|
|
destinations, stale/missing members, unresolved referenced types, invalid
|
|
statuses, missing assembly representatives, and platform-specific targets.
|
|
|
|
Public structs and enums that come from the protocol must be `#[non_exhaustive]`
|
|
when servers may add values. Do not derive `Serialize`/`Deserialize` as a
|
|
substitute for the specified wire encoding: LLUDP, LLSD, MessagePack, asset
|
|
formats, and login XML each require their own compatibility tests.
|
|
|
|
### 3.1 Full signature-shim gate
|
|
|
|
The first handover task after this structural shell is a complete callable API
|
|
shim. Do not start real behavior while completing it.
|
|
|
|
Build the authoritative API catalog during development from compiled .NET
|
|
metadata after running the original source generators, not from the line-
|
|
oriented regex ledger alone. This extraction is an inventory tool only and must
|
|
not become a build-time or runtime dependency of the Rust crates.
|
|
Build one supported target framework into a temporary output directory and use
|
|
`System.Reflection.Metadata` or Roslyn symbols to emit machine-readable records
|
|
for every public/nested-public:
|
|
|
|
- class, struct, record, interface, enum and delegate, including generic
|
|
parameters and constraints;
|
|
- constructor and method, including overloads, ref/out/in parameters, optional
|
|
values and return type;
|
|
- property/indexer and getter/setter visibility;
|
|
- field and constant, including value and numeric width;
|
|
- event and delegate payload;
|
|
- base type, implemented interface and relevant public attribute.
|
|
|
|
`api/public-api.json` is that catalog. Its `external_types` section records each
|
|
non-LibreMetaverse type used by a public signature, its defining assembly, and
|
|
every owning LibreMetaverse documentation ID; it intentionally does not copy
|
|
the external assembly's member surface. Create `api/RUST-MAPPING.tsv` with one
|
|
row per C# member and resolve every external entry to Rust core/std, an adopted
|
|
crate, or a native MetaCrate replacement: stable C# documentation ID, C#
|
|
signature, Rust crate/path, Rust signature, mapping decision, and status. A
|
|
coverage checker must fail for missing members, duplicate Rust destinations,
|
|
unresolved external types, or stale source IDs. `api/SURFACE.tsv` remains useful
|
|
for source navigation but is not the coverage authority.
|
|
|
|
Each mapped Rust item must actually compile and be callable:
|
|
|
|
- methods, constructors and property accessors have their final reviewed Rust
|
|
names, parameters, ownership/borrowing, asyncness, and return types;
|
|
- overloads use stable descriptive Rust names or typed argument structs, with
|
|
the mapping recorded explicitly;
|
|
- enums contain their real variants/discriminants and flags contain their real
|
|
bits, because tests and downstream code must compile against them;
|
|
- event payloads, delegates and interfaces have their real fields/trait methods;
|
|
- generic bounds and thread-safety guarantees are deliberate, not erased to an
|
|
untyped placeholder.
|
|
|
|
The body is the only part allowed to be fake. Fallible Rust APIs return a shared
|
|
typed `Error::NotImplemented(NotImplemented { csharp_member })` error. Infallible constructors,
|
|
accessors, operators and trait methods call a shared `unimplemented_api!`
|
|
placeholder that panics with the stable C# member ID. Constants and enum values
|
|
must have their real values. Never return plausible defaults such as `false`,
|
|
zero, an empty collection, or a nil UUID: those could make translated tests pass
|
|
for the wrong reason.
|
|
|
|
The failure contract lives in `libremetaverse-types` and every API crate exposes
|
|
it as `crate::Error`:
|
|
|
|
- synchronous fallible members return
|
|
`Err(Error::NotImplemented(NotImplemented::new(CSHARP_ID)))`;
|
|
async members do the same when first polled and start no work first;
|
|
- a stream-producing operation returns the typed error before exposing a
|
|
stream; an intrinsically infallible stream factory uses `unimplemented_api!`
|
|
instead of returning an empty stream;
|
|
- callback registration and stored-trait construction fail before retaining the
|
|
callback/object; generated callbacks are never silent no-ops;
|
|
- infallible constructors, accessors, operators, and trait methods call
|
|
`unimplemented_api!(CSHARP_ID)`, whose panic marker is distinct from every
|
|
typed domain error;
|
|
- constants, enum discriminants, flags, and inert public fields use their real
|
|
catalog values/layout. Generated shims do not derive `Default` unless the
|
|
reviewed API explicitly defines that behavior.
|
|
|
|
`python3 tools/generate_rust_mapping.py --check` scans every generated shim and
|
|
rejects `Default` derives, plausible false/zero/empty/nil returns, and any
|
|
function body that lacks the standardized typed error or panic mechanism.
|
|
|
|
The signature gate is complete only when:
|
|
|
|
- the metadata catalog reports 100% public type/member mapping coverage for all
|
|
13 library assemblies, including generated APIs;
|
|
- all crates build and `cargo doc` resolves every public signature;
|
|
- a downstream compile fixture can name/call every mapped Rust item;
|
|
- there are no erased `ShimValue`, dynamic argument bags, invented variadics,
|
|
or member signatures present only as ledger text;
|
|
- implementation bodies contain only the standardized failure placeholder,
|
|
constants, enum discriminants, and inert data-layout boilerplate.
|
|
|
|
### 3.2 Full semantic test-suite gate
|
|
|
|
After the full signature shim compiles, translate the entire C# test suite while
|
|
leaving all library bodies unimplemented. Every one of the 1,289 NUnit
|
|
invocations must become a Rust test with the same inputs, setup, operation,
|
|
observations, assertions, tolerances and expected error/event behavior. Shared
|
|
C# test helpers become shared Rust test helpers; embedded fixtures and literal
|
|
payloads are copied with license/source attribution and byte hashes.
|
|
|
|
The parity harness is in place: every invocation has a stable source/case ID,
|
|
parameter identity, body hash, category, fixture dependency list, Rust location,
|
|
and semantic-review status. Reviewed tests are identified by `parity-case`
|
|
markers in hand-written Rust files, so regeneration records unresolved cases
|
|
without creating fake executable tests and fails on body drift. The checked-in
|
|
audit reports pending, translated, ignored-live, benchmark, drifted, missing,
|
|
duplicate, stale, and unreviewed cases. The initial handover contained 1,289
|
|
unreviewed cases; the current ledger contains all 1,266 ordinary cases as
|
|
translated, 19 reviewed live-grid cases, four benchmark-reviewed cases, and no
|
|
pending or unreviewed cases.
|
|
|
|
The Rust tests must call the public APIs rather than internal replacements.
|
|
Where the C# tests call internal members through friend-assembly access, record
|
|
that fact and place equivalent Rust unit tests inside the owning crate without
|
|
making the member public solely for testing.
|
|
|
|
The tests must compile even though running them is red. An expected C# exception
|
|
maps to a specific Rust `Err` variant, never to `should_panic` around a public
|
|
API call; this ensures an `unimplemented_api!` panic remains a failure rather
|
|
than accidentally satisfying the test. Only tests whose C# purpose is genuinely
|
|
to verify a thrown runtime invariant may use `catch_unwind`/`should_panic`, and
|
|
they must distinguish the intended invariant from the placeholder panic.
|
|
|
|
The test-suite gate is complete only when:
|
|
|
|
- no generated placeholder or dispatcher body can count as reviewed;
|
|
- the parity catalog reports exactly one reviewed Rust case for every upstream
|
|
`[Test]`/`[TestCase]`, with the matching C# body hash;
|
|
- `cargo test --workspace --no-run` compiles all tests against public signatures;
|
|
- a controlled test run fails at standardized unimplemented member IDs rather
|
|
than from missing symbols, compile errors, malformed fixtures, or weakened
|
|
assertions;
|
|
- live-server/device tests are faithfully translated and are conditionally
|
|
ignored only while their documented credentials are absent;
|
|
- no production method contains real behavior beyond what is required to make
|
|
signatures and constants compile.
|
|
|
|
### 3.3 Fixed compatibility baseline
|
|
|
|
Milestone 03 closes against the pinned 1,289-invocation catalog, not the older
|
|
1,295 source-text estimate. The six-case difference is intentional: five
|
|
attributes were commented out and one parameterless `[Test]` marker duplicated
|
|
a parameterized `[TestCase]` method. Reintroducing those entries would create
|
|
tests that NUnit never runs.
|
|
|
|
`tests/red-suite-baseline.json` fixes the machine-checked ledger at 1,266
|
|
ordinary translations, 19 live-grid translations, four benchmarks, and zero
|
|
pending, drifted, duplicate, missing, or unreviewed cases. The historical
|
|
red-suite filename is retained for automation compatibility, but the reviewed
|
|
Rust translations and benchmarks are now required to pass. Run the controlled
|
|
audit with:
|
|
|
|
```sh
|
|
python3 tools/audit_red_suite.py
|
|
```
|
|
|
|
Without the explicit live-grid opt-in, the audit requires all 1,266 ordinary
|
|
translations and four benchmark cases to pass, conditionally ignores exactly
|
|
the 19 live-grid cases, and pins the independently counted support-test total.
|
|
Setting `RUN_LIVE_TESTS=1` with all three live-grid credentials makes those 19
|
|
cases required passes as well. Any ordinary failure, unexpected ignore,
|
|
nonstandard failure, parity-catalog drift, or support-test-count drift fails the
|
|
audit.
|
|
|
|
## 4. Validated Rust dependency map
|
|
|
|
Versions below were queried from crates.io on 2026-08-08 with `cargo search`
|
|
and, for risky/native crates, `cargo info`. They are candidates, not blanket
|
|
dependencies. Add one to a crate only when its first implementation uses it,
|
|
then commit `Cargo.lock` for application/test reproducibility. Recheck versions,
|
|
MSRV, features, and licenses at adoption time.
|
|
|
|
| .NET/NuGet responsibility | Rust candidate and validated version | Adoption notes |
|
|
|---|---|---|
|
|
| async runtime, channels, timers | [`tokio` 1.53.1](https://crates.io/crates/tokio/1.53.1), [`tokio-util` 0.7.19](https://crates.io/crates/tokio-util/0.7.19), [`futures` 0.3.33](https://crates.io/crates/futures/0.3.33) | One Tokio runtime owned by the application; libraries never create nested runtimes. |
|
|
| `System.Net.Http` / WinHTTP | [`reqwest` 0.13.4](https://crates.io/crates/reqwest/0.13.4) | MSRV 1.85. Use rustls/default TLS deliberately, connection pooling, streaming bodies, and injected client/timeouts. |
|
|
| JSON and general data mapping | [`serde` 1.0.229](https://crates.io/crates/serde/1.0.229), [`serde_json` 1.0.151](https://crates.io/crates/serde_json/1.0.151), [`serde_bytes` 0.11.19](https://crates.io/crates/serde_bytes/0.11.19) | Use only where the C# implementation is schema-driven JSON. Hand-write LLSD encodings. |
|
|
| MessagePack 3.1.8 | [`rmp-serde` 1.3.1](https://crates.io/crates/rmp-serde/1.3.1), [`rmpv` 1.3.1](https://crates.io/crates/rmpv/1.3.1) | Golden byte vectors must prove integer widths, maps, extensions, and field ordering. |
|
|
| UUID | [`uuid` 1.24.0](https://crates.io/crates/uuid/1.24.0) | Wrap it; do not expose dependency-specific endian assumptions as protocol behavior. WASM selects UUID's `rng-getrandom` adapter and `getrandom`'s Web Crypto-backed `wasm_js` feature while desktop targets retain operating-system entropy. |
|
|
| byte buffers | [`bytes` 1.12.1](https://crates.io/crates/bytes/1.12.1), [`bytemuck` 1.25.2](https://crates.io/crates/bytemuck/1.25.2) | Prefer explicit endian reads. `bytemuck` is only for proven POD layouts, never untrusted variable packets. |
|
|
| vector/matrix math | [`glam` 0.33.3](https://crates.io/crates/glam/0.33.3) | Keep compatibility newtypes to control component order, precision, equality, and serialization. |
|
|
| flags | [`bitflags` 2.13.1](https://crates.io/crates/bitflags/2.13.1) | Retain unknown bits from newer grids. |
|
|
| errors | [`thiserror` 2.0.20](https://crates.io/crates/thiserror/2.0.20) | Application binaries may use richer reporting; public libraries return typed errors. |
|
|
| ZLogger / Microsoft logging | [`tracing` 0.1.44](https://crates.io/crates/tracing/0.1.44), [`tracing-subscriber` 0.3.23](https://crates.io/crates/tracing-subscriber/0.3.23) | Libraries emit spans/events; binaries choose formatting and filters. Never log credentials or capability tokens. |
|
|
| rate limiting | [`governor` 0.10.4](https://crates.io/crates/governor/0.10.4) | Candidate for caps categories. First reproduce burst/refill/cancellation behavior with deterministic clock tests. |
|
|
| compression and tar archives | [`flate2` 1.1.9](https://crates.io/crates/flate2/1.1.9), [`tar` 0.4.46](https://crates.io/crates/tar/0.4.46) | Enforce decompressed-size, path traversal, and entry-count limits on untrusted OAR data. |
|
|
| release artifact inspection | [`goblin` 0.10.5](https://crates.io/crates/goblin/0.10.5) | Parse ELF, PE, and Mach-O imports, symbols, and linked libraries without executing an artifact or relying on platform-only inspection tools. |
|
|
| XML and URL | [`quick-xml` 0.41.0](https://crates.io/crates/quick-xml/0.41.0), [`url` 2.5.8](https://crates.io/crates/url/2.5.8), [`base64` 0.23.1](https://crates.io/crates/base64/0.23.1) | Streaming XML for LLSD/login; retain exact URL escaping behavior with fixtures. |
|
|
| CoreJ2K 2.3.3.91 | system [OpenJPEG](https://www.openjpeg.org/) 2.5.4+ through the private `libremetaverse-openjpeg` adapter | Opt-in `jpeg2000` feature; checked-in minimal bindings avoid a libclang build dependency. Bounded memory streams and header validation isolate native code. Golden tests cover channel order, alpha, 16-bit precision, dimensions, discard levels, lossless/lossy modes, and malformed input. |
|
|
| CoreJ2K pure-Rust extension | [`j2k`](https://crates.io/crates/j2k) 0.8.1 | Independent opt-in `rust-j2k` feature with no native library. The safe adapter provides J2K/JP2 encode/decode, native component preservation, bounded streams and allocations, reduced resolution, progressive quality layers, strict/permissive truncation, and OpenJPEG interoperability. |
|
|
| SkiaSharp 4.150.1 | [`skia-safe` 0.99.0](https://crates.io/crates/skia-safe/0.99.0) | Adopted behind the opt-in `skia` feature. Target-specific official binary-cache feature sets provide BMP/GIF/ICO/JPEG/PNG/WBMP/WebP decoding on macOS, Linux, and Windows; checked owned buffers keep Skia types out of the core image API. |
|
|
| SkiaSharp pure-Rust extension | [`skia-rs-codec` 0.3.0](https://crates.io/crates/skia-rs-codec/0.3.0) and [`image-webp` 0.2.4](https://crates.io/crates/image-webp/0.2.4) | Independent opt-in `rust-skia` feature with no native codec. Default features stay disabled: pure-Rust PNG/JPEG/GIF plus built-in BMP/ICO/WBMP come from `skia-rs-codec`, and pure-Rust WebP comes from `image-webp` instead of the native `libwebp-sys` path. |
|
|
| Pfim 0.11.4 | [`image` 0.25.10](https://crates.io/crates/image/0.25.10), [`ddsfile` 0.6.0](https://crates.io/crates/ddsfile/0.6.0) | `image` covers TGA and common DDS decoding; `ddsfile` exposes DDS container details. Golden files decide whether both are needed. |
|
|
| OggVorbisEncoder 1.2.2 | [`vorbis_rs` 0.5.6](https://crates.io/crates/vorbis_rs/0.5.6) | BSD-3-Clause, MSRV 1.82, backed by C libraries. Feature-gate native audio encoding. |
|
|
| SIPSorcery 8.0.23 | [`str0m` 0.22.0](https://crates.io/crates/str0m/0.22.0), [`cpal` 0.18.1](https://crates.io/crates/cpal/0.18.1), system libopus 1.3+ through `libremetaverse-opus` | The private safe adapter avoids the unmaintained `audiopus_sys` binding. Validate SDP, ICE, data-channel framing, audio formats, device hotplug, and native libopus deployment separately. Do not claim parity from successful compilation. |
|
|
| LSL generated parser | [`lalrpop` 0.23.1](https://crates.io/crates/lalrpop/0.23.1) | Candidate only. Preserve grammar conflicts, recovery, token positions, and diagnostics before replacing the generated parser. |
|
|
| NUnit/Moq | built-in test harness, [`mockall` 0.15.0](https://crates.io/crates/mockall/0.15.0), [`proptest` 1.11.0](https://crates.io/crates/proptest/1.11.0) | Prefer fakes and deterministic protocol fixtures; use mocks only for interaction contracts. Add properties after direct parity cases exist. |
|
|
| NUnit benchmarks | [`criterion` 0.8.2](https://crates.io/crates/criterion/0.8.2) | Port benchmark-category methods to `benches/`; never make timing thresholds correctness tests. |
|
|
| sample command lines | [`clap` 4.6.6](https://crates.io/crates/clap/4.6.6) | Add when implementing the first real program CLI; keep credentials in arguments/env only long enough to authenticate. |
|
|
|
|
`System.*`, BCL compatibility packages, immutable collections, DI abstractions,
|
|
and object pooling usually map to the Rust standard library and ownership. Do
|
|
not select a crate merely because NuGet used one. In particular, begin with
|
|
`Mutex<HashMap<...>>` rather than a concurrent-map dependency and add a pool
|
|
only after allocations appear in a profile.
|
|
|
|
### 4.1 Adopted direct dependency inventory
|
|
|
|
The candidates above are not the release inventory. The machine-checked
|
|
inventory is [`ci/dependency-policy.json`](ci/dependency-policy.json); each
|
|
locked direct version must appear there with its implemented purpose,
|
|
maintenance state, transitive-cost classification, and native-code status.
|
|
The current adopted set is:
|
|
|
|
| Responsibility | Adopted direct crates |
|
|
|---|---|
|
|
| Async and HTTP | `tokio`, `futures-channel`, `futures-util`, `reqwest` |
|
|
| Wire data and parsing | `base64`, `serde`, `serde_json`, `roxmltree`, `regex`, `unicode-general-category` |
|
|
| Identity, hashing, and platform facts | `uuid`, `getrandom`, `md-5`, `sha1`, `sha2`, `mac_address2`, `os_info` |
|
|
| Archives, images, and audio | `flate2`, `tar`, `bcdec_rs`, `skia-safe`, `skia-rs-codec`, `image-webp`, `hound`, `vorbis_rs`, `str0m`, `cpal` |
|
|
| Code generation, program CLI, and artifact inspection | `syn`, `prettyplease`, `clap`, `goblin` |
|
|
| Build and compatibility-test support | `pkg-config`, `vcpkg`, `stats_alloc` |
|
|
|
|
System libopus is accessed only through the private `libremetaverse-opus`
|
|
safe adapter; it replaces the unmaintained `audiopus`/`audiopus_sys` stack.
|
|
The adapter uses `pkg-config` on Linux, macOS, and Windows GNU, and `vcpkg` on
|
|
Windows MSVC. The dependency audit fails closed for any new direct crate,
|
|
resolved version, duplicate-version set, wildcard, Git source, or registry.
|
|
|
|
## 5. Module-by-module implementation guidance
|
|
|
|
### 5.1 `libremetaverse-types`
|
|
|
|
This is the first implementation crate because nearly every protocol and test
|
|
depends on it.
|
|
|
|
- **UUID:** wrap `uuid::Uuid`; implement zero/random/parse/format, byte-array
|
|
constructors, CRC/combine helpers, ordering, hashing, and every endian path
|
|
from `UUID.cs`. Keep the `UUID` name as a public compatibility type.
|
|
- **Vector2/3/3d/4, Quaternion, Matrix4, Ray, Color4:** preserve constructors,
|
|
constants, component order, normalization of zero values, quaternion
|
|
multiplication direction, approximate-comparison helpers, and binary/string
|
|
formats. Use `glam` internally only after conversion tests pass.
|
|
- **Enums and primitive enums:** translate exact numeric discriminants and flag
|
|
widths. Use `bitflags` where C# uses `[Flags]`; retain unknown values/bits.
|
|
- **CRC32 and conversions:** port byte-for-byte with fixed golden vectors and
|
|
explicit endianness. Avoid platform-sized integers on the wire.
|
|
- **CacheDictionary, ExpiringCache, TokenBucket:** inject a clock in tests;
|
|
preserve expiry/burst semantics. Start with `HashMap` plus one lock.
|
|
- **DoubleDictionary and MultiValueDictionary:** expose iterators and borrowing
|
|
rather than cloning collections. Preserve duplicate and replacement rules.
|
|
|
|
Completion gate: every Types-related parity test is a semantic Rust test, all
|
|
format/byte fixtures match C#, and no generated shim remains for a Types API.
|
|
|
|
### 5.2 `libremetaverse-structured-data`
|
|
|
|
- **OSD value model:** implement a non-exhaustive enum for undefined, boolean,
|
|
integer, real, string, UUID, date, URI, binary, array, and map. Preserve C#
|
|
conversion/default behavior deliberately; do not let `serde_json::Value`
|
|
define the public model.
|
|
- **OSDArray/OSDMap:** wrap `Vec<OSD>` and an order-appropriate map. Verify
|
|
whether serialized map ordering is observable in each format before choosing
|
|
`HashMap` versus `BTreeMap`/insertion order.
|
|
- **Binary LLSD:** streaming parser over `&[u8]`/`Read`; checked lengths,
|
|
recursion limits, exact endian rules, and no panics on malformed input.
|
|
- **Notation LLSD:** preserve escaping, length-prefixed strings/binary, numbers,
|
|
dates, URIs, whitespace, and error positions.
|
|
- **XML LLSD:** use `quick-xml`; reject/entity-limit hostile input and retain
|
|
exact element mapping.
|
|
- **JSON LLSD:** use `serde_json` behind explicit OSD conversions so UUID/date/
|
|
URI/binary tagging remains compatible.
|
|
- **Protobuf OSD:** first identify whether this is a stable external format. If
|
|
stable, preserve tags with generated Rust types; otherwise keep it internal.
|
|
|
|
Completion gate: parse and emit C# golden bytes/text for every format, including
|
|
malformed, deep, empty, Unicode, NaN/infinity, and cross-format cases.
|
|
|
|
### 5.3 `libremetaverse-imaging`
|
|
|
|
- Translate `ManagedImage` as an owned image buffer with explicit dimensions,
|
|
channel flags/layout, stride, and checked size arithmetic.
|
|
- Translate `ITextureCodec` to an object-safe `TextureCodec` trait only if
|
|
runtime codec selection is needed. Codec methods return typed decode/encode
|
|
errors and accept limits.
|
|
- Keep public APIs independent of Skia/OpenJPEG types so pure-Rust or platform
|
|
codecs can be substituted.
|
|
|
|
Completion gate: channel interleave/deinterleave, resize, alpha/bump behavior,
|
|
and invalid-dimension tests match the original.
|
|
|
|
### 5.4 `libremetaverse-imaging-skia`
|
|
|
|
Implement `SkiaTextureCodec` as an optional adapter with `skia-safe`, plus an
|
|
independently selectable `RustSkiaTextureCodec` using `skia-rs-codec` and
|
|
`image-webp`. Convert at the boundary to/from `ManagedImage`; never expose
|
|
backend-specific types in the core trait. CI must cover macOS, Linux, Windows,
|
|
and the Rust-only WASM compile boundary. Validate native binary provenance and
|
|
the locked Rust dependency/license inventory.
|
|
|
|
### 5.5 `libremetaverse-prim-mesher`
|
|
|
|
- Port `Coord`, `Quat`, faces, paths, profiles, viewer faces, and indexing as
|
|
value types with checked indices.
|
|
- Port linear/circular extrusion before sculpt meshing. Preserve winding,
|
|
normals, UVs, face numbering, hollow/profile cuts, twists, tapers, and shears.
|
|
- Port `SculptMap` decoding through `libremetaverse-imaging`, then `SculptMesh`.
|
|
- Make OBJ export a formatting layer over mesh data; deterministic output is a
|
|
testable contract.
|
|
|
|
Completion gate: all 48 PrimMesher rendering-test invocations and all
|
|
MeshFoundry tests are semantic ports, face indices stay in range, and C# versus
|
|
Rust mesh fixtures compare within the original tolerances.
|
|
|
|
### 5.6 `libremetaverse` protocol and client core
|
|
|
|
Implement the central crate in dependency slices, not file order.
|
|
|
|
#### Wire protocol and generated packets
|
|
|
|
- Port `BitPack`, zero coding, packet headers, ACK handling, sequence numbers,
|
|
fragmentation, throttles, and `UDPPacketBuffer` first.
|
|
- Replace the six Roslyn source generators with a Rust `codegen` tool/build step
|
|
reading the same `data/` and `linden/` inputs. Commit generated packet,
|
|
visual-parameter, skeleton, tree, genepool, and attention Rust sources.
|
|
- The generator must produce deterministic output; CI regenerates to a temporary
|
|
directory and fails on diff. Generated packet tests compare exact bytes and
|
|
IDs against C# fixtures.
|
|
- Parsers operate on borrowed slices where practical and reject truncated,
|
|
oversized, or unknown blocks without unchecked indexing.
|
|
|
|
#### Networking and capabilities
|
|
|
|
- `UDPBase`, `NetworkManager`, `Simulator`, `ProtocolManager`, and `UdpThrottle`
|
|
use Tokio UDP tasks with explicit ownership and cancellation. One task owns
|
|
socket writes; decoded events cross bounded channels to prevent unbounded
|
|
memory growth.
|
|
- `Caps`, `HttpCapsClient`, event queue, caps-to-packet, and rate limiting use an
|
|
injected `reqwest::Client`. Preserve redirects, content types, retry policy,
|
|
cancellation, progress reporting, and non-HTTP location rejection.
|
|
- Login/XML-RPC, seed capabilities, region crossing, reconnect, logout, and
|
|
timeout state transitions must be modeled explicitly and tested with local
|
|
fake servers before live-grid use.
|
|
- Capability URLs/tokens and login credentials are secrets: redact them from
|
|
`Debug` and logs.
|
|
|
|
#### Grid client and service composition
|
|
|
|
- `GridClient` owns managers and shared runtime state. Construct it with a
|
|
`GridClientBuilder` only for real optional policies/codecs; required services
|
|
remain constructor arguments.
|
|
- Replace `IGridClient` and service-collection extensions with focused traits at
|
|
test seams. Avoid a general service locator.
|
|
- `Settings/*` become typed configuration structs grouped like the C# settings,
|
|
with validated durations/sizes and defaults proven by tests.
|
|
- `Logger` maps to `tracing`; `UtilizationStatistics` exports snapshots without
|
|
making a metrics backend mandatory.
|
|
|
|
#### Agent, movement, chat, effects, teleport and money
|
|
|
|
- Port `AgentManager` partial files as modules under `agent`, but expose one
|
|
coherent `AgentManager` API. Keep camera/movement state separate from network
|
|
commands.
|
|
- Event-argument classes become event structs. Subscriptions must be removable
|
|
and must not retain the whole client after drop.
|
|
- Teleport and region crossing are explicit state machines with timeout,
|
|
cancellation, progress, failure reason, and simulator handoff tests.
|
|
- Money and permissions paths use fixed integer widths and never log sensitive
|
|
transaction descriptions without opt-in.
|
|
|
|
#### Inventory
|
|
|
|
- Port inventory enums/base records/nodes/store before `InventoryManager`.
|
|
Preserve link resolution, folder/item distinctions, ownership and permissions.
|
|
- Separate local tree/cache mutation from UDP/capability transport. This makes
|
|
`InventoryAISClient` independently testable with recorded LLSD fixtures.
|
|
- Async fetch, give, rez, task inventory, current outfit folder, and callbacks
|
|
must have cancellation and deduplication tests. Do not hold locks across
|
|
`.await`.
|
|
- Persisted cache format needs versioning, atomic replacement, corruption
|
|
handling, and bounded deserialization.
|
|
|
|
#### Assets, appearance, avatar and animesh
|
|
|
|
- Port the base `Asset` model and individual asset types as typed wrappers over
|
|
validated bytes. Decode lazily where the C# behavior permits.
|
|
- `AssetManager`, `AssetCache`, `DownloadManager`, and `TexturePipeline` share a
|
|
deduplicating request layer with bounded concurrency and cancellation.
|
|
- Port wearables, baking, current-outfit policies, texture compositing, avatar
|
|
definitions, skeleton, visual params, animations, gestures, and animesh in
|
|
that order. Copy and retain the upstream Linden assets only after reviewing
|
|
their separate `cc-by-sa-3.0.txt` obligations.
|
|
- Animation interpolation, joint transforms, skinning, attachment rigs, and
|
|
physics must use the original test tolerances and matrix multiplication order.
|
|
|
|
#### Objects, parcels, terrain, environment and rendering model
|
|
|
|
- Port `Primitive`, texture entry, media, particles, permissions, materials,
|
|
and simulator collections before `ObjectManager`.
|
|
- Object updates need an explicit decode/apply split; preserve terse/compressed/
|
|
full update semantics, stale-update behavior, and parent-child linking.
|
|
- Parcel, estate, grid, terrain, sound, environment, interest-list, directory,
|
|
friends, groups, marketplace, and experience managers each get a transport-
|
|
independent state/model layer plus network handlers.
|
|
- Terrain codecs and GLTF/material handling use golden fixtures and size limits.
|
|
GLTF JSON may use Serde, but binary buffers/accessors require checked offset
|
|
arithmetic.
|
|
|
|
#### Archives and import/export
|
|
|
|
- `OarFile`, tar reader/writer, region settings, and asset archiver use `tar` and
|
|
`flate2` with path normalization, extraction-root enforcement, and size/count
|
|
limits.
|
|
- Collada/OBJ/GLTF import/export keeps format DTOs separate from world objects.
|
|
Deterministic output and round-trip fixtures are required; live upload is a
|
|
later layer.
|
|
|
|
#### Threading and utilities
|
|
|
|
- Replace custom events/semaphores/read-write locks with Tokio or `std::sync`
|
|
primitives where semantics match. Port custom optimistic/spin locks only if a
|
|
benchmark and correctness test show they are still required.
|
|
- Replace array pools with `Vec`/`Bytes` reuse first. Add pooling only after
|
|
profiling and with a bounded pool.
|
|
- Observable/event dictionaries expose change streams without executing user
|
|
callbacks while holding internal locks.
|
|
|
|
Completion gate for the core crate: no public core shim remains; all offline
|
|
parity tests pass; malformed-input fuzz targets do not panic; fake-server login,
|
|
caps, UDP, reconnect, and cancellation scenarios pass on all supported OSes.
|
|
|
|
### 5.7 Rendering crates
|
|
|
|
`libremetaverse-rendering-simple` ports `SimpleRenderer` first as the reference
|
|
implementation of the core rendering trait. It should favor clarity and serve
|
|
as a fixture generator.
|
|
|
|
`libremetaverse-rendering-mesh-foundry` then ports the complete MeshFoundry
|
|
pipeline using types, OSD, core assets, PrimMesher, and imaging. Preserve face
|
|
grouping, materials, transforms, skin weights, normals/tangents, texture
|
|
coordinates, and output ordering. Do not merge the two crates: users should be
|
|
able to choose the small renderer without the full mesh pipeline.
|
|
|
|
### 5.8 `libremetaverse-lsl-tools`
|
|
|
|
- Treat `Tools/` as the parser-generator implementation and `YYClass/` as
|
|
generated grammar output. Do not line-by-line port all generated classes.
|
|
- First translate lexer tokens, source locations, errors, comments, precedence,
|
|
grammar productions, and recovery behavior into compatibility tests.
|
|
- Re-express the grammar in LALRPOP only if it reproduces accepted/rejected
|
|
programs and diagnostic locations. Otherwise port the existing table machine
|
|
with generated Rust tables.
|
|
- Generated tables are committed and deterministically reproducible. Keep public
|
|
token/parser entry points recognizable for migration, while Rust iterators and
|
|
`Result` replace C# enumerators/exceptions.
|
|
|
|
Completion gate: all grammar fixtures and error-recovery cases match, and the
|
|
generated code is warning-free without 150 hand-maintained class shims.
|
|
|
|
### 5.9 `libremetaverse-rlv`
|
|
|
|
- Port message/command parsing and enums first, preserving case, separators,
|
|
attachment point aliases, option handling, and invalid-command errors.
|
|
- Port blacklist/provider/callback interfaces as narrow traits. Return typed
|
|
actions from pure parsing/decision code; execute viewer actions through an
|
|
injected callback adapter.
|
|
- Port restriction state, camera settings, shared folders, inventory maps,
|
|
locked folders, attachment requests, permissions, and service orchestration.
|
|
- Use immutable snapshots or short locks for restriction reads; callbacks run
|
|
after locks are released.
|
|
|
|
Completion gate: every RLV command/query/restriction/exception test case is a
|
|
semantic Rust test and all original expected error distinctions remain visible.
|
|
|
|
### 5.10 `libremetaverse-utilities`
|
|
|
|
Port the convenience functions only after their owning lower-level APIs exist.
|
|
Prefer free functions or extension traits with explicit client references. Do
|
|
not use this crate to bypass ownership, cancellation, or error handling in the
|
|
core crate.
|
|
|
|
### 5.11 Voice crates
|
|
|
|
For `libremetaverse-voice-vivox`:
|
|
|
|
- port XML/control definitions, account/session/participant models, TCP pipe,
|
|
connector, gateway, and manager as a protocol adapter;
|
|
- keep blocking compatibility APIs as thin wrappers in the program layer, not
|
|
library methods that create runtimes;
|
|
- Vivox SDK/server availability and licensing are external prerequisites. The
|
|
crate can be protocol-complete without shipping proprietary native binaries.
|
|
|
|
For `libremetaverse-voice-webrtc`:
|
|
|
|
- port signaling messages and data-channel framing before peer/media handling;
|
|
- adapt `webrtc`, `cpal`, and `opus` behind local traits so protocol tests do not
|
|
require audio hardware;
|
|
- make device selection, sample format/rate/channel conversion, mute, reconnect,
|
|
ICE/SDP negotiation, and teardown explicit state transitions;
|
|
- CI uses virtual/fake audio; real-device and live voice validation is a manual
|
|
gate on macOS, Linux, and Windows.
|
|
|
|
Completion gate: recorded signaling/data-channel fixtures pass offline, fake
|
|
audio round-trips have bounded latency/loss behavior, teardown leaves no tasks,
|
|
and opt-in live validation succeeds without logging tokens.
|
|
|
|
## 6. Test-first migration workflow
|
|
|
|
The compatibility suite is deliberately red in the shell stage. First finish
|
|
the complete signature gate in section 3.1 across all crates. Then translate the
|
|
complete suite in section 3.2 across all crates. Only after both global gates
|
|
may implementation proceed in dependency order.
|
|
|
|
For each pending catalog case:
|
|
|
|
1. Open the exact C# source/line and verify its method-body SHA-256 still matches
|
|
`tests/upstream-tests.json`.
|
|
2. Identify setup, action, assertions, parameterized cases, fixtures, categories,
|
|
and intended exception/event/timeout behavior.
|
|
3. Write an explicit Rust `#[test]` body in a hand-written test file, add its
|
|
`parity-case` marker, and call the mapped library API directly.
|
|
4. Port literal bytes, UUIDs, timestamps, tolerances, culture assumptions, and
|
|
ordering exactly. Do not weaken an assertion merely because the shim differs.
|
|
5. If the needed API is absent, stop and treat that as a signature-coverage
|
|
defect: add the missing catalog/mapping entry and final failing signature,
|
|
then rerun the 100% API coverage gate.
|
|
6. Keep the test red at the standardized unimplemented member boundary; do not
|
|
implement behavior during the test-translation stage.
|
|
7. Mark the JSON entry with a future `status: semantic` field only after review;
|
|
enhance the generator/checker to preserve those reviewed entries rather than
|
|
overwrite them.
|
|
|
|
Test translation rules:
|
|
|
|
- NUnit `[TestCase]` invocations remain separate Rust tests so failures identify
|
|
the exact case.
|
|
- `Assert.Throws` maps to matching a specific Rust error variant, not any error.
|
|
- floating assertions retain the original tolerance and NaN behavior.
|
|
- async tests use paused/injected time where possible; no arbitrary sleeps.
|
|
- Moq interaction tests prefer a small fake recording calls; use Mockall only
|
|
when a trait contract has many independent expectations.
|
|
- `RequiresLiveServer` tests use `GRID_USER`, `GRID_PASSWORD`, and
|
|
`GRID_LOGIN_URL` from the process environment or the ignored workspace
|
|
`.env`; they are ignored when any value is absent and run automatically when
|
|
all three are present. `.env.example` documents the non-secret shape.
|
|
- benchmark-category methods move to Criterion and are removed from correctness
|
|
pass/fail counts.
|
|
- malformed wire/asset tests become fuzz seeds after direct parity is passing.
|
|
|
|
Required gates:
|
|
|
|
```sh
|
|
cargo fmt --all -- --check
|
|
python3 tools/generate_rust_mapping.py --check
|
|
python3 tools/generate_api_shims.py --check
|
|
python3 tools/check_api_coverage.py
|
|
cargo check --workspace --all-targets
|
|
cargo check --locked --manifest-path tests/api-compile/Cargo.toml
|
|
cargo test --workspace --no-run
|
|
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
|
cargo doc --workspace --no-deps
|
|
```
|
|
|
|
During Stage 0, pending cases exist only in the parity catalog. During the full
|
|
test-suite gate, the catalog's pending count is the explicit remaining-work
|
|
count and CI prohibits increases. At the end of that gate, tests still fail,
|
|
but only because callable production signatures report standardized
|
|
not-implemented member IDs. During implementation, that failure inventory then
|
|
shrinks as behavior turns the fixed suite green.
|
|
|
|
## 7. Program and live-grid validation plan
|
|
|
|
The `programs` package contains these binary targets:
|
|
|
|
| Rust binary | C# project | Port purpose |
|
|
|---|---|---|
|
|
| `simple-bot` | `SimpleBot` | login, IM/chat events, movement and animations |
|
|
| `prim-inspector` | `PrimInspector` | object discovery/properties and transforms |
|
|
| `inventory-explorer` | `InventoryExplorer` | inventory fetch, search, stats and export |
|
|
| `packet-dump` | `PacketDump` | raw/decoded packet subscription and logging |
|
|
| `irc-gateway` | `IRCGateway` | concurrent external chat bridge |
|
|
| `test-client` | `TestClient` | full interactive client and all 128 command targets |
|
|
| `vivox-test` | `VivoxTest` | Vivox account/session/participant validation |
|
|
| `webrtc-test` | `WebRtcTest` | WebRTC signaling, devices and audio validation |
|
|
| `osd-inspector` | `OSDInspector` | offline LLSD inspect/convert/validate/primitive round trip |
|
|
|
|
Port `osd-inspector` first because it is offline and validates StructuredData.
|
|
Then port `simple-bot`, `packet-dump`, `prim-inspector`, and
|
|
`inventory-explorer` as vertical grid slices. Port TestClient commands by folder
|
|
(system/login first, then communication, inventory, objects, movement, land,
|
|
groups, friends, directory, appearance, voice, stats) while keeping
|
|
`commands::TEST_CLIENT_COMMANDS` as the completeness inventory. Voice programs
|
|
come last because they need native/device and service prerequisites.
|
|
|
|
Live validation must use a dedicated test account and an explicitly supplied
|
|
grid URI. Secrets are read from environment variables or the OS secret store,
|
|
never source/config committed to Git. A live smoke run records sanitized:
|
|
|
|
- server/grid/version and Rust commit;
|
|
- login and initial simulator/capabilities completion;
|
|
- IM/chat send/receive;
|
|
- movement/teleport/region crossing;
|
|
- inventory fetch and one reversible test-folder operation;
|
|
- object discovery/property request;
|
|
- asset/texture download and decode;
|
|
- clean logout and task/socket teardown.
|
|
|
|
Operations that spend currency, modify estate/parcel state, upload assets,
|
|
delete inventory, or affect other users require separate opt-in flags and are
|
|
not part of the default smoke test.
|
|
|
|
## 8. Ordered delivery milestones
|
|
|
|
1. **Structural inventory shell (current Stage 0):** crates, type/declaration
|
|
ledgers, failing test parity entries, program targets, license, and this plan
|
|
compile. This is not the callable API or semantic test suite.
|
|
2. **Full public API signature shim:** extract authoritative compiled metadata,
|
|
map 100% of public members to reviewed Rust signatures, and implement only
|
|
standardized failing bodies. All signatures and downstream compile fixtures
|
|
build before test translation begins.
|
|
3. **Full semantic test translation:** replace all 1,289 pending cases with
|
|
exact reviewed Rust equivalents using the public shim. The entire suite
|
|
compiles and fails only at standardized unimplemented member boundaries.
|
|
4. **Types implementation:** UUID, math, enums, conversions, collections,
|
|
cache/token bucket; turn only the already-existing Types tests green.
|
|
5. **StructuredData implementation:** OSD plus binary/notation/XML/JSON/protobuf
|
|
compatibility; turn only the already-existing StructuredData tests green.
|
|
6. **Imaging and PrimMesher implementation:** managed images, codec trait,
|
|
TGA/J2K adapter,
|
|
extrusion/sculpt/OBJ and rendering tests.
|
|
7. **Wire codegen:** deterministic Rust packet/visual/skeleton/tree/genepool/
|
|
attention generation and golden packet bytes.
|
|
8. **Networking/client lifecycle:** UDP, caps, login, simulators, settings,
|
|
cancellation, reconnect and logout against fake servers.
|
|
9. **World vertical slices:** agent, inventory, assets/appearance, objects,
|
|
parcels/terrain/environment, social/directory/groups, marketplace/estate.
|
|
10. **Rendering, RLV, LSL tools:** implement against their already-translated
|
|
suites and add public migration docs.
|
|
11. **Programs/live grid:** offline inspector, four core examples, TestClient,
|
|
then voice adapters and opt-in live/device validation.
|
|
12. **Hardening/release:** fuzz untrusted parsers, cross-platform native CI,
|
|
docs/examples, semver/API audit, license/asset audit, performance
|
|
comparison against the separately run pinned C# implementation, and an
|
|
artifact/dependency audit proving there is no CLR, .NET assembly, subprocess,
|
|
RPC, or LibreMetaverse foreign-interface dependency.
|
|
|
|
The native release artifact gate packages all 16 release/source crates as one
|
|
versioned set, rebuilds those extracted packages offline without the reference
|
|
checkout, parses every shipped ELF/PE/Mach-O program and its native imports, and
|
|
runs credential-free helper-trapped smoke modes for all ten executables. The
|
|
policy, clean-machine prerequisites, and reproducible commands are documented
|
|
in [`docs/release-artifacts.md`](docs/release-artifacts.md).
|
|
|
|
Each milestone ends with zero unreviewed generated shims in its owned modules,
|
|
no weakened parity assertions, formatted/clippy-clean code, public Rust docs
|
|
that cite the corresponding C# concept, and a refreshed completeness report
|
|
from the checked-in ledgers.
|