Generate Types and StructuredData API shims
This commit is contained in:
@@ -7,9 +7,9 @@ host, bind to, invoke, or ship the .NET implementation. The current
|
||||
stage is a compiling structural shell: crate boundaries mirror the .NET library
|
||||
projects, public C# type names have Rust declarations, every upstream NUnit case
|
||||
has a traceable placeholder, and every sample/tool project has a Rust binary
|
||||
target. Callable member signatures and semantic Rust test translations are the
|
||||
next two required gates; the current generated declarations are not yet the
|
||||
complete public API shim.
|
||||
target. The Types and StructuredData assemblies now have complete callable,
|
||||
failure-only signatures; the remaining assembly slices and semantic Rust test
|
||||
translations are still required before the public API shim is complete.
|
||||
|
||||
The source snapshot, compatibility rules, dependency research, and ordered
|
||||
implementation plan are in [RUSTREWRITE.md](RUSTREWRITE.md).
|
||||
@@ -27,6 +27,7 @@ cannot look like successful behavior.
|
||||
cargo build --workspace
|
||||
cargo test --workspace --no-run
|
||||
python3 tools/generate_rust_mapping.py --check
|
||||
python3 tools/generate_api_shims.py --check
|
||||
```
|
||||
|
||||
Running `cargo test --workspace` is intentionally red during the shim stage.
|
||||
|
||||
@@ -45,23 +45,27 @@ in the same change.
|
||||
The current workspace is a structural baseline, not a working client:
|
||||
|
||||
- all 13 public .NET library projects have corresponding Rust crates;
|
||||
- 1,705 public C# type declarations produce compiling Rust type/trait shims;
|
||||
- 1,705 source-discovered public C# type declarations produce compiling Rust
|
||||
type/trait shims;
|
||||
- 12,195 public declaration lines are retained in the API ledger;
|
||||
- compiled metadata records 3,066 public types and 30,789 public members with
|
||||
matching independent ECMA-335 counts, with all 142 external signature types
|
||||
resolved to cross-platform Rust or project-owned boundary types;
|
||||
- the Types and StructuredData slices expose 61 final Rust-facing types and all
|
||||
1,237 callable members with real fields/constants/enum values and standardized
|
||||
failing bodies;
|
||||
- all 1,295 NUnit `[Test]`/`[TestCase]` invocations have compiling Rust test
|
||||
entries that intentionally panic with their source identity and body hash;
|
||||
- all nine sample/tool projects have compiling Rust binary targets;
|
||||
- the 128 TestClient command source files are retained as a command inventory.
|
||||
|
||||
The generated type shims preserve names and module placement, but they do not
|
||||
yet expose callable C# methods, properties, fields, event payloads, enum values,
|
||||
or delegate signatures. `api/SURFACE.tsv` is an inventory aid, not proof of API
|
||||
coverage: it records declaration lines but does not fully parse multiline
|
||||
signatures. Likewise, a generated `pending` test is a parity placeholder, not a
|
||||
semantic translation. **This repository is therefore at Stage 0 and does not
|
||||
yet satisfy either the full API-shim gate or the full test-suite gate.**
|
||||
The remaining generated type shims preserve names and module placement, but do
|
||||
not yet expose their callable members. `api/SURFACE.tsv` is an inventory aid,
|
||||
not proof of API coverage: it records declaration lines but does not fully parse
|
||||
multiline signatures. Likewise, a generated `pending` test is a parity
|
||||
placeholder, not a semantic translation. **This repository is therefore at
|
||||
Stage 0 and does not yet satisfy either the full API-shim gate or the full
|
||||
test-suite gate.**
|
||||
|
||||
No functional porting starts until both missing gates are complete across the
|
||||
entire workspace, rather than one crate at a time:
|
||||
|
||||
@@ -48,7 +48,7 @@ External member APIs are not copied. Each signature type resolves to Rust core/s
|
||||
- Delegates and flags enums: **40 / 46**
|
||||
- Optional/ref/out/in parameters: **470 / 6,144 / 555 / 0**
|
||||
- Async Rust signatures: **360**
|
||||
- Descriptively renamed overloads: **7,131**
|
||||
- Descriptively renamed overloads: **7,137**
|
||||
- Explicit CLR delegate APM differences: **80**
|
||||
|
||||
Run `python3 tools/generate_rust_mapping.py --check` to regenerate in memory and enforce the complete gate.
|
||||
|
||||
@@ -37,6 +37,8 @@ Regenerate or validate the Rust mapping with only Python's standard library:
|
||||
```sh
|
||||
python3 tools/generate_rust_mapping.py
|
||||
python3 tools/generate_rust_mapping.py --check
|
||||
python3 tools/generate_api_shims.py
|
||||
python3 tools/generate_api_shims.py --check
|
||||
```
|
||||
|
||||
The checker rejects duplicate C# IDs, duplicate Rust destinations, invalid
|
||||
@@ -48,3 +50,9 @@ fallbacks, and generated function bodies that do not use the standardized
|
||||
The two support traits in `RUST-TYPES.tsv` are included because compiled public
|
||||
inheritance records reference them even though they are absent from the public
|
||||
type table.
|
||||
|
||||
`generate_api_shims.py` consumes the catalog and reviewed mapping ledger. It
|
||||
currently owns the completed Types and StructuredData slices, formats their
|
||||
generated Rust deterministically, and rejects any missing or stale type/member
|
||||
documentation ID. `SHIM-COVERAGE.md` distinguishes callable slices from the
|
||||
remaining milestone work; mapping coverage alone is not callable coverage.
|
||||
|
||||
9578
api/RUST-MAPPING.tsv
9578
api/RUST-MAPPING.tsv
File diff suppressed because it is too large
Load Diff
@@ -3092,7 +3092,7 @@ T:System.Action`1 System.Action<T> System.Private.CoreLib delegate core callback
|
||||
T:System.Action`2 System.Action<T1, T2> System.Private.CoreLib delegate core callback Box<dyn Fn(T1, T2) + Send + Sync> language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Action`3 System.Action<T1, T2, T3> System.Private.CoreLib delegate core callback Box<dyn Fn(T1, T2, T3) + Send + Sync> language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.ApplicationException System.ApplicationException System.Private.CoreLib class libremetaverse-types compat libremetaverse_types::compat::ApplicationException native_cross_platform_replacement not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Array System.Array System.Private.CoreLib class std Vec<T> stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Array System.Array System.Private.CoreLib class libremetaverse-types libremetaverse_types::compat::Array stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.AsyncCallback System.AsyncCallback System.Private.CoreLib delegate Box<dyn Fn(&dyn std Box<dyn Fn(&dyn std::any::Any) + Send + Sync> stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Attribute System.Attribute System.Private.CoreLib class libremetaverse-types libremetaverse_types::compat::Attribute stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Boolean System.Boolean System.Private.CoreLib struct core primitive bool language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
@@ -3135,7 +3135,7 @@ T:System.Func`1 System.Func<TResult> System.Private.CoreLib delegate core callba
|
||||
T:System.Func`2 System.Func<T, TResult> System.Private.CoreLib delegate core callback Box<dyn Fn(T) -> TResult + Send + Sync> language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Globalization.CultureInfo System.Globalization.CultureInfo System.Private.CoreLib class libremetaverse-types compat libremetaverse_types::compat::CultureInfo native_cross_platform_replacement not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Globalization.UnicodeCategory System.Globalization.UnicodeCategory System.Private.CoreLib enum libremetaverse-types compat libremetaverse_types::compat::UnicodeCategory native_cross_platform_replacement not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Guid System.Guid System.Private.CoreLib struct uuid uuid::Uuid stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Guid System.Guid System.Private.CoreLib struct libremetaverse-types libremetaverse_types::compat::Guid stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.IAsyncDisposable System.IAsyncDisposable System.Private.CoreLib interface libremetaverse-types compat libremetaverse_types::compat::IAsyncDisposable native_cross_platform_replacement not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.IAsyncResult System.IAsyncResult System.Private.CoreLib interface Box<dyn std Box<dyn std::any::Any + Send + Sync> stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.ICloneable System.ICloneable System.Private.CoreLib interface std Clone stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
@@ -3197,7 +3197,7 @@ T:System.Type System.Type System.Private.CoreLib class libremetaverse-types lib
|
||||
T:System.UInt16 System.UInt16 System.Private.CoreLib struct core primitive u16 language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.UInt32 System.UInt32 System.Private.CoreLib struct core primitive u32 language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.UInt64 System.UInt64 System.Private.CoreLib struct core primitive u64 language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Uri System.Uri System.Private.Uri class url url::Url stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Uri System.Uri System.Private.Uri class libremetaverse-types libremetaverse_types::compat::Uri stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.ValueTuple`2 System.ValueTuple<T1, T2> System.Private.CoreLib struct core tuple (T1, T2) language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.ValueTuple`3 System.ValueTuple<T1, T2, T3> System.Private.CoreLib struct core tuple (T1, T2, T3) language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.ValueTuple`4 System.ValueTuple<T1, T2, T3, T4> System.Private.CoreLib struct core tuple (T1, T2, T3, T4) language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
@@ -3205,7 +3205,7 @@ T:System.ValueTuple`6 System.ValueTuple<T1, T2, T3, T4, T5, T6> System.Private.C
|
||||
T:System.ValueType System.ValueType System.Private.CoreLib class libremetaverse-types libremetaverse_types::compat::Value stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Void System.Void System.Private.CoreLib struct core primitive () language not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Xml.XmlElement System.Xml.XmlElement System.Private.Xml class libremetaverse-structured-data libremetaverse_structured_data::xml::Element stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Xml.XmlReader System.Xml.XmlReader System.Private.Xml class libremetaverse-structured-data libremetaverse_structured_data::xml::Reader<R> stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Xml.XmlReader System.Xml.XmlReader System.Private.Xml class libremetaverse-structured-data libremetaverse_structured_data::xml::Reader stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Xml.XmlTextReader System.Xml.XmlTextReader System.Private.Xml class libremetaverse-types compat libremetaverse_types::compat::XmlTextReader native_cross_platform_replacement not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Xml.XmlTextWriter System.Xml.XmlTextWriter System.Private.Xml class libremetaverse-types compat libremetaverse_types::compat::XmlTextWriter native_cross_platform_replacement not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Xml.XmlWriter System.Xml.XmlWriter System.Private.Xml class libremetaverse-structured-data libremetaverse_structured_data::xml::Writer<W> stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
T:System.Xml.XmlWriter System.Xml.XmlWriter System.Private.Xml class libremetaverse-structured-data libremetaverse_structured_data::xml::Writer stdlib_or_adopted_crate not_applicable not_applicable not_applicable mapped external member surface is not copied
|
||||
|
||||
|
Can't render this file because it is too large.
|
21
api/SHIM-COVERAGE.md
Normal file
21
api/SHIM-COVERAGE.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# Callable shim coverage
|
||||
|
||||
Generated by `python3 tools/generate_api_shims.py`; do not edit by hand.
|
||||
|
||||
| Assembly | Types | Members | Status |
|
||||
|---|---:|---:|---|
|
||||
| `LibreMetaverse` | 2,711 | 27,281 | pending milestone issue |
|
||||
| `LibreMetaverse.Imaging.Abstractions` | 3 | 20 | pending milestone issue |
|
||||
| `LibreMetaverse.Imaging.Skia` | 1 | 3 | pending milestone issue |
|
||||
| `LibreMetaverse.LslTools` | 164 | 768 | pending milestone issue |
|
||||
| `LibreMetaverse.PrimMesher` | 17 | 207 | pending milestone issue |
|
||||
| `LibreMetaverse.RLV` | 28 | 499 | pending milestone issue |
|
||||
| `LibreMetaverse.Rendering.MeshFoundry` | 1 | 14 | pending milestone issue |
|
||||
| `LibreMetaverse.Rendering.Simple` | 1 | 6 | pending milestone issue |
|
||||
| `LibreMetaverse.StructuredData` | 16 | 295 | callable failure-only shim |
|
||||
| `LibreMetaverse.Types` | 45 | 942 | callable failure-only shim |
|
||||
| `LibreMetaverse.Utilities` | 3 | 13 | pending milestone issue |
|
||||
| `LibreMetaverse.Voice.Vivox` | 64 | 531 | pending milestone issue |
|
||||
| `LibreMetaverse.Voice.WebRTC` | 12 | 210 | pending milestone issue |
|
||||
|
||||
Current callable coverage: **61 types / 1,237 members**.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,7 +1,17 @@
|
||||
//! Structured-data types corresponding to `LibreMetaverse.StructuredData`.
|
||||
|
||||
extern crate self as libremetaverse_structured_data;
|
||||
|
||||
mod generated;
|
||||
|
||||
pub mod xml {
|
||||
/// Native XML reader boundary; external XML APIs are not copied.
|
||||
pub struct Reader;
|
||||
|
||||
/// Native XML writer boundary; external XML APIs are not copied.
|
||||
pub struct Writer;
|
||||
}
|
||||
|
||||
pub use generated::*;
|
||||
pub use libremetaverse_types as types;
|
||||
pub use libremetaverse_types::NotImplemented as Error;
|
||||
|
||||
61
crates/libremetaverse-types/src/compat.rs
Normal file
61
crates/libremetaverse-types/src/compat.rs
Normal file
@@ -0,0 +1,61 @@
|
||||
//! Project-owned replacements for external types exposed by C# signatures.
|
||||
//!
|
||||
//! These types preserve Rust API identity without copying external member
|
||||
//! surfaces. Their behavior is implemented only when the owning API slice is
|
||||
//! ported.
|
||||
|
||||
use std::marker::PhantomData;
|
||||
|
||||
pub trait Collection<T> {}
|
||||
|
||||
pub trait ReadWrite: std::io::Read + std::io::Write {}
|
||||
|
||||
impl<T: std::io::Read + std::io::Write> ReadWrite for T {}
|
||||
|
||||
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct Object(u8);
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct Utf16CodeUnit(pub u16);
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct Guid(pub [u8; 16]);
|
||||
|
||||
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct Uri(pub String);
|
||||
|
||||
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct Regex(pub String);
|
||||
|
||||
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct CultureInfo(pub String);
|
||||
|
||||
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct ExternalError(pub String);
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct EnumValue(pub i64);
|
||||
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct ArrayList(pub Vec<Object>);
|
||||
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct Array(pub Vec<Object>);
|
||||
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct Hashtable(pub std::collections::HashMap<Object, Object>);
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct IDictionaryEnumerator;
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct IEqualityComparer<T>(pub PhantomData<fn(T)>);
|
||||
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct StringReader(pub String);
|
||||
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct StringWriter(pub String);
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
|
||||
pub struct TypeId(pub &'static str);
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,8 @@
|
||||
//! Core value types corresponding to `LibreMetaverse.Types`.
|
||||
|
||||
extern crate self as libremetaverse_types;
|
||||
|
||||
pub mod compat;
|
||||
mod generated;
|
||||
pub mod shim;
|
||||
|
||||
|
||||
33
tests/compat/tests/types_structured_shims.rs
Normal file
33
tests/compat/tests/types_structured_shims.rs
Normal file
@@ -0,0 +1,33 @@
|
||||
use libremetaverse_structured_data::{OSD, OSDParser};
|
||||
use libremetaverse_types::{AssetType, Color4, InventoryType, PrimFlags, Utils};
|
||||
|
||||
#[test]
|
||||
fn catalog_values_and_inert_layouts_are_real() {
|
||||
assert_eq!(AssetType::Material as i8, 57);
|
||||
assert_eq!(InventoryType::CATEGORY.0, 8);
|
||||
assert_eq!(InventoryType::FOLDER.0, 8);
|
||||
assert_eq!(PrimFlags::PHYSICS.0, 1);
|
||||
assert_eq!(Utils::PI.to_bits(), 0x4049_0fdb);
|
||||
|
||||
let color = Color4 {
|
||||
r: 0.25,
|
||||
g: 0.5,
|
||||
b: 0.75,
|
||||
a: 1.0,
|
||||
};
|
||||
assert_eq!((color.r, color.g, color.b, color.a), (0.25, 0.5, 0.75, 1.0));
|
||||
|
||||
let value = OSD::Binary(vec![1, 2, 3]);
|
||||
assert!(matches!(value, OSD::Binary(bytes) if bytes == [1, 2, 3]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serializer_entry_point_returns_its_exact_member_id() {
|
||||
let Err(error) = OSDParser::deserialize_with_bytes(Vec::new()) else {
|
||||
panic!("failure-only serializer unexpectedly returned a value");
|
||||
};
|
||||
assert_eq!(
|
||||
error.csharp_member(),
|
||||
"M:LibreMetaverse.StructuredData.OSDParser.Deserialize(System.Byte[])"
|
||||
);
|
||||
}
|
||||
280
tools/generate_api_shims.py
Normal file
280
tools/generate_api_shims.py
Normal file
@@ -0,0 +1,280 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate callable, failure-only Rust shims from the authoritative catalog."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
import generate_rust_mapping as mapping
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CATALOG = ROOT / "api/public-api.json"
|
||||
LEDGER = ROOT / "api/RUST-MAPPING.tsv"
|
||||
COVERAGE = ROOT / "api/SHIM-COVERAGE.md"
|
||||
|
||||
TARGETS = {
|
||||
"LibreMetaverse.Types": ROOT / "crates/libremetaverse-types/src/generated.rs",
|
||||
"LibreMetaverse.StructuredData": ROOT / "crates/libremetaverse-structured-data/src/generated.rs",
|
||||
}
|
||||
|
||||
UNDERLYING = {
|
||||
"System.SByte": "i8", "System.Byte": "u8", "System.Int16": "i16", "System.UInt16": "u16",
|
||||
"System.Int32": "i32", "System.UInt32": "u32", "System.Int64": "i64", "System.UInt64": "u64",
|
||||
}
|
||||
|
||||
PRIVATE_LAYOUTS = {
|
||||
"T:LibreMetaverse.CacheDictionary`2": [("entries", "std::collections::HashMap<TKey, TValue>")],
|
||||
"T:LibreMetaverse.DoubleDictionary`3": [
|
||||
("by_first", "std::collections::HashMap<TKey1, TValue>"),
|
||||
("by_second", "std::collections::HashMap<TKey2, TValue>"),
|
||||
],
|
||||
"T:LibreMetaverse.EmptyRemovalStrategy`1": [("keys", "Vec<TKey>")],
|
||||
"T:LibreMetaverse.ExpiringCache`2": [("entries", "std::collections::HashMap<TKey, TValue>")],
|
||||
"T:LibreMetaverse.LruRemovalStrategy`1": [("keys", "Vec<TKey>")],
|
||||
"T:LibreMetaverse.MruRemovalStrategy`1": [("keys", "Vec<TKey>")],
|
||||
"T:LibreMetaverse.MultiValueDictionary`2": [("entries", "std::collections::HashMap<TKey, Vec<TValue>>")],
|
||||
"T:LibreMetaverse.TokenBucket": [("content", "i32"), ("max_burst", "i32"), ("drip_rate", "i32")],
|
||||
"T:LibreMetaverse.UUID": [("bytes", "[u8; 16]")],
|
||||
"T:LibreMetaverse.StructuredData.OSDArray": [("values", "Vec<OSD>")],
|
||||
"T:LibreMetaverse.StructuredData.OSDBinary": [("value", "Vec<u8>")],
|
||||
"T:LibreMetaverse.StructuredData.OSDBoolean": [("value", "bool")],
|
||||
"T:LibreMetaverse.StructuredData.OSDDate": [("value", "std::time::SystemTime")],
|
||||
"T:LibreMetaverse.StructuredData.OSDException": [("message", "String")],
|
||||
"T:LibreMetaverse.StructuredData.OSDInteger": [("value", "i32")],
|
||||
"T:LibreMetaverse.StructuredData.OSDMap": [("values", "std::collections::HashMap<String, OSD>")],
|
||||
"T:LibreMetaverse.StructuredData.OSDReal": [("value", "f64")],
|
||||
"T:LibreMetaverse.StructuredData.OSDString": [("value", "String")],
|
||||
"T:LibreMetaverse.StructuredData.OSDUUID": [("value", "libremetaverse_types::UUID")],
|
||||
"T:LibreMetaverse.StructuredData.OSDUri": [("value", "libremetaverse_types::compat::Uri")],
|
||||
}
|
||||
|
||||
|
||||
def generic_names(item: dict) -> list[str]:
|
||||
return [parameter["name"] for parameter in item.get("generic_parameters", [])]
|
||||
|
||||
|
||||
def format_rust(source: str) -> str:
|
||||
result = subprocess.run(
|
||||
["rustfmt", "--edition", "2024", "--emit", "stdout"],
|
||||
input=source,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=True,
|
||||
)
|
||||
return result.stdout
|
||||
|
||||
|
||||
def type_name(type_row: dict) -> str:
|
||||
return type_row["rust_path"].rsplit("::", 1)[-1]
|
||||
|
||||
|
||||
def generic_suffix(names: list[str]) -> str:
|
||||
return "<" + ", ".join(names) + ">" if names else ""
|
||||
|
||||
|
||||
def flags_enum(item: dict) -> bool:
|
||||
return any(attribute["type"] == "System.FlagsAttribute" for attribute in item.get("attributes", []))
|
||||
|
||||
|
||||
def duplicate_enum_values(item: dict) -> bool:
|
||||
values = [member["value"]["value"] for member in item["members"] if member["kind"] == "enum_value"]
|
||||
return len(values) != len(set(values))
|
||||
|
||||
|
||||
def render_enum(item: dict, rust_name: str) -> list[str]:
|
||||
underlying = UNDERLYING[item["enum_underlying_type"]]
|
||||
values = [member for member in item["members"] if member["kind"] == "enum_value"]
|
||||
lines = [f"/// C# type: `{item['doc_id']}`."]
|
||||
if flags_enum(item) or duplicate_enum_values(item):
|
||||
lines += [
|
||||
"#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]",
|
||||
"#[repr(transparent)]",
|
||||
f"pub struct {rust_name}(pub {underlying});",
|
||||
f"impl {rust_name} {{",
|
||||
]
|
||||
for member in values:
|
||||
name = mapping.snake(member["name"]).upper()
|
||||
value = member["value"]["value"]
|
||||
lines += [f" /// C# member: `{member['doc_id']}`.", f" pub const {name}: Self = Self({value});"]
|
||||
lines.append("}")
|
||||
return lines
|
||||
lines += [
|
||||
"#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]",
|
||||
f"#[repr({underlying})]",
|
||||
f"pub enum {rust_name} {{",
|
||||
]
|
||||
for member in values:
|
||||
lines += [f" /// C# member: `{member['doc_id']}`.", f" {mapping.pascal(member['name'])} = {member['value']['value']},"]
|
||||
lines.append("}")
|
||||
return lines
|
||||
|
||||
|
||||
def render_body(signature: str, member_id: str, error_model: str, trait: bool) -> str:
|
||||
if trait:
|
||||
signature = signature.removeprefix("pub ")
|
||||
failure = (
|
||||
f"libremetaverse_types::not_implemented({json.dumps(member_id)})"
|
||||
if error_model.startswith("Result")
|
||||
else f"libremetaverse_types::unimplemented_api!({json.dumps(member_id)})"
|
||||
)
|
||||
return f" {signature} {{ {failure} }}"
|
||||
|
||||
|
||||
def render_type(item: dict, type_row: dict, member_rows: dict[str, dict[str, str]], mapper: mapping.Mapper) -> str:
|
||||
rust_name = type_name(type_row)
|
||||
if item["kind"] == "enum":
|
||||
return "\n".join(render_enum(item, rust_name))
|
||||
names = generic_names(item)
|
||||
suffix = generic_suffix(names)
|
||||
trait = item["kind"] == "interface"
|
||||
lines = [f"/// C# type: `{item['doc_id']}`."]
|
||||
if trait:
|
||||
lines.append(f"pub trait {rust_name}{suffix} {{")
|
||||
else:
|
||||
fields = [member for member in item["members"] if member["kind"] == "field" and not member.get("static")]
|
||||
private_fields = PRIVATE_LAYOUTS.get(item["doc_id"], [])
|
||||
if item["doc_id"] == "T:LibreMetaverse.StructuredData.OSD":
|
||||
lines += [
|
||||
"#[non_exhaustive]",
|
||||
"pub enum OSD {",
|
||||
" Undefined, Boolean(bool), Integer(i32), Real(f64), String(String),",
|
||||
" UUID(libremetaverse_types::UUID), Date(std::time::SystemTime),",
|
||||
" Uri(libremetaverse_types::compat::Uri), Binary(Vec<u8>),",
|
||||
" Array(Vec<OSD>), Map(std::collections::HashMap<String, OSD>), LlsdXml(String),",
|
||||
"}",
|
||||
]
|
||||
elif not fields and not names and not private_fields:
|
||||
lines.append(f"pub struct {rust_name};")
|
||||
else:
|
||||
lines.append(f"pub struct {rust_name}{suffix} {{")
|
||||
generics = set(names)
|
||||
for field in fields:
|
||||
field_type = mapper.type(field["type"], field.get("nullability"), generics)
|
||||
lines += [f" /// C# member: `{field['doc_id']}`.", f" pub {mapping.snake(field['name'])}: {field_type},"]
|
||||
for field_name, field_type in private_fields:
|
||||
lines.append(f" {field_name}: {field_type},")
|
||||
if names:
|
||||
tuple_type = "(" + ", ".join(names) + ("," if len(names) == 1 else "") + ")"
|
||||
lines.append(f" _marker: std::marker::PhantomData<{tuple_type}>,")
|
||||
lines.append("}")
|
||||
lines += [f"impl{suffix} {rust_name}{suffix} {{"]
|
||||
|
||||
for member in item["members"]:
|
||||
if member["kind"] in {"enum_value"} or (member["kind"] == "field" and not member.get("static")):
|
||||
continue
|
||||
row = member_rows[member["doc_id"]]
|
||||
lines.append(f" /// C# member: `{member['doc_id']}`.")
|
||||
if member["kind"] == "constant":
|
||||
value = member["value"]["value"]
|
||||
lines.append(f" {row['rust_signature']} = {value};")
|
||||
continue
|
||||
signatures = row["rust_signature"].split(" ; ")
|
||||
for index, signature in enumerate(signatures):
|
||||
if index:
|
||||
lines.append(f" /// Setter for C# member: `{member['doc_id']}`.")
|
||||
lines.append(render_body(signature, member["doc_id"], row["error_model"], trait))
|
||||
lines.append("}")
|
||||
if not trait:
|
||||
for interface in item.get("interfaces", []):
|
||||
base, arguments, _ = mapping.type_parts(interface)
|
||||
if base not in mapper.interfaces:
|
||||
continue
|
||||
mapped_arguments = [mapper.type(argument, None, set(names)) for argument in arguments]
|
||||
trait_path = mapper.resolved[base] + generic_suffix(mapped_arguments)
|
||||
lines.append(f"impl{suffix} {trait_path} for {rust_name}{suffix} {{}}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def generate_sources(catalog: dict) -> tuple[dict[Path, str], dict[str, tuple[int, int]]]:
|
||||
type_rows, resolved = mapping.build_type_rows(catalog)
|
||||
type_by_id = {row["csharp_type_id"]: row for row in type_rows}
|
||||
interfaces = {mapping.type_parts(item["signature"])[0] for assembly in catalog["assemblies"] for item in assembly["types"] if item["kind"] == "interface"}
|
||||
mapper = mapping.Mapper(resolved, interfaces)
|
||||
member_rows = {row["csharp_id"]: row for row in csv.DictReader(LEDGER.open(), delimiter="\t")}
|
||||
outputs = {}
|
||||
coverage = {}
|
||||
for assembly in catalog["assemblies"]:
|
||||
name = assembly["identity"]["name"]
|
||||
if name not in TARGETS:
|
||||
continue
|
||||
chunks = [
|
||||
"// @generated by tools/generate_api_shims.py; do not edit by hand.",
|
||||
"#![allow(clippy::missing_errors_doc)]",
|
||||
"#![allow(clippy::must_use_candidate)]",
|
||||
"#![allow(clippy::all, clippy::pedantic, dead_code, unused_variables)]",
|
||||
"#![allow(non_camel_case_types)]",
|
||||
"#![allow(non_snake_case)]",
|
||||
"",
|
||||
]
|
||||
for item in assembly["types"]:
|
||||
chunks += [render_type(item, type_by_id[item["doc_id"]], member_rows, mapper), ""]
|
||||
source = format_rust("\n".join(chunks).rstrip() + "\n")
|
||||
expected_types = {item["doc_id"] for item in assembly["types"]}
|
||||
expected_members = {member["doc_id"] for item in assembly["types"] for member in item["members"]}
|
||||
emitted_types = set(re.findall(r"C# type: `(.*)`\.", source))
|
||||
emitted_members = set(re.findall(r"C# member: `(.*)`\.", source))
|
||||
if emitted_types != expected_types or emitted_members != expected_members:
|
||||
raise ValueError(
|
||||
f"generated catalog ID mismatch for {name}: "
|
||||
f"types missing/stale={len(expected_types - emitted_types)}/{len(emitted_types - expected_types)}, "
|
||||
f"members missing/stale={len(expected_members - emitted_members)}/{len(emitted_members - expected_members)}"
|
||||
)
|
||||
outputs[TARGETS[name]] = source
|
||||
coverage[name] = (len(assembly["types"]), sum(len(item["members"]) for item in assembly["types"]))
|
||||
return outputs, coverage
|
||||
|
||||
|
||||
def coverage_report(catalog: dict, coverage: dict[str, tuple[int, int]]) -> str:
|
||||
lines = [
|
||||
"# Callable shim coverage", "", "Generated by `python3 tools/generate_api_shims.py`; do not edit by hand.", "",
|
||||
"| Assembly | Types | Members | Status |", "|---|---:|---:|---|",
|
||||
]
|
||||
total_types = total_members = 0
|
||||
for assembly in catalog["assemblies"]:
|
||||
name = assembly["identity"]["name"]
|
||||
types = len(assembly["types"])
|
||||
members = sum(len(item["members"]) for item in assembly["types"])
|
||||
if name in coverage:
|
||||
got_types, got_members = coverage[name]
|
||||
if (types, members) != (got_types, got_members):
|
||||
raise ValueError(f"incomplete generated slice for {name}")
|
||||
status = "callable failure-only shim"
|
||||
total_types += types
|
||||
total_members += members
|
||||
else:
|
||||
status = "pending milestone issue"
|
||||
lines.append(f"| `{name}` | {types:,} | {members:,} | {status} |")
|
||||
lines += ["", f"Current callable coverage: **{total_types:,} types / {total_members:,} members**.", ""]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def generate() -> dict[Path, str]:
|
||||
catalog = json.loads(CATALOG.read_text())
|
||||
sources, coverage = generate_sources(catalog)
|
||||
sources[COVERAGE] = coverage_report(catalog, coverage)
|
||||
return sources
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--check", action="store_true")
|
||||
args = parser.parse_args()
|
||||
outputs = generate()
|
||||
if args.check:
|
||||
stale = [str(path.relative_to(ROOT)) for path, text in outputs.items() if not path.exists() or path.read_text() != text]
|
||||
if stale:
|
||||
raise SystemExit("stale generated API shims: " + ", ".join(stale))
|
||||
else:
|
||||
for path, text in outputs.items():
|
||||
path.write_text(text)
|
||||
print("generated " + ", ".join(str(path.relative_to(ROOT)) for path in outputs))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -54,7 +54,7 @@ PRIMITIVES = {
|
||||
}
|
||||
|
||||
STDLIB = {
|
||||
"System.Array": "Vec<T>",
|
||||
"System.Array": "libremetaverse_types::compat::Array",
|
||||
"System.ArraySegment": "libremetaverse_types::compat::ArraySegment<T>",
|
||||
"System.Collections.BitArray": "Vec<bool>",
|
||||
"System.Collections.Generic.Dictionary": "std::collections::HashMap<K, V>",
|
||||
@@ -96,8 +96,8 @@ STDLIB = {
|
||||
"System.DateTime": "std::time::SystemTime",
|
||||
"System.DateTimeOffset": "std::time::SystemTime",
|
||||
"System.TimeSpan": "std::time::Duration",
|
||||
"System.Guid": "uuid::Uuid",
|
||||
"System.Uri": "url::Url",
|
||||
"System.Guid": "libremetaverse_types::compat::Guid",
|
||||
"System.Uri": "libremetaverse_types::compat::Uri",
|
||||
"System.Version": "libremetaverse_types::compat::Version",
|
||||
"System.Random": "libremetaverse_types::compat::RandomSource",
|
||||
"System.Exception": "libremetaverse_types::compat::ExternalError",
|
||||
@@ -114,7 +114,7 @@ STDLIB = {
|
||||
"System.ValueType": "libremetaverse_types::compat::Value",
|
||||
"System.Text.Encoding": "libremetaverse_types::compat::TextEncoding",
|
||||
"System.Text.StringBuilder": "String",
|
||||
"System.Text.RegularExpressions.Regex": "regex::Regex",
|
||||
"System.Text.RegularExpressions.Regex": "libremetaverse_types::compat::Regex",
|
||||
"System.IO.Stream": "Box<dyn libremetaverse_types::compat::ReadWrite + Send>",
|
||||
"System.IO.MemoryStream": "std::io::Cursor<Vec<u8>>",
|
||||
"System.IO.TextReader": "Box<dyn std::io::BufRead + Send>",
|
||||
@@ -142,8 +142,8 @@ STDLIB = {
|
||||
"System.ValueTuple": "(T1, T2)",
|
||||
"System.Xml.XmlNode": "libremetaverse_structured_data::xml::Node",
|
||||
"System.Xml.XmlElement": "libremetaverse_structured_data::xml::Element",
|
||||
"System.Xml.XmlReader": "libremetaverse_structured_data::xml::Reader<R>",
|
||||
"System.Xml.XmlWriter": "libremetaverse_structured_data::xml::Writer<W>",
|
||||
"System.Xml.XmlReader": "libremetaverse_structured_data::xml::Reader",
|
||||
"System.Xml.XmlWriter": "libremetaverse_structured_data::xml::Writer",
|
||||
"System.Text.Json.JsonElement": "libremetaverse_voice_webrtc::signaling::JsonValue",
|
||||
}
|
||||
|
||||
@@ -459,8 +459,9 @@ def build_type_rows(catalog: dict) -> tuple[list[dict[str, str]], dict[str, str]
|
||||
|
||||
|
||||
class Mapper:
|
||||
def __init__(self, resolved: dict[str, str]):
|
||||
def __init__(self, resolved: dict[str, str], interfaces: set[str] | None = None):
|
||||
self.resolved = resolved
|
||||
self.interfaces = interfaces or set()
|
||||
|
||||
def type(self, value: str, nullability: dict | None = None, generics: set[str] | None = None) -> str:
|
||||
base, arguments, suffix = type_parts(value)
|
||||
@@ -498,8 +499,10 @@ class Mapper:
|
||||
"System.ValueTuple", "System.Threading.Tasks.Task", "System.Threading.Tasks.ValueTask",
|
||||
}:
|
||||
target = substitute(target, mapped_arguments)
|
||||
if "<" not in target and target not in {"Ord", "Eq", "Clone", "std::fmt::Display"}:
|
||||
if "<" not in target and not target.startswith("(") and target not in {"Ord", "Eq", "Clone", "std::fmt::Display"}:
|
||||
target += "<" + ", ".join(mapped_arguments) + ">"
|
||||
if base in self.interfaces:
|
||||
target = f"Box<dyn {target}>"
|
||||
for _ in re.findall(r"\[[,]*\]", suffix):
|
||||
target = f"Vec<{target}>"
|
||||
if "*" in suffix:
|
||||
@@ -551,7 +554,7 @@ def map_parameter(mapper: Mapper, parameter: dict, generics: set[str]) -> tuple[
|
||||
def member_base_name(item: dict) -> str:
|
||||
if item["kind"] == "enum_value":
|
||||
return pascal(item["name"])
|
||||
if item["kind"] == "constant" or (item["kind"] == "field" and item.get("static")):
|
||||
if item["kind"] == "constant":
|
||||
return snake(item["name"]).upper()
|
||||
if item["kind"] == "constructor":
|
||||
return "new"
|
||||
@@ -566,6 +569,14 @@ def member_base_name(item: dict) -> str:
|
||||
return operators.get(item["name"], snake(item["name"].removesuffix("Async")))
|
||||
|
||||
|
||||
def parameter_slug(value: str) -> str:
|
||||
base, _, suffix = type_parts(value)
|
||||
slug = snake(base.rsplit(".", 1)[-1])
|
||||
if "[" in suffix:
|
||||
return "bytes" if base == "System.Byte" else f"{slug}_array"
|
||||
return slug
|
||||
|
||||
|
||||
def member_signature(mapper: Mapper, owner: dict, item: dict, rust_name: str) -> tuple[str, str, str, str, str]:
|
||||
generics = generic_names(item, owner)
|
||||
mapped_parameters = [map_parameter(mapper, parameter, generics) for parameter in item.get("parameters", [])]
|
||||
@@ -581,7 +592,8 @@ def member_signature(mapper: Mapper, owner: dict, item: dict, rust_name: str) ->
|
||||
elif kind == "constant":
|
||||
signature = f"pub const {rust_name}: {value_type}"
|
||||
elif item.get("static"):
|
||||
signature = f"pub static {rust_name}: {value_type}"
|
||||
signature = f"pub fn {rust_name}() -> {value_type}"
|
||||
return signature, "none", asyncness, error_model, "static_field_accessor"
|
||||
else:
|
||||
signature = f"pub {rust_name}: {value_type}"
|
||||
return signature, "owned_value", asyncness, "none", "direct"
|
||||
@@ -653,16 +665,22 @@ def build_member_rows(catalog: dict, mapper: Mapper) -> list[dict[str, str]]:
|
||||
for owner in assembly["types"]:
|
||||
_, module, owner_path = declared_type_path(assembly_name, owner)
|
||||
groups = Counter(member_base_name(member) for member in owner["members"])
|
||||
enum_values = [member.get("value", {}).get("value") for member in owner["members"] if member["kind"] == "enum_value"]
|
||||
enum_constants = owner["kind"] == "enum" and (
|
||||
any(attribute["type"] == "System.FlagsAttribute" for attribute in owner.get("attributes", []))
|
||||
or len(enum_values) != len(set(enum_values))
|
||||
)
|
||||
used = set()
|
||||
for item in owner["members"]:
|
||||
base_name = member_base_name(item)
|
||||
if item["kind"] == "enum_value" and enum_constants:
|
||||
base_name = snake(item["name"]).upper()
|
||||
rust_name = base_name
|
||||
decision = "direct_snake_case"
|
||||
if groups[base_name] > 1:
|
||||
suffixes = []
|
||||
for parameter in item.get("parameters", []):
|
||||
base, _, _ = type_parts(parameter["type"])
|
||||
suffixes.append(snake(base.rsplit(".", 1)[-1]))
|
||||
suffixes.append(parameter_slug(parameter["type"]))
|
||||
suffix = "_".join(suffixes) or item["kind"]
|
||||
rust_name = f"{base_name}_with_{suffix}"
|
||||
decision = "descriptive_overload_name"
|
||||
@@ -808,7 +826,8 @@ def coverage_report(catalog: dict, type_rows: list[dict[str, str]], member_rows:
|
||||
def generate() -> dict[Path, str]:
|
||||
catalog = json.loads(CATALOG.read_text())
|
||||
type_rows, resolved = build_type_rows(catalog)
|
||||
mapper = Mapper(resolved)
|
||||
interfaces = {type_parts(item["signature"])[0] for assembly in catalog["assemblies"] for item in assembly["types"] if item["kind"] == "interface"}
|
||||
mapper = Mapper(resolved, interfaces)
|
||||
member_rows = build_member_rows(catalog, mapper)
|
||||
validate(catalog, type_rows, member_rows, mapper)
|
||||
type_fields = ["csharp_type_id", "csharp_signature", "source_assembly", "source_kind", "rust_crate", "rust_module", "rust_path", "mapping_decision", "generic_bounds", "inheritance_mapping", "enum_mapping", "status", "notes"]
|
||||
|
||||
Reference in New Issue
Block a user