Build strict NUnit parity harness
Some checks failed
Rust API gates / api-gates (push) Has been cancelled
Some checks failed
Rust API gates / api-gates (push) Has been cancelled
This commit is contained in:
@@ -17,6 +17,9 @@ jobs:
|
||||
python3 tools/generate_rust_mapping.py --check
|
||||
python3 tools/generate_api_shims.py --check
|
||||
python3 tools/check_api_coverage.py
|
||||
python3 tools/generate_surface.py --check
|
||||
python3 tools/check_test_parity.py
|
||||
python3 -m unittest discover -s tools -p 'test_*.py'
|
||||
- name: Formatting
|
||||
run: cargo fmt --all -- --check
|
||||
- name: Workspace check
|
||||
|
||||
17
README.md
17
README.md
@@ -5,9 +5,9 @@ MetaCrate is a test-driven, clean native Rust reimplementation of
|
||||
is a behavioral and API reference only: the finished project must not load,
|
||||
host, bind to, invoke, or ship the .NET implementation. The current
|
||||
stage is a compiling structural shell: crate boundaries mirror the .NET library
|
||||
projects, public C# type names have Rust declarations, every upstream NUnit case
|
||||
has a traceable placeholder, and every sample/tool project has a Rust binary
|
||||
target. Types, StructuredData, Imaging, PrimMesher, both rendering adapters, and
|
||||
projects, public C# type names have Rust declarations, every upstream NUnit
|
||||
invocation has a traceable catalog entry, and every sample/tool project has a
|
||||
Rust binary target. Types, StructuredData, Imaging, PrimMesher, both rendering adapters, and
|
||||
the main assembly's packet/message/asset/primitive wire-data, core
|
||||
runtime/networking, avatar-facing manager, world/social/service manager, RLV,
|
||||
LSL tools, Utilities, Vivox, and WebRTC slices now have complete callable,
|
||||
@@ -15,6 +15,10 @@ failure-only signatures. The independent downstream fixture compiles every
|
||||
cataloged type and member with zero exclusions; semantic Rust test translations
|
||||
are the next stage.
|
||||
|
||||
Pending NUnit cases are inventory only, not generated fake tests. A case counts
|
||||
as translated only when an explicit Rust test body calls the mapped API and
|
||||
retains the upstream identity and body hash.
|
||||
|
||||
The source snapshot, compatibility rules, dependency research, and ordered
|
||||
implementation plan are in [RUSTREWRITE.md](RUSTREWRITE.md).
|
||||
|
||||
@@ -32,6 +36,13 @@ cargo build --workspace
|
||||
cargo test --workspace --no-run
|
||||
python3 tools/generate_rust_mapping.py --check
|
||||
python3 tools/generate_api_shims.py --check
|
||||
python3 tools/check_test_parity.py
|
||||
```
|
||||
|
||||
`tests/upstream-tests.json` is the machine-readable NUnit parity catalog.
|
||||
Translated tests live in hand-written Rust files with the `parity-case` marker
|
||||
documented in `tests/PARITY.md`; `python3 tools/generate_surface.py --check`
|
||||
verifies the catalog against the pinned adjacent LibreMetaverse checkout without
|
||||
overwriting those files.
|
||||
|
||||
Running `cargo test --workspace` is intentionally red during the shim stage.
|
||||
|
||||
@@ -36,9 +36,11 @@ Generated inventories are intentionally checked in:
|
||||
Run `python3 tools/generate_surface.py` after deliberately changing the pinned
|
||||
upstream snapshot, then run `python3 tools/extract_public_api.py` to rebuild the
|
||||
authoritative metadata catalog twice and require byte-identical output. The
|
||||
source generator rejects a test count other than 1,295 so upstream drift cannot
|
||||
source generator rejects a test count other than 1,289 so upstream drift cannot
|
||||
silently remove tests. Update both commit constants and review all ledger diffs
|
||||
in the same change.
|
||||
in the same change. The earlier 1,295 inventory included five commented-out
|
||||
`[Test]` attributes and double-counted the parameterless `[Test]` marker on one
|
||||
parameterized `[TestCase]` method; those are not NUnit invocations.
|
||||
|
||||
### Current shell status
|
||||
|
||||
@@ -57,8 +59,8 @@ The current workspace is a structural baseline, not a working client:
|
||||
slices expose all 3,066 final Rust-facing types and all 30,789 callable
|
||||
members with real fields/constants/enum values and standardized failing
|
||||
bodies;
|
||||
- all 1,295 NUnit `[Test]`/`[TestCase]` invocations have compiling Rust test
|
||||
entries that intentionally panic with their source identity and body hash;
|
||||
- all 1,289 real NUnit `[Test]`/`[TestCase]` invocations have stable catalog
|
||||
entries; pending cases do not generate executable placeholder tests;
|
||||
- all nine sample/tool projects have compiling Rust binary targets;
|
||||
- the 128 TestClient command source files are retained as a command inventory.
|
||||
|
||||
@@ -66,8 +68,8 @@ No generated type shells remain pending: all cataloged members are callable,
|
||||
and the standalone downstream fixture names or invokes all 3,066 types and
|
||||
30,789 members with zero exclusions. `api/SURFACE.tsv` is still an inventory
|
||||
aid, not proof of API coverage: it records declaration lines but does not fully
|
||||
parse multiline signatures. Likewise, a generated `pending` test is a parity
|
||||
placeholder, not a semantic translation. **The full public API signature shim
|
||||
parse multiline signatures. Likewise, a pending catalog entry is not a
|
||||
semantic translation. **The full public API signature shim
|
||||
gate is complete; semantic test translation is the next stage.**
|
||||
|
||||
The avatar-facing slice preserves the C# agent, movement/camera, appearance and
|
||||
@@ -316,12 +318,21 @@ The signature gate is complete only when:
|
||||
### 3.2 Full semantic test-suite gate
|
||||
|
||||
After the full signature shim compiles, translate the entire C# test suite while
|
||||
leaving all library bodies unimplemented. Every one of the 1,295 NUnit
|
||||
leaving all library bodies unimplemented. Every one of the 1,289 NUnit
|
||||
invocations must become a Rust test with the same inputs, setup, operation,
|
||||
observations, assertions, tolerances and expected error/event behavior. Shared
|
||||
C# test helpers become shared Rust test helpers; embedded fixtures and literal
|
||||
payloads are copied with license/source attribution and byte hashes.
|
||||
|
||||
The parity harness is in place: every invocation has a stable source/case ID,
|
||||
parameter identity, body hash, category, fixture dependency list, Rust location,
|
||||
and semantic-review status. Reviewed tests are identified by `parity-case`
|
||||
markers in hand-written Rust files, so regeneration records unresolved cases
|
||||
without creating fake executable tests and fails on body drift. The checked-in
|
||||
audit reports pending, translated, ignored-live, benchmark, drifted, missing,
|
||||
duplicate, stale, and unreviewed cases; the initial handover contains 1,289
|
||||
unreviewed cases.
|
||||
|
||||
The Rust tests must call the public APIs rather than internal replacements.
|
||||
Where the C# tests call internal members through friend-assembly access, record
|
||||
that fact and place equivalent Rust unit tests inside the owning crate without
|
||||
@@ -336,7 +347,7 @@ they must distinguish the intended invariant from the placeholder panic.
|
||||
|
||||
The test-suite gate is complete only when:
|
||||
|
||||
- `pending(` occurs zero times in Rust test sources;
|
||||
- no generated placeholder or dispatcher body can count as reviewed;
|
||||
- the parity catalog reports exactly one reviewed Rust case for every upstream
|
||||
`[Test]`/`[TestCase]`, with the matching C# body hash;
|
||||
- `cargo test --workspace --no-run` compiles all tests against public signatures;
|
||||
@@ -669,14 +680,14 @@ the complete signature gate in section 3.1 across all crates. Then translate the
|
||||
complete suite in section 3.2 across all crates. Only after both global gates
|
||||
may implementation proceed in dependency order.
|
||||
|
||||
For each generated test:
|
||||
For each pending catalog case:
|
||||
|
||||
1. Open the exact C# source/line and verify its method-body SHA-256 still matches
|
||||
`tests/upstream-tests.json`.
|
||||
2. Identify setup, action, assertions, parameterized cases, fixtures, categories,
|
||||
and intended exception/event/timeout behavior.
|
||||
3. Replace only that Rust `pending(...)` body with semantically equivalent Rust.
|
||||
Keep the generated Rust test name or update the ledger in the same change.
|
||||
3. Write an explicit Rust `#[test]` body in a hand-written test file, add its
|
||||
`parity-case` marker, and call the mapped library API directly.
|
||||
4. Port literal bytes, UUIDs, timestamps, tolerances, culture assumptions, and
|
||||
ordering exactly. Do not weaken an assertion merely because the shim differs.
|
||||
5. If the needed API is absent, stop and treat that as a signature-coverage
|
||||
@@ -717,8 +728,8 @@ cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||
cargo doc --workspace --no-deps
|
||||
```
|
||||
|
||||
During Stage 0, `cargo test --workspace` fails in generated parity placeholders.
|
||||
During the full test-suite gate, `pending(` is the explicit remaining-work
|
||||
During Stage 0, pending cases exist only in the parity catalog. During the full
|
||||
test-suite gate, the catalog's pending count is the explicit remaining-work
|
||||
count and CI prohibits increases. At the end of that gate, tests still fail,
|
||||
but only because callable production signatures report standardized
|
||||
not-implemented member IDs. During implementation, that failure inventory then
|
||||
@@ -774,7 +785,7 @@ not part of the default smoke test.
|
||||
map 100% of public members to reviewed Rust signatures, and implement only
|
||||
standardized failing bodies. All signatures and downstream compile fixtures
|
||||
build before test translation begins.
|
||||
3. **Full semantic test translation:** replace all 1,295 `pending` cases with
|
||||
3. **Full semantic test translation:** replace all 1,289 pending cases with
|
||||
exact reviewed Rust equivalents using the public shim. The entire suite
|
||||
compiles and fails only at standardized unimplemented member boundaries.
|
||||
4. **Types implementation:** UUID, math, enums, conversions, collections,
|
||||
|
||||
2601
tests/PARITY.md
2601
tests/PARITY.md
File diff suppressed because it is too large
Load Diff
@@ -1,13 +1,275 @@
|
||||
//! Shared failure marker used by generated compatibility tests.
|
||||
//! Shared deterministic fixtures used by translated compatibility tests.
|
||||
|
||||
/// Fails a not-yet-translated upstream test while retaining parity metadata.
|
||||
use std::collections::BTreeMap;
|
||||
use std::io;
|
||||
use std::path::{Component, Path, PathBuf};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::Duration;
|
||||
|
||||
/// Asserts equality within the exact absolute tolerance carried by an upstream test.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Always, with the retained upstream test identity.
|
||||
/// Panics when the tolerance is invalid or the values differ beyond it.
|
||||
#[track_caller]
|
||||
pub fn pending(source: &str, line: u64, test: &str, attribute: &str, body_sha256: &str) -> ! {
|
||||
panic!(
|
||||
"pending LibreMetaverse parity test {test} ({attribute}) from {source}:{line}; C# body sha256={body_sha256}"
|
||||
)
|
||||
pub fn assert_close(actual: f64, expected: f64, tolerance: f64) {
|
||||
assert!(
|
||||
tolerance.is_finite() && tolerance >= 0.0,
|
||||
"tolerance must be finite and non-negative"
|
||||
);
|
||||
assert!(
|
||||
(actual - expected).abs() <= tolerance,
|
||||
"expected {expected:?} +/- {tolerance:?}, got {actual:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// Asserts byte equality and reports the first differing offset.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics when the byte slices differ.
|
||||
#[track_caller]
|
||||
pub fn assert_bytes_eq(actual: &[u8], expected: &[u8]) {
|
||||
if actual == expected {
|
||||
return;
|
||||
}
|
||||
let offset = actual
|
||||
.iter()
|
||||
.zip(expected)
|
||||
.position(|(actual, expected)| actual != expected)
|
||||
.unwrap_or_else(|| actual.len().min(expected.len()));
|
||||
panic!(
|
||||
"byte fixtures differ at offset {offset}: expected {} bytes, got {} bytes",
|
||||
expected.len(),
|
||||
actual.len()
|
||||
);
|
||||
}
|
||||
|
||||
/// Decodes a whitespace-separated or contiguous hexadecimal byte fixture.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns an error for an odd digit count or a non-hexadecimal digit.
|
||||
pub fn decode_hex(input: &str) -> Result<Vec<u8>, String> {
|
||||
let digits: Vec<_> = input
|
||||
.bytes()
|
||||
.filter(|byte| !byte.is_ascii_whitespace())
|
||||
.collect();
|
||||
if digits.len() % 2 != 0 {
|
||||
return Err("hex fixture has an odd number of digits".into());
|
||||
}
|
||||
digits
|
||||
.chunks_exact(2)
|
||||
.enumerate()
|
||||
.map(|(index, pair)| {
|
||||
let high = hex_digit(pair[0]);
|
||||
let low = hex_digit(pair[1]);
|
||||
high.zip(low)
|
||||
.map(|(high, low)| high << 4 | low)
|
||||
.ok_or_else(|| format!("invalid hex byte at digit {}", index * 2))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn hex_digit(byte: u8) -> Option<u8> {
|
||||
match byte {
|
||||
b'0'..=b'9' => Some(byte - b'0'),
|
||||
b'a'..=b'f' => Some(byte - b'a' + 10),
|
||||
b'A'..=b'F' => Some(byte - b'A' + 10),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolves test data below a fixture root without permitting path traversal.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns [`io::ErrorKind::InvalidInput`] for absolute or parent-relative paths.
|
||||
pub fn test_data_path(root: &Path, relative: &Path) -> io::Result<PathBuf> {
|
||||
if relative.components().any(|component| {
|
||||
matches!(
|
||||
component,
|
||||
Component::ParentDir | Component::RootDir | Component::Prefix(_)
|
||||
)
|
||||
}) {
|
||||
return Err(io::Error::new(
|
||||
io::ErrorKind::InvalidInput,
|
||||
"test data path must stay below its fixture root",
|
||||
));
|
||||
}
|
||||
Ok(root.join(relative))
|
||||
}
|
||||
|
||||
/// Loads a test-data file through [`test_data_path`].
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Returns path validation and file-read errors.
|
||||
pub fn load_test_data(root: &Path, relative: &Path) -> io::Result<Vec<u8>> {
|
||||
std::fs::read(test_data_path(root, relative)?)
|
||||
}
|
||||
|
||||
/// A cloneable clock advanced only by the test.
|
||||
#[derive(Clone, Debug, Default)]
|
||||
pub struct ManualClock(Arc<Mutex<Duration>>);
|
||||
|
||||
impl ManualClock {
|
||||
/// Returns the deterministic elapsed time.
|
||||
#[must_use]
|
||||
pub fn now(&self) -> Duration {
|
||||
*self
|
||||
.0
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
}
|
||||
|
||||
/// Advances the clock and returns its new value.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics if the resulting duration exceeds [`Duration::MAX`].
|
||||
#[must_use]
|
||||
pub fn advance(&self, duration: Duration) -> Duration {
|
||||
let mut now = self
|
||||
.0
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
*now += duration;
|
||||
*now
|
||||
}
|
||||
}
|
||||
|
||||
/// A recorded deterministic HTTP-like request.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct RecordedRequest {
|
||||
pub method: String,
|
||||
pub uri: String,
|
||||
pub body: Vec<u8>,
|
||||
}
|
||||
|
||||
/// A canned deterministic HTTP-like response.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct FakeResponse {
|
||||
pub status: u16,
|
||||
pub content_type: String,
|
||||
pub body: Vec<u8>,
|
||||
}
|
||||
|
||||
/// An in-memory exact-URI/path fake with ordered request capture.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct FakeNetwork {
|
||||
exact: Mutex<BTreeMap<(String, String), FakeResponse>>,
|
||||
paths: Mutex<BTreeMap<(String, String), FakeResponse>>,
|
||||
requests: Mutex<Vec<RecordedRequest>>,
|
||||
}
|
||||
|
||||
impl FakeNetwork {
|
||||
/// Adds an exact method/URI response.
|
||||
pub fn add_response(&self, method: &str, uri: &str, response: FakeResponse) {
|
||||
self.exact
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.insert((method.to_owned(), uri.to_owned()), response);
|
||||
}
|
||||
|
||||
/// Adds a response matched after removing the request query string.
|
||||
pub fn add_path_response(&self, method: &str, uri_without_query: &str, response: FakeResponse) {
|
||||
self.paths
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.insert((method.to_owned(), uri_without_query.to_owned()), response);
|
||||
}
|
||||
|
||||
/// Records a request and returns its configured response, or a deterministic 404.
|
||||
pub fn send(&self, method: &str, uri: &str, body: impl Into<Vec<u8>>) -> FakeResponse {
|
||||
self.requests
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.push(RecordedRequest {
|
||||
method: method.to_owned(),
|
||||
uri: uri.to_owned(),
|
||||
body: body.into(),
|
||||
});
|
||||
let key = (method.to_owned(), uri.to_owned());
|
||||
if let Some(response) = self
|
||||
.exact
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.get(&key)
|
||||
{
|
||||
return response.clone();
|
||||
}
|
||||
let path_key = (
|
||||
method.to_owned(),
|
||||
uri.split('?').next().unwrap_or(uri).to_owned(),
|
||||
);
|
||||
self.paths
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.get(&path_key)
|
||||
.cloned()
|
||||
.unwrap_or_else(|| FakeResponse {
|
||||
status: 404,
|
||||
content_type: "application/octet-stream".into(),
|
||||
body: Vec::new(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Returns all captured requests in send order.
|
||||
#[must_use]
|
||||
pub fn requests(&self) -> Vec<RecordedRequest> {
|
||||
self.requests
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.clone()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn deterministic_harness_covers_clock_network_and_bytes() {
|
||||
assert_close(1.001, 1.0, 0.01);
|
||||
assert_bytes_eq(&decode_hex("00 ff 2A").unwrap(), &[0, 255, 42]);
|
||||
assert!(decode_hex("é").is_err());
|
||||
|
||||
let clock = ManualClock::default();
|
||||
assert_eq!(
|
||||
clock.advance(Duration::from_millis(25)),
|
||||
Duration::from_millis(25)
|
||||
);
|
||||
assert_eq!(clock.now(), Duration::from_millis(25));
|
||||
|
||||
let network = FakeNetwork::default();
|
||||
network.add_path_response(
|
||||
"GET",
|
||||
"https://example.test/cap",
|
||||
FakeResponse {
|
||||
status: 200,
|
||||
content_type: "application/llsd+xml".into(),
|
||||
body: b"fixture".to_vec(),
|
||||
},
|
||||
);
|
||||
assert_eq!(
|
||||
network.send("GET", "https://example.test/cap?tid=1", []),
|
||||
FakeResponse {
|
||||
status: 200,
|
||||
content_type: "application/llsd+xml".into(),
|
||||
body: b"fixture".to_vec(),
|
||||
}
|
||||
);
|
||||
assert_eq!(network.requests()[0].uri, "https://example.test/cap?tid=1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_data_rejects_parent_traversal() {
|
||||
assert_eq!(
|
||||
test_data_path(Path::new("fixtures"), Path::new("../secret"))
|
||||
.unwrap_err()
|
||||
.kind(),
|
||||
io::ErrorKind::InvalidInput
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
25
tools/check_test_parity.py
Normal file
25
tools/check_test_parity.py
Normal file
@@ -0,0 +1,25 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Audit the checked-in NUnit-to-Rust parity catalog and source markers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
from generate_surface import check_test_parity
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--root", type=Path, default=Path("."))
|
||||
parser.add_argument("--require-reviewed", action="store_true")
|
||||
args = parser.parse_args()
|
||||
report = check_test_parity(args.root.resolve(), args.require_reviewed)
|
||||
print(
|
||||
"test parity: "
|
||||
+ ", ".join(f"{name}={count}" for name, count in report.items())
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -13,7 +13,9 @@ import hashlib
|
||||
import json
|
||||
import keyword
|
||||
import re
|
||||
from collections import defaultdict
|
||||
import subprocess
|
||||
import tempfile
|
||||
from collections import Counter, defaultdict
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
@@ -51,13 +53,33 @@ TYPE_RE = re.compile(
|
||||
)
|
||||
NAMESPACE_RE = re.compile(r"^\s*namespace\s+([A-Za-z_][A-Za-z0-9_.]*)", re.MULTILINE)
|
||||
TEST_ATTR_RE = re.compile(r"\[(Test|TestCase)(?:\((.*?)\))?\]", re.DOTALL)
|
||||
CATEGORY_RE = re.compile(r'\[Category\("([^"]+)"\)\]')
|
||||
METHOD_RE = re.compile(
|
||||
r"\b(?:public|internal)\s+(?:static\s+)?(?:async\s+)?"
|
||||
r"(?:void|Task(?:\s*<[^>]+>)?|ValueTask(?:\s*<[^>]+>)?)\s+"
|
||||
r"([A-Za-z_][A-Za-z0-9_]*)\s*\(",
|
||||
re.DOTALL,
|
||||
)
|
||||
CLASS_RE = re.compile(r"\b(?:public|internal)\s+(?:sealed\s+|partial\s+)?class\s+([A-Za-z_][A-Za-z0-9_]*)")
|
||||
CLASS_RE = re.compile(
|
||||
r"\b(?:(?:public|internal|private|protected|sealed|partial|abstract|static)\s+)*"
|
||||
r"class\s+([A-Za-z_][A-Za-z0-9_]*)"
|
||||
)
|
||||
PARITY_MARKER_RE = re.compile(
|
||||
r"^// parity-case: (?P<id>\S+) (?P<body_sha256>[0-9a-f]{64}) "
|
||||
r"(?P<status>translated|ignored-live|benchmark)$",
|
||||
re.MULTILINE,
|
||||
)
|
||||
RUST_TEST_RE = re.compile(r"(?:#\[[^\]]+\]\s*)*fn\s+([A-Za-z_][A-Za-z0-9_]*)\s*\(", re.MULTILINE)
|
||||
FORBIDDEN_REVIEW_BODY_RE = re.compile(
|
||||
r"\bpending\s*\(|\bassert_[A-Za-z0-9_]*case\s*\(|"
|
||||
r"\b(?:todo|unimplemented|unimplemented_api)\s*!|\bnot_implemented\s*\("
|
||||
)
|
||||
EXPECTED_TESTS = 1289
|
||||
PARITY_FILES = (
|
||||
Path("tests/compat/tests/generated_parity.rs"),
|
||||
Path("tests/PARITY.md"),
|
||||
Path("tests/upstream-tests.json"),
|
||||
)
|
||||
|
||||
|
||||
def snake(name: str) -> str:
|
||||
@@ -88,6 +110,151 @@ def source_files(root: Path) -> list[Path]:
|
||||
return sorted(p for p in root.rglob("*.cs") if "obj" not in p.parts and "bin" not in p.parts)
|
||||
|
||||
|
||||
def mask_csharp_comments(text: str) -> str:
|
||||
"""Blank comments without shifting source offsets or touching string literals."""
|
||||
masked = list(text)
|
||||
index = 0
|
||||
state = "code"
|
||||
while index < len(text):
|
||||
char = text[index]
|
||||
following = text[index + 1] if index + 1 < len(text) else ""
|
||||
if state == "code":
|
||||
if char == '"':
|
||||
state = "string"
|
||||
elif char == "'":
|
||||
state = "char"
|
||||
elif char == "/" and following == "/":
|
||||
masked[index] = masked[index + 1] = " "
|
||||
index += 1
|
||||
state = "line_comment"
|
||||
elif char == "/" and following == "*":
|
||||
masked[index] = masked[index + 1] = " "
|
||||
index += 1
|
||||
state = "block_comment"
|
||||
elif state in {"string", "char"}:
|
||||
if char == "\\":
|
||||
index += 1
|
||||
elif (state == "string" and char == '"') or (state == "char" and char == "'"):
|
||||
state = "code"
|
||||
elif state == "line_comment":
|
||||
if char == "\n":
|
||||
state = "code"
|
||||
else:
|
||||
masked[index] = " "
|
||||
elif state == "block_comment":
|
||||
if char == "*" and following == "/":
|
||||
masked[index] = masked[index + 1] = " "
|
||||
index += 1
|
||||
state = "code"
|
||||
elif char != "\n":
|
||||
masked[index] = " "
|
||||
index += 1
|
||||
return "".join(masked)
|
||||
|
||||
|
||||
def mask_rust_noncode(text: str) -> str:
|
||||
"""Blank Rust comments and string/character contents without shifting offsets."""
|
||||
masked = list(text)
|
||||
index = 0
|
||||
state = "code"
|
||||
block_depth = 0
|
||||
raw_hashes = 0
|
||||
while index < len(text):
|
||||
char = text[index]
|
||||
following = text[index + 1] if index + 1 < len(text) else ""
|
||||
if state == "code":
|
||||
raw = re.match(r'(?:b|c)?r(#{0,255})"', text[index:])
|
||||
if raw:
|
||||
raw_hashes = len(raw.group(1))
|
||||
for offset in range(len(raw.group(0))):
|
||||
masked[index + offset] = " "
|
||||
index += len(raw.group(0)) - 1
|
||||
state = "raw_string"
|
||||
elif char == '"':
|
||||
masked[index] = " "
|
||||
state = "string"
|
||||
elif char == "'" and re.match(r"'(?:\\.|[^\\'\n])'", text[index:]):
|
||||
masked[index] = " "
|
||||
state = "char"
|
||||
elif char == "/" and following == "/":
|
||||
masked[index] = masked[index + 1] = " "
|
||||
index += 1
|
||||
state = "line_comment"
|
||||
elif char == "/" and following == "*":
|
||||
masked[index] = masked[index + 1] = " "
|
||||
index += 1
|
||||
block_depth = 1
|
||||
state = "block_comment"
|
||||
elif state in {"string", "char"}:
|
||||
masked[index] = " "
|
||||
if char == "\\" and index + 1 < len(text):
|
||||
masked[index + 1] = " "
|
||||
index += 1
|
||||
elif (state == "string" and char == '"') or (state == "char" and char == "'"):
|
||||
state = "code"
|
||||
elif state == "raw_string":
|
||||
masked[index] = " "
|
||||
ending = '"' + "#" * raw_hashes
|
||||
if text.startswith(ending, index):
|
||||
for offset in range(len(ending)):
|
||||
masked[index + offset] = " "
|
||||
index += len(ending) - 1
|
||||
state = "code"
|
||||
elif state == "line_comment":
|
||||
if char == "\n":
|
||||
state = "code"
|
||||
else:
|
||||
masked[index] = " "
|
||||
elif state == "block_comment":
|
||||
masked[index] = " "
|
||||
if char == "/" and following == "*":
|
||||
masked[index + 1] = " "
|
||||
index += 1
|
||||
block_depth += 1
|
||||
elif char == "*" and following == "/":
|
||||
masked[index + 1] = " "
|
||||
index += 1
|
||||
block_depth -= 1
|
||||
if block_depth == 0:
|
||||
state = "code"
|
||||
index += 1
|
||||
return "".join(masked)
|
||||
|
||||
|
||||
def rust_test_body(text: str, rust_test: re.Match[str]) -> str:
|
||||
masked = mask_rust_noncode(text)
|
||||
brace = masked.find("{", rust_test.end())
|
||||
if brace < 0:
|
||||
raise RuntimeError(f"Rust test {rust_test.group(1)} has no body")
|
||||
depth = 0
|
||||
for index in range(brace, len(masked)):
|
||||
if masked[index] == "{":
|
||||
depth += 1
|
||||
elif masked[index] == "}":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return text[brace + 1 : index]
|
||||
raise RuntimeError(f"Rust test {rust_test.group(1)} has an unterminated body")
|
||||
|
||||
|
||||
def validate_reviewed_body(path: Path, rust_test: re.Match[str], body: str) -> str:
|
||||
code = mask_rust_noncode(body).strip()
|
||||
if not code:
|
||||
raise RuntimeError(f"Reviewed parity test has an empty body: {path}:{rust_test.group(1)}")
|
||||
forbidden = FORBIDDEN_REVIEW_BODY_RE.search(code)
|
||||
if forbidden:
|
||||
raise RuntimeError(
|
||||
f"Reviewed parity test uses forbidden placeholder/dispatcher `{forbidden.group(0)}`: "
|
||||
f"{path}:{rust_test.group(1)}"
|
||||
)
|
||||
if not re.search(r"\bassert(?:_[A-Za-z0-9_]+)?!|\.expect\s*\(|\.unwrap(?:_err)?\s*\(|\?", code):
|
||||
raise RuntimeError(
|
||||
f"Reviewed parity test has no explicit assertion or checked API result: "
|
||||
f"{path}:{rust_test.group(1)}"
|
||||
)
|
||||
return hashlib.sha256(body.encode()).hexdigest()
|
||||
|
||||
|
||||
def namespace_at(text: str, position: int) -> str:
|
||||
matches = [m for m in NAMESPACE_RE.finditer(text) if m.start() <= position]
|
||||
return matches[-1].group(1) if matches else ""
|
||||
@@ -200,9 +367,28 @@ def generate_apis(upstream: Path, output: Path) -> tuple[int, int]:
|
||||
return total_types, total_members
|
||||
|
||||
|
||||
def enclosing_class(text: str, position: int) -> str:
|
||||
matches = [m for m in CLASS_RE.finditer(text) if m.start() <= position]
|
||||
return matches[-1].group(1) if matches else "UnknownFixture"
|
||||
def enclosing_class_match(text: str, position: int) -> re.Match[str] | None:
|
||||
containing: list[re.Match[str]] = []
|
||||
for match in CLASS_RE.finditer(text, 0, position):
|
||||
brace = text.find("{", match.end())
|
||||
if brace == -1 or brace >= position:
|
||||
continue
|
||||
depth = 0
|
||||
for index in range(brace, len(text)):
|
||||
if text[index] == "{":
|
||||
depth += 1
|
||||
elif text[index] == "}":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
if position < index:
|
||||
containing.append(match)
|
||||
break
|
||||
return containing[-1] if containing else None
|
||||
|
||||
|
||||
def attributes_before(text: str, position: int) -> str:
|
||||
match = re.search(r"((?:\s*\[[^\]]+\]\s*)+)$", text[:position], re.DOTALL)
|
||||
return match.group(1) if match else ""
|
||||
|
||||
|
||||
def method_body_hash(text: str, method_start: int) -> str:
|
||||
@@ -227,76 +413,299 @@ def method_body_hash(text: str, method_start: int) -> str:
|
||||
return hashlib.sha256(body.encode()).hexdigest()
|
||||
|
||||
|
||||
def generate_tests(upstream: Path, output: Path) -> int:
|
||||
def stable_case_id(source: str, fixture: str, method: str, attribute: str, parameters: str | None) -> str:
|
||||
test_id = f"{source}::{fixture}.{method}"
|
||||
if attribute == "Test":
|
||||
return f"{test_id}::test"
|
||||
normalized = " ".join((parameters or "").split())
|
||||
digest = hashlib.sha256(normalized.encode()).hexdigest()[:16]
|
||||
return f"{test_id}::case:{digest}"
|
||||
|
||||
|
||||
def helper_types(roots: tuple[Path, ...], upstream: Path) -> dict[str, str]:
|
||||
helpers: dict[str, str] = {}
|
||||
for root in roots:
|
||||
for path in source_files(root):
|
||||
text = path.read_text(encoding="utf-8-sig")
|
||||
searchable = mask_csharp_comments(text)
|
||||
if TEST_ATTR_RE.search(searchable):
|
||||
continue
|
||||
relative = path.relative_to(upstream).as_posix()
|
||||
for match in CLASS_RE.finditer(searchable):
|
||||
helpers.setdefault(match.group(1), relative)
|
||||
return helpers
|
||||
|
||||
|
||||
def extract_tests(upstream: Path) -> list[dict[str, object]]:
|
||||
roots = (upstream / "LibreMetaverse.Tests", upstream / "LibreMetaverse.Rendering.Tests")
|
||||
helpers = helper_types(roots, upstream)
|
||||
tests: list[dict[str, object]] = []
|
||||
for root in roots:
|
||||
for path in source_files(root):
|
||||
text = path.read_text(encoding="utf-8-sig")
|
||||
searchable = mask_csharp_comments(text)
|
||||
relative = path.relative_to(upstream).as_posix()
|
||||
attrs = list(TEST_ATTR_RE.finditer(text))
|
||||
dependencies = sorted(
|
||||
helper_path
|
||||
for helper, helper_path in helpers.items()
|
||||
if helper_path != relative and re.search(rf"\b{re.escape(helper)}\b", text)
|
||||
)
|
||||
attrs = list(TEST_ATTR_RE.finditer(searchable))
|
||||
for attr_index, attr in enumerate(attrs):
|
||||
next_attr = attrs[attr_index + 1].start() if attr_index + 1 < len(attrs) else len(text)
|
||||
method = METHOD_RE.search(text, attr.end(), next_attr + 1200)
|
||||
method = METHOD_RE.search(searchable, attr.end(), next_attr + 1200)
|
||||
if method is None:
|
||||
method = METHOD_RE.search(text, attr.end())
|
||||
method = METHOD_RE.search(searchable, attr.end())
|
||||
if method is None:
|
||||
raise RuntimeError(f"No test method after {relative}:{text.count(chr(10), 0, attr.start()) + 1}")
|
||||
if attr.group(1) == "Test" and TEST_ATTR_RE.search(searchable, attr.end(), method.start()):
|
||||
continue
|
||||
name = method.group(1)
|
||||
fixture = enclosing_class(text, method.start())
|
||||
class_match = enclosing_class_match(searchable, method.start())
|
||||
fixture = class_match.group(1) if class_match else "UnknownFixture"
|
||||
attr_text = " ".join(attr.group(0).split())
|
||||
parameters = " ".join((attr.group(2) or "").split()) or None
|
||||
line = text.count("\n", 0, attr.start()) + 1
|
||||
category_text = searchable[attr.start() : method.start()]
|
||||
if class_match:
|
||||
category_text += attributes_before(searchable, class_match.start())
|
||||
categories = sorted(set(CATEGORY_RE.findall(category_text)))
|
||||
tests.append(
|
||||
{
|
||||
"id": stable_case_id(relative, fixture, name, attr.group(1), parameters),
|
||||
"csharp_test_id": f"{relative}::{fixture}.{name}",
|
||||
"parameter_case": parameters,
|
||||
"source": relative,
|
||||
"line": line,
|
||||
"fixture": fixture,
|
||||
"method": name,
|
||||
"attribute": attr_text,
|
||||
"body_sha256": method_body_hash(text, method.start()),
|
||||
"categories": categories,
|
||||
"fixture_dependencies": dependencies,
|
||||
}
|
||||
)
|
||||
seen: defaultdict[str, int] = defaultdict(int)
|
||||
ids = [str(test["id"]) for test in tests]
|
||||
duplicates = sorted(case_id for case_id, count in Counter(ids).items() if count > 1)
|
||||
if duplicates:
|
||||
raise RuntimeError("Duplicate stable NUnit case IDs: " + ", ".join(duplicates))
|
||||
return tests
|
||||
|
||||
|
||||
def reviewed_tests(root: Path) -> dict[str, dict[str, object]]:
|
||||
reviews: dict[str, dict[str, object]] = {}
|
||||
review_roots = (root / "tests" / "compat" / "tests", root / "crates")
|
||||
for path in sorted(path for review_root in review_roots for path in review_root.rglob("*.rs")):
|
||||
if path.name == "generated_parity.rs":
|
||||
continue
|
||||
text = path.read_text()
|
||||
for marker in PARITY_MARKER_RE.finditer(text):
|
||||
next_marker = PARITY_MARKER_RE.search(text, marker.end())
|
||||
rust_test = RUST_TEST_RE.search(text, marker.end(), next_marker.start() if next_marker else len(text))
|
||||
if rust_test is None:
|
||||
raise RuntimeError(f"Parity marker has no following Rust test: {path}:{text.count(chr(10), 0, marker.start()) + 1}")
|
||||
body_sha256 = validate_reviewed_body(path, rust_test, rust_test_body(text, rust_test))
|
||||
case_id = marker.group("id")
|
||||
if case_id in reviews:
|
||||
raise RuntimeError(f"Duplicate reviewed parity case: {case_id}")
|
||||
reviews[case_id] = {
|
||||
"body_sha256": marker.group("body_sha256"),
|
||||
"rust_body_sha256": body_sha256,
|
||||
"status": marker.group("status"),
|
||||
"rust_file": path.relative_to(root).as_posix(),
|
||||
"rust_line": text.count("\n", 0, marker.start()) + 1,
|
||||
"rust_test": rust_test.group(1),
|
||||
}
|
||||
return reviews
|
||||
|
||||
|
||||
def render_parity_report(tests: list[dict[str, object]]) -> str:
|
||||
status_counts = {status: sum(test["status"] == status for test in tests) for status in ("pending", "translated", "ignored-live", "benchmark", "drifted")}
|
||||
live_candidates = sum("RequiresLiveServer" in test["categories"] for test in tests)
|
||||
benchmark_candidates = sum("Benchmark" in test["categories"] for test in tests)
|
||||
lines = [
|
||||
"# NUnit to Rust parity ledger",
|
||||
"",
|
||||
f"Generated from LibreMetaverse `{UPSTREAM_COMMIT}`. Stable IDs identify one NUnit `[Test]` or `[TestCase]` invocation; the body hash covers the original C# method declaration and body.",
|
||||
"",
|
||||
"## Status",
|
||||
"",
|
||||
f"- Total: **{len(tests):,}**",
|
||||
f"- Pending/unreviewed: **{status_counts['pending']:,}**",
|
||||
f"- Translated/reviewed: **{status_counts['translated']:,}**",
|
||||
f"- Ignored live/reviewed: **{status_counts['ignored-live']:,}** ({live_candidates:,} upstream live candidates)",
|
||||
f"- Benchmarks/reviewed: **{status_counts['benchmark']:,}** ({benchmark_candidates:,} upstream benchmark candidates)",
|
||||
f"- Drifted: **{status_counts['drifted']:,}**",
|
||||
"",
|
||||
"Reviewed Rust tests live outside `generated_parity.rs` and carry a `parity-case` marker. Regeneration preserves those files and fails if their source body hash drifts.",
|
||||
"",
|
||||
"| Stable case ID | C# test | Parameter case | Source | Categories | Fixtures | Rust location | Status | Body SHA-256 |",
|
||||
"|---|---|---|---|---|---|---|---|---|",
|
||||
]
|
||||
for test in tests:
|
||||
parameter = str(test["parameter_case"] or "").replace("|", "|")
|
||||
categories = ", ".join(test["categories"])
|
||||
fixtures = ", ".join(test["fixture_dependencies"])
|
||||
rust_location = (
|
||||
f"{test['rust_file']}:{test['rust_line']} (`{test['rust_test']}`)"
|
||||
if test["status"] != "pending"
|
||||
else "pending source translation"
|
||||
)
|
||||
lines.append(
|
||||
f"| `{test['id']}` | `{test['fixture']}.{test['method']}` | `{parameter}` | `{test['source']}:{test['line']}` | `{categories}` | `{fixtures}` | `{rust_location}` | `{test['status']}` | `{test['body_sha256']}` |"
|
||||
)
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def generate_tests(upstream: Path, output: Path, review_root: Path | None = None) -> int:
|
||||
tests = extract_tests(upstream)
|
||||
reviews = reviewed_tests(review_root or output)
|
||||
tests_by_id = {str(test["id"]): test for test in tests}
|
||||
stale = sorted(set(reviews) - set(tests_by_id))
|
||||
if stale:
|
||||
raise RuntimeError("Reviewed cases missing from pinned upstream: " + ", ".join(stale))
|
||||
drifted = sorted(
|
||||
case_id
|
||||
for case_id, review in reviews.items()
|
||||
if review["body_sha256"] != tests_by_id[case_id]["body_sha256"]
|
||||
)
|
||||
if drifted:
|
||||
raise RuntimeError("Reviewed cases have drifted C# bodies: " + ", ".join(drifted))
|
||||
|
||||
rust_lines = [
|
||||
"// @generated by tools/generate_surface.py; do not edit by hand.",
|
||||
f"// Source: LibreMetaverse {UPSTREAM_COMMIT}",
|
||||
"use libremetaverse_compat_tests::pending;",
|
||||
"// Pending cases are catalog entries, not fake executable Rust tests.",
|
||||
"",
|
||||
]
|
||||
parity_lines = [
|
||||
"# NUnit to Rust parity ledger",
|
||||
"",
|
||||
f"Generated from LibreMetaverse `{UPSTREAM_COMMIT}`. Each row is one NUnit `[Test]` or `[TestCase]` invocation. The body hash covers the original C# method declaration and body.",
|
||||
"",
|
||||
"| Rust test | C# test | Source | Attribute | Body SHA-256 |",
|
||||
"|---|---|---|---|---|",
|
||||
]
|
||||
for test in tests:
|
||||
base = snake(f"{test['fixture']}_{test['method']}")
|
||||
seen[base] += 1
|
||||
rust_name = base if seen[base] == 1 else f"{base}_case_{seen[base]}"
|
||||
test["rust_test"] = rust_name
|
||||
identity = f"{test['fixture']}.{test['method']}"
|
||||
rust_lines.extend(
|
||||
[
|
||||
"#[test]",
|
||||
f"fn {rust_name}() {{",
|
||||
f" pending({json.dumps(test['source'])}, {test['line']}, {json.dumps(identity)}, {json.dumps(test['attribute'])}, {json.dumps(test['body_sha256'])});",
|
||||
"}",
|
||||
"",
|
||||
]
|
||||
)
|
||||
parity_lines.append(
|
||||
f"| `{rust_name}` | `{identity}` | `{test['source']}:{test['line']}` | `{str(test['attribute']).replace('|', '|')}` | `{test['body_sha256']}` |"
|
||||
review = reviews.get(str(test["id"]))
|
||||
if review:
|
||||
test.update(review)
|
||||
test["semantic_review"] = "reviewed"
|
||||
continue
|
||||
test.update(
|
||||
{
|
||||
"rust_file": "tests/compat/tests/generated_parity.rs",
|
||||
"rust_line": len(rust_lines) + 1,
|
||||
"rust_test": "",
|
||||
"rust_body_sha256": None,
|
||||
"status": "pending",
|
||||
"semantic_review": "unreviewed",
|
||||
}
|
||||
)
|
||||
rust_lines.append(f"// pending-parity-case: {test['id']} {test['body_sha256']}")
|
||||
generated_dir = output / "tests" / "compat" / "tests"
|
||||
generated_dir.mkdir(parents=True, exist_ok=True)
|
||||
(generated_dir / "generated_parity.rs").write_text("\n".join(rust_lines))
|
||||
(output / "tests" / "PARITY.md").write_text("\n".join(parity_lines) + "\n")
|
||||
(output / "tests" / "upstream-tests.json").write_text(json.dumps({"upstream_commit": UPSTREAM_COMMIT, "tests": tests}, indent=2) + "\n")
|
||||
(generated_dir / "generated_parity.rs").write_text("\n".join(rust_lines) + "\n")
|
||||
(output / "tests" / "PARITY.md").write_text(render_parity_report(tests))
|
||||
report = {
|
||||
"pending": sum(test["status"] == "pending" for test in tests),
|
||||
"translated": sum(test["status"] == "translated" for test in tests),
|
||||
"ignored_live": sum(test["status"] == "ignored-live" for test in tests),
|
||||
"benchmark": sum(test["status"] == "benchmark" for test in tests),
|
||||
"drifted": 0,
|
||||
"unreviewed": sum(test["semantic_review"] == "unreviewed" for test in tests),
|
||||
}
|
||||
catalog = {
|
||||
"schema_version": 2,
|
||||
"upstream_commit": UPSTREAM_COMMIT,
|
||||
"expected_cases": EXPECTED_TESTS,
|
||||
"report": report,
|
||||
"tests": tests,
|
||||
}
|
||||
(output / "tests" / "upstream-tests.json").write_text(json.dumps(catalog, indent=2) + "\n")
|
||||
return len(tests)
|
||||
|
||||
|
||||
def scan_parity_markers(root: Path) -> dict[str, list[dict[str, object]]]:
|
||||
return {case_id: [review] for case_id, review in reviewed_tests(root).items()}
|
||||
|
||||
|
||||
def check_test_parity(root: Path, require_reviewed: bool = False) -> dict[str, int]:
|
||||
catalog_path = root / "tests" / "upstream-tests.json"
|
||||
catalog = json.loads(catalog_path.read_text())
|
||||
if catalog.get("schema_version") != 2:
|
||||
raise RuntimeError("Unsupported test parity catalog schema")
|
||||
tests = catalog.get("tests", [])
|
||||
expected = catalog.get("expected_cases")
|
||||
if expected != EXPECTED_TESTS or len(tests) != EXPECTED_TESTS:
|
||||
raise RuntimeError(f"Expected {EXPECTED_TESTS} cataloged NUnit invocations, found {len(tests)}")
|
||||
if catalog.get("upstream_commit") != UPSTREAM_COMMIT:
|
||||
raise RuntimeError("Test parity catalog targets the wrong upstream commit")
|
||||
|
||||
ids = [test["id"] for test in tests]
|
||||
duplicate_ids = sorted(case_id for case_id, count in Counter(ids).items() if count > 1)
|
||||
if duplicate_ids:
|
||||
raise RuntimeError("Duplicate catalog case IDs: " + ", ".join(duplicate_ids))
|
||||
by_id = {test["id"]: test for test in tests}
|
||||
markers = scan_parity_markers(root)
|
||||
reviewed_ids = {case_id for case_id, test in by_id.items() if test["status"] != "pending"}
|
||||
missing = sorted(reviewed_ids - set(markers))
|
||||
stale = sorted(set(markers) - set(by_id))
|
||||
premature = sorted(set(markers) - reviewed_ids)
|
||||
duplicates = sorted(case_id for case_id, entries in markers.items() if len(entries) != 1)
|
||||
if missing:
|
||||
raise RuntimeError("Missing Rust parity cases: " + ", ".join(missing))
|
||||
if stale:
|
||||
raise RuntimeError("Stale Rust parity cases: " + ", ".join(stale))
|
||||
if premature:
|
||||
raise RuntimeError("Catalog still marks reviewed Rust cases pending: " + ", ".join(premature))
|
||||
if duplicates:
|
||||
raise RuntimeError("Duplicate Rust parity cases: " + ", ".join(duplicates))
|
||||
|
||||
drifted: list[str] = []
|
||||
mismatched: list[str] = []
|
||||
for case_id in reviewed_ids:
|
||||
test = by_id[case_id]
|
||||
marker = markers[case_id][0]
|
||||
if marker["body_sha256"] != test["body_sha256"]:
|
||||
drifted.append(case_id)
|
||||
for key in ("status", "rust_file", "rust_line", "rust_test", "rust_body_sha256"):
|
||||
if marker[key] != test[key]:
|
||||
mismatched.append(f"{case_id}:{key}")
|
||||
if drifted:
|
||||
raise RuntimeError("Drifted reviewed C# test bodies: " + ", ".join(drifted))
|
||||
if mismatched:
|
||||
raise RuntimeError("Stale parity catalog metadata: " + ", ".join(mismatched))
|
||||
|
||||
report = {
|
||||
"pending": sum(test["status"] == "pending" for test in tests),
|
||||
"translated": sum(test["status"] == "translated" for test in tests),
|
||||
"ignored_live": sum(test["status"] == "ignored-live" for test in tests),
|
||||
"benchmark": sum(test["status"] == "benchmark" for test in tests),
|
||||
"drifted": 0,
|
||||
"unreviewed": sum(test["semantic_review"] != "reviewed" for test in tests),
|
||||
}
|
||||
if catalog.get("report") != report:
|
||||
raise RuntimeError("Stale parity summary report")
|
||||
if require_reviewed and report["unreviewed"]:
|
||||
raise RuntimeError(f"{report['unreviewed']} NUnit invocations remain semantically unreviewed")
|
||||
return report
|
||||
|
||||
|
||||
def verify_upstream(upstream: Path) -> None:
|
||||
commit = subprocess.run(
|
||||
["git", "-C", str(upstream), "rev-parse", "HEAD"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout.strip()
|
||||
if commit != UPSTREAM_COMMIT:
|
||||
raise RuntimeError(f"Expected LibreMetaverse {UPSTREAM_COMMIT}, found {commit}")
|
||||
|
||||
|
||||
def check_test_regeneration(upstream: Path, output: Path) -> None:
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
generated = Path(temporary)
|
||||
count = generate_tests(upstream, generated, review_root=output)
|
||||
if count != EXPECTED_TESTS:
|
||||
raise RuntimeError(f"Expected {EXPECTED_TESTS} NUnit invocations at {UPSTREAM_COMMIT}, found {count}")
|
||||
stale = [str(path) for path in PARITY_FILES if not (output / path).exists() or (output / path).read_bytes() != (generated / path).read_bytes()]
|
||||
if stale:
|
||||
raise RuntimeError("Stale generated test parity files: " + ", ".join(stale))
|
||||
|
||||
|
||||
def generate_program_manifest(upstream: Path, output: Path) -> int:
|
||||
roots = [
|
||||
upstream / "Programs" / "VivoxTest",
|
||||
@@ -331,14 +740,28 @@ def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--upstream", type=Path, default=Path("../libremetaverse"))
|
||||
parser.add_argument("--output", type=Path, default=Path("."))
|
||||
parser.add_argument("--tests-only", action="store_true")
|
||||
parser.add_argument("--check", action="store_true")
|
||||
args = parser.parse_args()
|
||||
upstream = args.upstream.resolve()
|
||||
output = args.output.resolve()
|
||||
verify_upstream(upstream)
|
||||
if args.check:
|
||||
check_test_regeneration(upstream, output)
|
||||
report = check_test_parity(output)
|
||||
print(f"test parity is current: {report}")
|
||||
return
|
||||
if args.tests_only:
|
||||
tests = generate_tests(upstream, output)
|
||||
if tests != EXPECTED_TESTS:
|
||||
raise RuntimeError(f"Expected {EXPECTED_TESTS} NUnit invocations at {UPSTREAM_COMMIT}, found {tests}")
|
||||
print(f"generated {tests} test parity cases")
|
||||
return
|
||||
types, members = generate_apis(upstream, output)
|
||||
tests = generate_tests(upstream, output)
|
||||
programs = generate_program_manifest(upstream, output)
|
||||
if tests != 1295:
|
||||
raise RuntimeError(f"Expected 1295 NUnit invocations at {UPSTREAM_COMMIT}, found {tests}")
|
||||
if tests != EXPECTED_TESTS:
|
||||
raise RuntimeError(f"Expected {EXPECTED_TESTS} NUnit invocations at {UPSTREAM_COMMIT}, found {tests}")
|
||||
print(f"generated {types} type declarations, {members} public declaration lines, {tests} tests, and {programs} program manifests")
|
||||
|
||||
|
||||
|
||||
133
tools/test_generate_surface.py
Normal file
133
tools/test_generate_surface.py
Normal file
@@ -0,0 +1,133 @@
|
||||
import json
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from generate_surface import (
|
||||
check_test_parity,
|
||||
extract_tests,
|
||||
generate_tests,
|
||||
mask_csharp_comments,
|
||||
reviewed_tests,
|
||||
)
|
||||
|
||||
|
||||
class ParityGenerationTests(unittest.TestCase):
|
||||
def test_reviewed_tests_include_owning_crate_unit_tests(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
root = Path(temporary)
|
||||
rust_file = root / "crates" / "example" / "src" / "internal_semantics.rs"
|
||||
rust_file.parent.mkdir(parents=True)
|
||||
case_id = "Example.Tests.cs::ExampleTests.InternalMember::test"
|
||||
rust_file.write_text(
|
||||
f"// parity-case: {case_id} {'a' * 64} translated\n"
|
||||
"#[test]\nfn internal_member() {\n"
|
||||
" let fixture: &'static str = r#\"{ fixture }\"#;\n"
|
||||
" assert_eq!(fixture, \"{ fixture }\");\n"
|
||||
" Example::internal_member().expect(\"internal member\");\n"
|
||||
"}\n"
|
||||
)
|
||||
|
||||
review = reviewed_tests(root)[case_id]
|
||||
self.assertEqual(review["rust_file"], "crates/example/src/internal_semantics.rs")
|
||||
self.assertEqual(review["rust_test"], "internal_member")
|
||||
self.assertRegex(str(review["rust_body_sha256"]), r"^[0-9a-f]{64}$")
|
||||
|
||||
def test_comment_masking_preserves_offsets_and_ignores_fake_tests(self) -> None:
|
||||
source = '''var url = "https://example.com/[Test]"; // [Test]\n/* [TestCase(1)] */\n[Test]\n'''
|
||||
masked = mask_csharp_comments(source)
|
||||
self.assertEqual(len(masked), len(source))
|
||||
self.assertEqual(masked.count("\n"), source.count("\n"))
|
||||
self.assertIn('"https://example.com/[Test]"', masked)
|
||||
self.assertEqual(masked.count("[Test]"), 2)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
upstream = Path(temporary)
|
||||
tests = upstream / "LibreMetaverse.Tests"
|
||||
tests.mkdir()
|
||||
(upstream / "LibreMetaverse.Rendering.Tests").mkdir()
|
||||
(tests / "Comments.cs").write_text(
|
||||
"""public class Comments
|
||||
{
|
||||
// [Test]
|
||||
// public void CommentedOut() {}
|
||||
[Test]
|
||||
[TestCase(1)]
|
||||
[TestCase(2)]
|
||||
public void Cases(int value) {}
|
||||
}
|
||||
"""
|
||||
)
|
||||
self.assertEqual([case["parameter_case"] for case in extract_tests(upstream)], ["1", "2"])
|
||||
|
||||
def test_reviewed_rust_test_is_preserved_and_pending_cases_are_not_fake_tests(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
root = Path(temporary)
|
||||
upstream = root / "upstream"
|
||||
source = upstream / "LibreMetaverse.Tests" / "ExampleTests.cs"
|
||||
source.parent.mkdir(parents=True)
|
||||
(upstream / "LibreMetaverse.Rendering.Tests").mkdir()
|
||||
source.write_text(
|
||||
"""namespace LibreMetaverse.Tests;
|
||||
[Category("Example")]
|
||||
public class ExampleTests
|
||||
{
|
||||
[TestCase(1, Description = "one")]
|
||||
public void KeepsIdentity(int value) { Assert.That(value, Is.EqualTo(1)); }
|
||||
[Test]
|
||||
public void RemainsPending() { Assert.Pass(); }
|
||||
}
|
||||
"""
|
||||
)
|
||||
reviewed, _pending = extract_tests(upstream)
|
||||
output = root / "output"
|
||||
rust_file = output / "tests" / "compat" / "tests" / "example.rs"
|
||||
rust_file.parent.mkdir(parents=True)
|
||||
rust_file.write_text(
|
||||
f"// parity-case: {reviewed['id']} {reviewed['body_sha256']} translated\n"
|
||||
"#[test]\nfn translated_case() {\n"
|
||||
" let actual = Example::keeps_identity(1).expect(\"KeepsIdentity\");\n"
|
||||
" assert_eq!(actual, 1);\n"
|
||||
"}\n"
|
||||
)
|
||||
original = rust_file.read_bytes()
|
||||
|
||||
with patch("generate_surface.EXPECTED_TESTS", 2):
|
||||
self.assertEqual(generate_tests(upstream, output), 2)
|
||||
self.assertEqual(rust_file.read_bytes(), original)
|
||||
generated = output / "tests/compat/tests/generated_parity.rs"
|
||||
self.assertNotIn(reviewed["id"], generated.read_text())
|
||||
self.assertNotIn("#[test]", generated.read_text())
|
||||
self.assertNotIn("pending(", generated.read_text())
|
||||
catalog = json.loads((output / "tests/upstream-tests.json").read_text())
|
||||
self.assertEqual(catalog["tests"][0]["status"], "translated")
|
||||
self.assertEqual(catalog["tests"][0]["rust_test"], "translated_case")
|
||||
self.assertEqual(check_test_parity(output)["unreviewed"], 1)
|
||||
|
||||
rust_file.unlink()
|
||||
with self.assertRaisesRegex(RuntimeError, "Missing Rust parity cases"):
|
||||
check_test_parity(output)
|
||||
|
||||
def test_placeholder_and_dispatcher_bodies_cannot_count_as_reviewed(self) -> None:
|
||||
bodies = [
|
||||
"",
|
||||
' pending("case");\n',
|
||||
' assert_reviewed_case("Source.cs", "Method");\n',
|
||||
' libremetaverse_types::unimplemented_api!("member");\n',
|
||||
]
|
||||
for body in bodies:
|
||||
with self.subTest(body=body), tempfile.TemporaryDirectory() as temporary:
|
||||
root = Path(temporary)
|
||||
rust_file = root / "tests" / "compat" / "tests" / "invalid.rs"
|
||||
rust_file.parent.mkdir(parents=True)
|
||||
rust_file.write_text(
|
||||
f"// parity-case: Example.cs::Example.Invalid::test {'a' * 64} translated\n"
|
||||
f"#[test]\nfn invalid() {{\n{body}}}\n"
|
||||
)
|
||||
with self.assertRaisesRegex(RuntimeError, "empty body|forbidden placeholder/dispatcher"):
|
||||
reviewed_tests(root)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user