Benchmark Rust against pinned C# reference (#105)
Some checks failed
API and SemVer surface / api-surface (push) Failing after 1m8s
Native code generation / deterministic (push) Failing after 2m6s
Concurrency and resource soak audit / soak (push) Failing after 12m13s
Documentation / documentation (push) Failing after 1m36s
Imaging and meshing gate / native (push) Failing after 3m2s
JPEG 2000 feature / linux (push) Successful in 2m48s
performance evidence / audit (push) Failing after 13m49s
Release platform and feature matrix / audit (push) Successful in 44s
Native Rust workspace compile / compile (push) Failing after 55s
Skia feature / linux (push) Successful in 31m13s
Dependency and supply-chain audit / audit (push) Failing after 9m13s
Release platform and feature matrix / matrix (false, linux-stable-minimal, x86_64-unknown-linux-gnu, stable) (push) Failing after 9m49s
Release platform and feature matrix / matrix (false, windows-stable-portable, x86_64-pc-windows-gnu, stable) (push) Has been cancelled
Release platform and feature matrix / matrix (true, linux-msrv-portable, x86_64-unknown-linux-gnu, 1.96.0) (push) Has been cancelled
Release platform and feature matrix / matrix (true, linux-stable-default, x86_64-unknown-linux-gnu, stable) (push) Has been cancelled
Release platform and feature matrix / matrix (true, linux-stable-features, x86_64-unknown-linux-gnu, stable) (push) Has been cancelled
Release platform and feature matrix / matrix (true, linux-stable-release-surface, x86_64-unknown-linux-gnu, stable) (push) Has been cancelled
Release platform and feature matrix / matrix (false, macos-stable-portable, x86_64-apple-darwin, stable) (push) Has been cancelled

This commit is contained in:
2026-08-12 03:26:56 +00:00
parent b71386dc31
commit 738fe3933e
27 changed files with 4746 additions and 21 deletions

90
benchmarks/README.md Normal file
View File

@@ -0,0 +1,90 @@
# Pinned cross-runtime performance evidence
This directory compares representative offline work in native Rust with the
pinned C# source at commit `2aa70bb68513b39795da5d13c88f31b86e85a3ba`.
The runners are separate executables. The Rust executable never starts a CLR,
loads a managed assembly, accesses a grid, or reads `.env`.
## Workloads and fixtures
`fixtures/manifest.json` records SHA-256 for every shared input. Both runners
refuse a changed fixture. The suite measures UUID/matrix/quaternion operations,
all five supported LLSD codecs, packet encode/decode, notecard/TGA/mesh decode,
inventory and object updates, simple rendering, and a batched offline client
message pipeline. Each report contains one cold operation and seven warm
samples. It records latency, throughput, allocated bytes, retained heap bytes,
and allocation count where the runtime exposes it. .NET reports a null
allocation count because its supported in-process GC API exposes bytes, not
operation count; it does not substitute a misleading zero.
The checked-in inputs are deliberately modest so the suite completes on a
four-core, 8 GiB release worker. Iteration counts live in `workloads.json`.
Regenerate binary inputs only after an intentional fixture review:
```sh
cargo run -p metacrate-performance --profile benchmark -- \
fixtures --fixture-root benchmarks/fixtures
```
## Reproducing the reports
Use the same otherwise-idle machine for both runs. Disable adaptive power or
thermal throttling where the host permits it. The committed evidence was
captured on Linux x86-64 with four AMD EPYC-Genoa vCPUs and 7.6 GiB RAM, Rust
1.97.1, .NET SDK 10.0.400, and the .NET 8.0.30 runtime. C# is built in Release
for `net8.0`. Rust uses the repository's `benchmark` profile (opt-level 1,
debug/incremental disabled): higher optimization of the generated core exceeds
the memory limit on this release worker, so this is a conservative Rust result.
```sh
REFERENCE_ROOT=/absolute/path/to/libremetaverse
test "$(git -C "$REFERENCE_ROOT" rev-parse HEAD)" = \
2aa70bb68513b39795da5d13c88f31b86e85a3ba
dotnet build benchmarks/csharp-reference/MetaCrate.ReferenceBenchmarks.csproj \
-c Release -p:ReferenceRoot="$REFERENCE_ROOT"
dotnet benchmarks/csharp-reference/bin/Release/net8.0/MetaCrate.ReferenceBenchmarks.dll \
run --reference-root "$REFERENCE_ROOT" \
--fixture-root benchmarks/fixtures \
--output benchmarks/results/csharp-linux-x86_64.json
cargo run -p metacrate-performance --profile benchmark -- \
run --fixture-root benchmarks/fixtures \
--output benchmarks/results/rust-linux-x86_64.json
cargo run -p metacrate-performance --profile benchmark -- \
compare --rust benchmarks/results/rust-linux-x86_64.json \
--reference benchmarks/results/csharp-linux-x86_64.json \
--output benchmarks/results/comparison.json
cargo run -p metacrate-performance --profile benchmark -- \
audit --fixture-root benchmarks/fixtures \
--rust benchmarks/results/rust-linux-x86_64.json \
--reference benchmarks/results/csharp-linux-x86_64.json \
--comparison benchmarks/results/comparison.json
```
The C# runner executes `git rev-parse` itself and rejects any other source
commit. The comparison rejects different fixture hashes, operating systems, or
architectures.
## Release criteria and reviewed differences
`release-criteria.json` is executable policy. A warm median at least 25% slower
than the reference is material and must be reviewed; the hard default is no
more than 2.0x latency or allocated bytes. Reference operations below one
microsecond are latency-noise exempt, but not allocation exempt.
Three workload-specific differences are accepted:
- Rust LLSD JSON owns and validates the bounded decoded tree. It is over four
times faster in this evidence, so up to 2.1x allocated bytes is accepted.
- Rust rendering performs the completed behavior: six faces, 24 vertices, 36
indices, checked normals, UVs, extents, and materials. The pinned C#
`SimpleRenderer` still returns a one-face/eight-vertex placeholder cube.
Slowing Rust down by removing correctness would violate the milestone, so the
policy accepts up to 8x latency and 4x allocated bytes for this workload.
- Rust client throughput returns checked `Result` values and owns the bounded
OSD map throughout the message pipeline. It allocates fewer bytes than C#;
its measured 1.64x latency is reviewed up to the unchanged 2x hard limit.
No exception permits changed wire data, decoded values, fixture output, or
public behavior. Any other threshold failure blocks the audit.