Codegen: establish deterministic generator framework and source provenance #45

New Issue

hugo · 2026-08-08T08:18:17Z

hugo commented

2026-08-08 08:18:17 +00:00

Objective

Create a native Rust code-generation toolchain for the pinned LibreMetaverse data inputs without requiring Roslyn or .NET in normal Rust builds.

Deliverables

Inventory every input consumed by the six C# source generators and record hash/license/provenance.
Define deterministic parsing, diagnostics, output formatting, generated-file headers, and explicit regeneration commands.
Keep generated Rust checked in so consumers need only Rust/native prerequisites.

Validation

Clean regeneration succeeds from documented inputs.
Two runs are byte-identical.
Normal cargo build does not execute .NET or access the sibling checkout.

Prerequisite

The fixed API/test gates, Types, StructuredData, and relevant imaging foundations are complete.

Project constraints

All LibreMetaverse-specific generation and wire behavior is new native Rust code.
Generated Rust is deterministic, checked in, and covered by the fixed public API and semantic tests.
No normal build or release may depend on .NET, Roslyn, C# assemblies, or the reference checkout.

## Objective Create a native Rust code-generation toolchain for the pinned LibreMetaverse data inputs without requiring Roslyn or .NET in normal Rust builds. ## Deliverables - Inventory every input consumed by the six C# source generators and record hash/license/provenance. - Define deterministic parsing, diagnostics, output formatting, generated-file headers, and explicit regeneration commands. - Keep generated Rust checked in so consumers need only Rust/native prerequisites. ## Validation - Clean regeneration succeeds from documented inputs. - Two runs are byte-identical. - Normal cargo build does not execute .NET or access the sibling checkout. ## Prerequisite The fixed API/test gates, Types, StructuredData, and relevant imaging foundations are complete. ## Project constraints - All LibreMetaverse-specific generation and wire behavior is new native Rust code. - Generated Rust is deterministic, checked in, and covered by the fixed public API and semantic tests. - No normal build or release may depend on .NET, Roslyn, C# assemblies, or the reference checkout.

hugo added this to the 07 - Wire codegen milestone 2026-08-08 08:18:17 +00:00

hugo added the enhancement label 2026-08-08 08:18:17 +00:00

hugo referenced this issue from a commit

2026-08-09 05:26:57 +00:00

Establish native deterministic codegen framework (#45)

hugo commented

2026-08-09 05:27:17 +00:00

Implemented and pushed as 9f7a4e3.

Native Rust code-generation framework:

Added the libremetaverse-codegen workspace tool with generate, check, and hash-gated vendor commands; no build script or consumer crate executes it.
Inventoried all six pinned C# generator sources and all eight actual AdditionalFile inputs, recording upstream repository/commit, BSD-3-Clause license, SHA-256, format, reference path, and vendored path. The absent optional visualparamtemplate.cs case and embedded golden template are explicitly documented through the VisualParam generator source hash.
Vendored exact pinned message template, avatar LAD/skeleton, attention tables, genepool, tree, and grass inputs after verifying both generator and input hashes.
Added deterministic UTF-8/BOM/line-ending normalization, NUL/UTF-8 path-line-column diagnostics, safe relative-path validation, stable ordering, provenance-rich generated headers, and single-LF output.
Checked in the generated Rust source/generator manifest and documented all regeneration and provenance commands.
Added Linux/macOS/Windows CI that tests byte identity, checks vendored hashes/stale output, regenerates twice, lints the native tool, and proves a locked normal consumer build does not run generation.

Verification:

Two independent filesystem regenerations produced identical SHA-256 0fbe663c4ed73bd1d88e81bb179b6fe15e8c8115b1f914551970fcc7e10daf9b.
3 native framework tests pass, including malformed inventory/path/hash/reference cases and deterministic diagnostics.
cargo run -p libremetaverse-codegen -- check passes using checked-in inputs only.
cargo build -p libremetaverse --locked passes without reference checkout access or generator execution.
Controlled workspace audit: 271 passed, 1155 expected standardized failures, 0 ignored/unexpected.
Formatting, mapping/shim generation, API coverage, reviewed parity, milestone-06 audit, API compile, default/all-feature all-target checks, default/all-feature Clippy with warnings denied, rustdoc with warnings denied, default/all-feature workspace no-run, and package listing all passed.

Reviewed against issue #45 and the pinned generator implementations. The framework is complete and ready for packet and XML data generators in the next milestone-07 issues.

Implemented and pushed as 9f7a4e3. Native Rust code-generation framework: - Added the libremetaverse-codegen workspace tool with generate, check, and hash-gated vendor commands; no build script or consumer crate executes it. - Inventoried all six pinned C# generator sources and all eight actual AdditionalFile inputs, recording upstream repository/commit, BSD-3-Clause license, SHA-256, format, reference path, and vendored path. The absent optional visualparamtemplate.cs case and embedded golden template are explicitly documented through the VisualParam generator source hash. - Vendored exact pinned message template, avatar LAD/skeleton, attention tables, genepool, tree, and grass inputs after verifying both generator and input hashes. - Added deterministic UTF-8/BOM/line-ending normalization, NUL/UTF-8 path-line-column diagnostics, safe relative-path validation, stable ordering, provenance-rich generated headers, and single-LF output. - Checked in the generated Rust source/generator manifest and documented all regeneration and provenance commands. - Added Linux/macOS/Windows CI that tests byte identity, checks vendored hashes/stale output, regenerates twice, lints the native tool, and proves a locked normal consumer build does not run generation. Verification: - Two independent filesystem regenerations produced identical SHA-256 0fbe663c4ed73bd1d88e81bb179b6fe15e8c8115b1f914551970fcc7e10daf9b. - 3 native framework tests pass, including malformed inventory/path/hash/reference cases and deterministic diagnostics. - cargo run -p libremetaverse-codegen -- check passes using checked-in inputs only. - cargo build -p libremetaverse --locked passes without reference checkout access or generator execution. - Controlled workspace audit: 271 passed, 1155 expected standardized failures, 0 ignored/unexpected. - Formatting, mapping/shim generation, API coverage, reviewed parity, milestone-06 audit, API compile, default/all-feature all-target checks, default/all-feature Clippy with warnings denied, rustdoc with warnings denied, default/all-feature workspace no-run, and package listing all passed. Reviewed against issue #45 and the pinned generator implementations. The framework is complete and ready for packet and XML data generators in the next milestone-07 issues.

hugo closed this issue

2026-08-09 05:28:14 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/MetaCrate#45