LSL tools: deterministic generated parser output and API cleanup #83

New Issue

hugo · 2026-08-08T08:23:18Z

hugo commented

2026-08-08 08:23:18 +00:00

Objective

Replace hand-maintained generated C# class-shaped shims with deterministic Rust parser/token table generation.

Deliverables

Generate checked-in Rust tables/types from reviewed grammar inputs.
Prove full mapped public API coverage or document deliberate idiomatic replacements with migration paths.
Remove obsolete generated type shims and add regeneration CI.

Validation

Two generations are byte-identical.
No C# parser generator or .NET runtime is required.
LSL crate builds warning-free and all translated tests pass.

Prerequisite

All global API/test gates and the foundational/world implementations needed by these extension crates are complete.

Project constraints

Full native Rust implementation only; no wrapped C# parser, viewer, renderer, or service.
Preserve mapped APIs and exact translated tests; reuse lower layers and keep generated output deterministic.
Validate bounded input, callbacks/tasks, docs, coverage, and workspace gates.

## Objective Replace hand-maintained generated C# class-shaped shims with deterministic Rust parser/token table generation. ## Deliverables - Generate checked-in Rust tables/types from reviewed grammar inputs. - Prove full mapped public API coverage or document deliberate idiomatic replacements with migration paths. - Remove obsolete generated type shims and add regeneration CI. ## Validation - Two generations are byte-identical. - No C# parser generator or .NET runtime is required. - LSL crate builds warning-free and all translated tests pass. ## Prerequisite All global API/test gates and the foundational/world implementations needed by these extension crates are complete. ## Project constraints - Full native Rust implementation only; no wrapped C# parser, viewer, renderer, or service. - Preserve mapped APIs and exact translated tests; reuse lower layers and keep generated output deterministic. - Validate bounded input, callbacks/tasks, docs, coverage, and workspace gates.

hugo added this to the 10 - Rendering, RLV, and LSL tools milestone 2026-08-08 08:23:18 +00:00

hugo added the enhancement label 2026-08-08 08:23:18 +00:00

hugo referenced this issue

2026-08-11 01:55:51 +00:00

LSL tools: implement token model, lexer, source locations, and diagnostics #81

hugo referenced this issue

2026-08-11 02:46:19 +00:00

LSL tools: implement grammar, parser tables, precedence, and recovery #82

hugo referenced this issue from a commit

2026-08-11 03:36:28 +00:00

Generate native LSL parser tables (#83)

hugo commented

2026-08-11 03:36:58 +00:00

Implemented and pushed in commit 2558e2f.

Implementation:

Added the reviewed BSD-3-Clause grammar input pinned to upstream commit 2aa70bb68513b39795da5d13c88f31b86e85a3ba, covering 14 tokens, 8 nonterminals, and all 25 productions.
Added deterministic checked-in Rust parser/token generation with rustfmt-normalized output, generated lexer/parser factories, 84 syntax classes, 13 token classes, and the four cs0 wrappers.
Replaced every remaining LSL generated shim with native Rust mappings: 164/164 types and 768/768 members now have native coverage; issue-owned generation covers 120 types and 365 members.
Implemented the generator compatibility layer in Rust, including bounded regex matching, epsilon-closed NFA subset determinization, deterministic 4.5 serialization, delegates with callback completion, factories, symbol/token generator state, and migration APIs.
Added regeneration/audit CI on ubuntu-latest and documented the source, deterministic contract, and idiomatic migration paths.

Verification:

Two clean generations are byte-identical and the checked-in output passes the regeneration check.
All 46 libremetaverse-lsl-tools tests pass, including 9 issue-focused generator fixtures.
Strict Clippy and rustdoc warning gates pass.
The complete pre-created API compile compatibility fixture passes with the refreshed locked dependency graph.
The issue audit and generated API shim checks pass; no C# parser generator or .NET runtime is used.

A final review against the current issue deliverables and validation criteria found the implementation complete.

Implemented and pushed in commit 2558e2f. Implementation: - Added the reviewed BSD-3-Clause grammar input pinned to upstream commit 2aa70bb68513b39795da5d13c88f31b86e85a3ba, covering 14 tokens, 8 nonterminals, and all 25 productions. - Added deterministic checked-in Rust parser/token generation with rustfmt-normalized output, generated lexer/parser factories, 84 syntax classes, 13 token classes, and the four cs0 wrappers. - Replaced every remaining LSL generated shim with native Rust mappings: 164/164 types and 768/768 members now have native coverage; issue-owned generation covers 120 types and 365 members. - Implemented the generator compatibility layer in Rust, including bounded regex matching, epsilon-closed NFA subset determinization, deterministic 4.5 serialization, delegates with callback completion, factories, symbol/token generator state, and migration APIs. - Added regeneration/audit CI on ubuntu-latest and documented the source, deterministic contract, and idiomatic migration paths. Verification: - Two clean generations are byte-identical and the checked-in output passes the regeneration check. - All 46 libremetaverse-lsl-tools tests pass, including 9 issue-focused generator fixtures. - Strict Clippy and rustdoc warning gates pass. - The complete pre-created API compile compatibility fixture passes with the refreshed locked dependency graph. - The issue audit and generated API shim checks pass; no C# parser generator or .NET runtime is used. A final review against the current issue deliverables and validation criteria found the implementation complete.

hugo closed this issue

2026-08-11 03:37:10 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/MetaCrate#83