Generate native LSL parser tables (#83)

crates/libremetaverse-lsl-tools/README.md

@@ -6,8 +6,8 @@ issue 81 boundary supplies source reading, comments, Unicode character sets,
 deterministic DFA execution, reserved words, terminal symbols, EOF, source
 locations, diagnostics, and Rust iterators. The issue 82 boundary supplies
 grammar productions, parser tables, precedence, reductions, and recovery.
-Deterministic checked-in generated grammar and token tables are added by issue
-83.
+The issue 83 boundary adds deterministic checked-in grammar/token tables and
+native replacements for the retired parser-generator object model.
 
 ## Source and position contract
 
@@ -81,9 +81,35 @@ recovery attempts.
 Compatibility `SymbolSet` clones share deterministic FIRST/FOLLOW membership,
 and cloned `CSymbol` values share registered production metadata. This retains
 the observable reference behavior of the C# grammar model without unsafe code.
-Parser-source serialization and the old `Builder`-driven generated-output
-members return `InvalidOperation` explicitly until issue 83 supplies the
-checked-in generator; they never report a false success.
+Legacy parser-runtime serialization members that cannot faithfully round-trip
+the richer native LALR machine remain explicit `InvalidOperation` results.
+Callers should use the stable `YyParser::emit` and `YyLexer::emit_dfa` formats,
+or regenerate the canonical checked-in table module described below.
+
+## Checked-in generator contract
+
+[`codegen/inputs/lsl_tools_grammar.json`](../../codegen/inputs/lsl_tools_grammar.json)
+is the reviewed source of truth reconstructed from the pinned 4.5
+`yycs0syntax.cs` and `yycs0tokens.cs` tables at upstream commit
+`2aa70bb68513b39795da5d13c88f31b86e85a3ba`. It records every token,
+nonterminal, production, and semantic-action class. The portable Python
+generator uses the standard library plus the workspace Rust formatter and emits
+`src/generated_tables.rs`; it does not invoke C#, a C# source generator, or a
+.NET runtime.
+
+Regenerate with `python3 tools/generate_lsl_tables.py`. Use
+`python3 tools/generate_lsl_tables.py --check` in reviews and CI. The check
+renders the complete output in memory and compares bytes, so repeated runs are
+independent of hash iteration, locale, host operating system, and timestamps.
+The checked-in module supplies the canonical 25-production grammar, a Unicode
+lexer with reserved `base`, `this`, and `new` tokens, the 84 generated syntax
+classes, the 13 generated token classes, and the four `cs0` runtime wrappers.
+
+The old `GenBase`, `SymbolsGen`, `TokensGen`, `Regex`, NFA, delegate, factory,
+and 4.5 `Serialiser` APIs are live Rust compatibility adapters. New code should
+prefer `Grammar`, typed `DfaState` builders, `generated_parser`, and
+`generated_lexer`; these avoid CLR-style method pointers and class-shaped
+generated inheritance while retaining deterministic mapped behavior.
 
 ## Diagnostics and migration
 
@@ -108,6 +134,9 @@ The principal API mapping is:
 | `CSymbol`, `SymbolSet`, `Production`, `Precedence` | Same mapped names plus typed `Grammar` builders |
 | `YyParser`, parser entries | Deterministic native tables and conflict records |
 | `Parser`, `ParseStackEntry`, `Error`, `recoveredError` | Bounded parsing, typed trees, and recovery |
+| `yycs0syntax`, `yycs0tokens` | `generated_parser`, `generated_lexer`, checked-in native types |
+| `GenBase`, `SymbolsGen`, `TokensGen`, `Regex`, `Nfa` | Portable Rust generator adapters; prefer typed grammar/DFA builders |
+| `Serialiser` | Deterministic 4.5-compatible integer stream for supported mapped values |
 
 No C#, .NET runtime, dynamically loaded class, macOS-only API, platform code
 page, or runtime source generation is used by this boundary.
@@ -123,8 +152,10 @@ CARGO_BUILD_JOBS=1 cargo clippy -p libremetaverse-lsl-tools --all-targets --lock
 RUSTDOCFLAGS='-D warnings' CARGO_BUILD_JOBS=1 cargo doc -p libremetaverse-lsl-tools --no-deps --locked
 python3 tools/check_milestone_10_issue_81.py
 python3 tools/check_milestone_10_issue_82.py
+python3 tools/check_milestone_10_issue_83.py
+python3 tools/generate_lsl_tables.py --check
 python3 tools/generate_api_shims.py --check
 ```
 
-The package contains 37 focused native and compatibility fixtures. The Gitea
+The package contains 46 focused native and compatibility fixtures. The Gitea
 workflow runs the audit and workspace compile on `ubuntu-latest` only.