hugo/MetaCrate
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

Implement native LSL parser runtime (#82)
Some checks failed
Native code generation / deterministic (push) Failing after 11m53s
Details
Native Rust workspace compile / compile (push) Failing after 1m57s
Details

Browse Source
This commit is contained in:
Chili Palmer
2026-08-11 02:45:31 +00:00
parent 73cef10054
commit 656d837778
11 changed files with 4008 additions and 1200 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
crates/libremetaverse-lsl-tools/README.md
View File

@@ -4,9 +4,10 @@
parser-generator runtime in the pinned `LibreMetaverse.LslTools` assembly. The
issue 81 boundary supplies source reading, comments, Unicode character sets,
deterministic DFA execution, reserved words, terminal symbols, EOF, source
locations, diagnostics, and Rust iterators. Grammar productions, reductions,
and recovery are added by issue 82; deterministic generated grammar and token
tables are added by issue 83.
locations, diagnostics, and Rust iterators. The issue 82 boundary supplies
grammar productions, parser tables, precedence, reductions, and recovery.
Deterministic checked-in generated grammar and token tables are added by issue
83.
## Source and position contract
@@ -53,6 +54,37 @@ after a diagnostic and never yield a fabricated token. Token and symbol
tables and populate the supplied parser-entry priority. Missing or malformed
table data is an explicit positioned error.
## Grammar and parser contract
`Grammar` builds validated canonical LR(0) item sets with deterministic LALR(1)
lookahead propagation. Symbols and productions retain stable numeric
identities. Table construction rejects missing start/EOF declarations,
undeclared right-hand-side symbols, invalid precedence declarations, oversized
state sets, incomplete goto tables, and reduction underflow. Empty productions
and recursive nonterminals are supported.
Shift/reduce conflicts use yacc-compatible rules: a declared higher precedence
wins, equal left precedence reduces, equal right precedence shifts, and equal
nonassociative precedence installs a rejecting table entry. Undeclared
shift/reduce conflicts shift; reduce/reduce conflicts select the lower
production number. `YyParser::conflicts` preserves every decision and `emit`
writes a byte-stable table description independent of hash iteration.
`Parser` consumes the native lexer and returns a typed `ParseTree` through the
mapped `SYMBOL` semantic value. Syntax recovery uses the conventional terminal
number zero: it pops to a state that can shift `error`, shifts an explicit error
node, and discards input to the next valid lookahead. Diagnostics distinguish
the original syntax error from successful recovery. Parsing is bounded to
65,536 states, 1,048,576 live stack entries, 16,777,216 operations, and 1,000
recovery attempts.
Compatibility `SymbolSet` clones share deterministic FIRST/FOLLOW membership,
and cloned `CSymbol` values share registered production metadata. This retains
the observable reference behavior of the C# grammar model without unsafe code.
Parser-source serialization and the old `Builder`-driven generated-output
members return `InvalidOperation` explicitly until issue 83 supplies the
checked-in generator; they never report a false success.
## Diagnostics and migration
`ErrorHandler` collects structured `Diagnostic` values. Each diagnostic has a
@@ -73,6 +105,9 @@ The principal API mapping is:
| `Lexer._Enumerator` | `LexerEnumerator`; prefer `LexerIterator` |
| `Charset`, `CatTest` | Same mapped names plus `DotNetUnicodeCategory` |
| `CSToolsException`, `ErrorHandler` | Same mapped names plus structured `Diagnostic` |
| `CSymbol`, `SymbolSet`, `Production`, `Precedence` | Same mapped names plus typed `Grammar` builders |
| `YyParser`, parser entries | Deterministic native tables and conflict records |
| `Parser`, `ParseStackEntry`, `Error`, `recoveredError` | Bounded parsing, typed trees, and recovery |
No C#, .NET runtime, dynamically loaded class, macOS-only API, platform code
page, or runtime source generation is used by this boundary.
@@ -87,8 +122,9 @@ CARGO_BUILD_JOBS=1 cargo check --manifest-path tests/api-compile/Cargo.toml --lo
CARGO_BUILD_JOBS=1 cargo clippy -p libremetaverse-lsl-tools --all-targets --locked -- -D warnings
RUSTDOCFLAGS='-D warnings' CARGO_BUILD_JOBS=1 cargo doc -p libremetaverse-lsl-tools --no-deps --locked
python3 tools/check_milestone_10_issue_81.py
python3 tools/check_milestone_10_issue_82.py
python3 tools/generate_api_shims.py --check
```
The package contains 27 focused native and compatibility fixtures. The Gitea
The package contains 37 focused native and compatibility fixtures. The Gitea
workflow runs the audit and workspace compile on `ubuntu-latest` only.