Implement native LSL parser runtime (#82)

.gitea/workflows/rust-workspace.yml

@@ -13,6 +13,7 @@ on:
       - "tools/check_milestone_10_issue_79.py"
       - "tools/check_milestone_10_issue_80.py"
       - "tools/check_milestone_10_issue_81.py"
+      - "tools/check_milestone_10_issue_82.py"
       - "**/*.rs"
       - "**/Cargo.toml"
       - "Cargo.lock"
@@ -28,6 +29,7 @@ on:
       - "tools/check_milestone_10_issue_79.py"
       - "tools/check_milestone_10_issue_80.py"
       - "tools/check_milestone_10_issue_81.py"
+      - "tools/check_milestone_10_issue_82.py"
       - "**/*.rs"
       - "**/Cargo.toml"
       - "Cargo.lock"
@@ -69,6 +71,7 @@ jobs:
           python3 tools/check_milestone_10_issue_79.py
           python3 tools/check_milestone_10_issue_80.py
           python3 tools/check_milestone_10_issue_81.py
+          python3 tools/check_milestone_10_issue_82.py
       - name: Test the complete native world milestone
         run: python3 tools/test_milestone_09.py
       - name: Compile every workspace target with bounded memory

api/SHIM-COVERAGE.md

@@ -7,7 +7,7 @@ Generated by `python3 tools/generate_api_shims.py`; do not edit by hand.
 | `LibreMetaverse` | 2,711 | 27,281 | native implementation: 401 types / 17,025 members; remaining surface is callable failure-only shims |
 | `LibreMetaverse.Imaging.Abstractions` | 3 | 20 | native implementation: 3 types / 20 members; no generated shims remain |
 | `LibreMetaverse.Imaging.Skia` | 1 | 3 | native implementation: 1 type / 3 members; no generated shims remain |
-| `LibreMetaverse.LslTools` | 164 | 768 | native implementation: 22 types / 188 members; remaining surface is callable failure-only shims |
+| `LibreMetaverse.LslTools` | 164 | 768 | native implementation: 44 types / 403 members; remaining surface is callable failure-only shims |
 | `LibreMetaverse.PrimMesher` | 17 | 207 | native implementation: 15 types / 200 members; remaining surface is callable failure-only shims |
 | `LibreMetaverse.RLV` | 28 | 499 | native implementation: 17 types / 200 members; remaining surface is callable failure-only shims |
 | `LibreMetaverse.Rendering.MeshFoundry` | 1 | 14 | native implementation: 1 type / 14 members; no generated shims remain |

crates/libremetaverse-lsl-tools/README.md

@@ -4,9 +4,10 @@
 parser-generator runtime in the pinned `LibreMetaverse.LslTools` assembly. The
 issue 81 boundary supplies source reading, comments, Unicode character sets,
 deterministic DFA execution, reserved words, terminal symbols, EOF, source
-locations, diagnostics, and Rust iterators. Grammar productions, reductions,
+locations, diagnostics, and Rust iterators. The issue 82 boundary supplies
-and recovery are added by issue 82; deterministic generated grammar and token
+grammar productions, parser tables, precedence, reductions, and recovery.
-tables are added by issue 83.
+Deterministic checked-in generated grammar and token tables are added by issue
+83.
 
 ## Source and position contract
 
@@ -53,6 +54,37 @@ after a diagnostic and never yield a fabricated token. Token and symbol
 tables and populate the supplied parser-entry priority. Missing or malformed
 table data is an explicit positioned error.
 
+## Grammar and parser contract
+
+`Grammar` builds validated canonical LR(0) item sets with deterministic LALR(1)
+lookahead propagation. Symbols and productions retain stable numeric
+identities. Table construction rejects missing start/EOF declarations,
+undeclared right-hand-side symbols, invalid precedence declarations, oversized
+state sets, incomplete goto tables, and reduction underflow. Empty productions
+and recursive nonterminals are supported.
+
+Shift/reduce conflicts use yacc-compatible rules: a declared higher precedence
+wins, equal left precedence reduces, equal right precedence shifts, and equal
+nonassociative precedence installs a rejecting table entry. Undeclared
+shift/reduce conflicts shift; reduce/reduce conflicts select the lower
+production number. `YyParser::conflicts` preserves every decision and `emit`
+writes a byte-stable table description independent of hash iteration.
+
+`Parser` consumes the native lexer and returns a typed `ParseTree` through the
+mapped `SYMBOL` semantic value. Syntax recovery uses the conventional terminal
+number zero: it pops to a state that can shift `error`, shifts an explicit error
+node, and discards input to the next valid lookahead. Diagnostics distinguish
+the original syntax error from successful recovery. Parsing is bounded to
+65,536 states, 1,048,576 live stack entries, 16,777,216 operations, and 1,000
+recovery attempts.
+
+Compatibility `SymbolSet` clones share deterministic FIRST/FOLLOW membership,
+and cloned `CSymbol` values share registered production metadata. This retains
+the observable reference behavior of the C# grammar model without unsafe code.
+Parser-source serialization and the old `Builder`-driven generated-output
+members return `InvalidOperation` explicitly until issue 83 supplies the
+checked-in generator; they never report a false success.
+
 ## Diagnostics and migration
 
 `ErrorHandler` collects structured `Diagnostic` values. Each diagnostic has a
@@ -73,6 +105,9 @@ The principal API mapping is:
 | `Lexer._Enumerator` | `LexerEnumerator`; prefer `LexerIterator` |
 | `Charset`, `CatTest` | Same mapped names plus `DotNetUnicodeCategory` |
 | `CSToolsException`, `ErrorHandler` | Same mapped names plus structured `Diagnostic` |
+| `CSymbol`, `SymbolSet`, `Production`, `Precedence` | Same mapped names plus typed `Grammar` builders |
+| `YyParser`, parser entries | Deterministic native tables and conflict records |
+| `Parser`, `ParseStackEntry`, `Error`, `recoveredError` | Bounded parsing, typed trees, and recovery |
 
 No C#, .NET runtime, dynamically loaded class, macOS-only API, platform code
 page, or runtime source generation is used by this boundary.
@@ -87,8 +122,9 @@ CARGO_BUILD_JOBS=1 cargo check --manifest-path tests/api-compile/Cargo.toml --lo
 CARGO_BUILD_JOBS=1 cargo clippy -p libremetaverse-lsl-tools --all-targets --locked -- -D warnings
 RUSTDOCFLAGS='-D warnings' CARGO_BUILD_JOBS=1 cargo doc -p libremetaverse-lsl-tools --no-deps --locked
 python3 tools/check_milestone_10_issue_81.py
+python3 tools/check_milestone_10_issue_82.py
 python3 tools/generate_api_shims.py --check
 ```
 
-The package contains 27 focused native and compatibility fixtures. The Gitea
+The package contains 37 focused native and compatibility fixtures. The Gitea
 workflow runs the audit and workspace compile on `ubuntu-latest` only.

crates/libremetaverse-lsl-tools/src/lexer.rs

@@ -543,7 +543,7 @@ impl LineManager {
 }
 
 /// Source location compatible with `SourceLineInfo`.
-#[derive(Clone, Debug)]
+#[derive(Clone, Debug, Default)]
 pub struct SourceLineInfo {
     pub char_position: i32,
     pub end_of_line: i32,
@@ -638,7 +638,10 @@ pub enum DiagnosticCategory {
     Encoding,
     InvalidCharacter,
     InvalidState,
+    ParserRecovery,
+    ParserStackLimit,
     Source,
+    Syntax,
     TokenTooLong,
     UnknownCharacterSet,
     UnexpectedEof,
@@ -986,7 +989,7 @@ impl ErrorHandler {
         std::mem::take(&mut self.diagnostics)
     }
 
-    fn push(&mut self, diagnostic: Diagnostic) -> Result<(), Error> {
+    pub(crate) fn push(&mut self, diagnostic: Diagnostic) -> Result<(), Error> {
         self.counter = self.counter.checked_add(1).ok_or(Error::IndexOutOfRange)?;
         if self.throw_exceptions {
             return Err(Error::InvalidOperation);
@@ -2007,6 +2010,24 @@ pub struct SYMBOL {
     pub pos_with_field: i32,
     pub yylx: Option<Box<Lexer>>,
     pub yyps: Option<Box<Parser>>,
+    name: String,
+    number: i32,
+    text: String,
+}
+
+impl Clone for SYMBOL {
+    fn clone(&self) -> Self {
+        Self {
+            kids: self.kids.clone(),
+            m_dollar: self.m_dollar.clone(),
+            pos_with_field: self.pos_with_field,
+            yylx: self.yylx.clone(),
+            yyps: self.yyps.clone(),
+            name: self.name.clone(),
+            number: self.number,
+            text: self.text.clone(),
+        }
+    }
 }
 
 impl fmt::Debug for SYMBOL {
@@ -2018,6 +2039,9 @@ impl fmt::Debug for SYMBOL {
             .field("pos", &self.pos_with_field)
             .field("has_lexer", &self.yylx.is_some())
             .field("has_parser", &self.yyps.is_some())
+            .field("name", &self.name)
+            .field("number", &self.number)
+            .field("text", &self.text)
             .finish()
     }
 }
@@ -2030,6 +2054,9 @@ impl SYMBOL {
             pos_with_field: lexer.yypos(),
             yylx: Some(Box::new(lexer)),
             yyps: None,
+            name: "SYMBOL".to_owned(),
+            number: 0,
+            text: String::new(),
         })
     }
 
@@ -2040,9 +2067,32 @@ impl SYMBOL {
             pos_with_field: 0,
             yylx: None,
             yyps: Some(Box::new(parser)),
+            name: "SYMBOL".to_owned(),
+            number: 0,
+            text: String::new(),
         })
     }
 
+    pub(crate) fn parser_symbol(
+        name: String,
+        number: i32,
+        text: String,
+        pos: i32,
+        value: Object,
+        kids: ObjectList,
+    ) -> Self {
+        Self {
+            kids,
+            m_dollar: value,
+            pos_with_field: pos,
+            yylx: None,
+            yyps: None,
+            name,
+            number,
+            text,
+        }
+    }
+
     fn location(&self) -> Result<SourceLineInfo, Error> {
         self.yylx
             .as_deref()
@@ -2074,8 +2124,8 @@ impl SYMBOL {
     pub fn is_terminal(&self) -> Result<bool, Error> {
         Ok(false)
     }
-    pub fn matches(&self, _value: String) -> Result<bool, Error> {
+    pub fn matches(&self, value: String) -> Result<bool, Error> {
-        Ok(false)
+        Ok(self.text == value)
     }
 
     pub fn pass_(
@@ -2093,7 +2143,11 @@ impl SYMBOL {
     }
     #[must_use]
     pub fn to_string(&self) -> String {
+        if self.text.is_empty() {
             self.yyname()
+        } else {
+            format!("{}<{}>", self.yyname(), self.text)
+        }
     }
 
     pub fn from(symbol: SYMBOL) -> i32 {
@@ -2133,11 +2187,11 @@ impl SYMBOL {
     }
     #[must_use]
     pub fn yyname(&self) -> String {
-        "SYMBOL".to_owned()
+        self.name.clone()
     }
     #[must_use]
     pub const fn yynum(&self) -> i32 {
-        0
+        self.number
     }
 }
 
@@ -2297,6 +2351,10 @@ fn lookup_parser_entry(
     state: i32,
     entry: &mut ParserEntry,
 ) -> Result<bool, Error> {
+    if let Some(native) = symbols.lookup_entry(number, state)? {
+        *entry = native;
+        return Ok(true);
+    }
     let Some(info) = symbols.symbol_info.0.get(&Object::Integer(number)) else {
         return Err(Error::Parse {
             position: 0,
@@ -2392,7 +2450,7 @@ impl Null {
 }
 
 /// Rust replacement for the C# cons-list, backed by contiguous storage.
-#[derive(Debug, Default)]
+#[derive(Clone, Debug, Default)]
 pub struct ObjectList {
     values: VecDeque<Object>,
     count_override: Option<i32>,

crates/libremetaverse-lsl-tools/src/lib.rs

@@ -4,6 +4,7 @@ extern crate self as libremetaverse_lsl_tools;
 
 mod generated;
 mod lexer;
+mod parser;
 
 pub use generated::*;
 pub use lexer::{
@@ -12,3 +13,7 @@ pub use lexer::{
     MAX_TOKEN_UNITS, TokenDefinition, UnicodeClass,
 };
 pub use libremetaverse_types::Error;
+pub use parser::{
+    Associativity, ERROR_TOKEN, Grammar, GrammarProduction, GrammarSymbol, MAX_PARSER_STACK,
+    MAX_PARSER_STATES, MAX_PARSER_STEPS, MAX_RECOVERY_ERRORS, ParseTree, ParserConflict,
+};

crates/libremetaverse-lsl-tools/tests/parser_compat.rs

@@ -0,0 +1,474 @@
+//! Focused compatibility fixtures for the native issue #82 parser boundary.
+
+use std::sync::{Arc, Mutex};
+
+use libremetaverse_lsl_tools::{
+    Associativity, CSymbol, CSymbolSymType, CharacterMatcher, Dfa, DfaAccept, DfaState,
+    DiagnosticCategory, Error, ErrorHandler, Grammar, Lexer, LexerAction, LslError, ParseTree,
+    Parser, ParserEntry, Precedence, PrecedencePrecType, Production, ResWds, SymbolSet,
+    TokenDefinition, YyLexer,
+};
+
+const EOF: i32 = 2;
+const ID: i32 = 3;
+const INTEGER: i32 = 4;
+const PLUS: i32 = 5;
+const STAR: i32 = 6;
+const MINUS: i32 = 7;
+const LPAREN: i32 = 8;
+const RPAREN: i32 = 9;
+const SEMICOLON: i32 = 10;
+const COMMA: i32 = 11;
+const STRING: i32 = 12;
+const LBRACE: i32 = 13;
+const RBRACE: i32 = 14;
+const EQUAL: i32 = 15;
+const DEFAULT: i32 = 20;
+const STATE_ENTRY: i32 = 21;
+
+fn definition(name: &str, number: i32) -> TokenDefinition {
+    TokenDefinition::new(name, number).expect("valid token")
+}
+
+fn accept(name: &str, number: i32, action: LexerAction) -> DfaAccept {
+    DfaAccept {
+        token: definition(name, number),
+        action,
+        action_number: number,
+        reserved_words: (number == ID).then(|| "keywords".to_owned()),
+    }
+}
+
+fn parser_lexer(source: &str) -> Lexer {
+    use libremetaverse_lsl_tools::DotNetUnicodeCategory as Category;
+
+    let letters = vec![
+        Category::UppercaseLetter,
+        Category::LowercaseLetter,
+        Category::TitlecaseLetter,
+        Category::ModifierLetter,
+        Category::OtherLetter,
+    ];
+    let whitespace = vec![
+        Category::Control,
+        Category::SpaceSeparator,
+        Category::LineSeparator,
+        Category::ParagraphSeparator,
+    ];
+    let states = vec![
+        DfaState::default()
+            .transition(CharacterMatcher::Categories(letters.clone()), 1)
+            .transition(CharacterMatcher::Exact(b'_'.into()), 1)
+            .transition(CharacterMatcher::Category(Category::DecimalDigitNumber), 2)
+            .transition(CharacterMatcher::Categories(whitespace.clone()), 3)
+            .transition(CharacterMatcher::Exact(b'+'.into()), 4)
+            .transition(CharacterMatcher::Exact(b'*'.into()), 5)
+            .transition(CharacterMatcher::Exact(b'-'.into()), 6)
+            .transition(CharacterMatcher::Exact(b'('.into()), 7)
+            .transition(CharacterMatcher::Exact(b')'.into()), 8)
+            .transition(CharacterMatcher::Exact(b';'.into()), 9)
+            .transition(CharacterMatcher::Exact(b','.into()), 10)
+            .transition(CharacterMatcher::Exact(b'{'.into()), 11)
+            .transition(CharacterMatcher::Exact(b'}'.into()), 12)
+            .transition(CharacterMatcher::Exact(b'"'.into()), 13)
+            .transition(CharacterMatcher::Exact(b'='.into()), 15),
+        DfaState::default()
+            .transition(CharacterMatcher::Categories(letters.clone()), 1)
+            .transition(CharacterMatcher::Exact(b'_'.into()), 1)
+            .transition(CharacterMatcher::Category(Category::DecimalDigitNumber), 1)
+            .accepting(accept("ID", ID, LexerAction::Emit)),
+        DfaState::default()
+            .transition(CharacterMatcher::Category(Category::DecimalDigitNumber), 2)
+            .accepting(accept("INTEGER", INTEGER, LexerAction::Emit)),
+        DfaState::default()
+            .transition(CharacterMatcher::Categories(whitespace), 3)
+            .accepting(accept("WS", 30, LexerAction::Skip)),
+        DfaState::default().accepting(accept("PLUS", PLUS, LexerAction::Emit)),
+        DfaState::default().accepting(accept("STAR", STAR, LexerAction::Emit)),
+        DfaState::default().accepting(accept("MINUS", MINUS, LexerAction::Emit)),
+        DfaState::default().accepting(accept("LPAREN", LPAREN, LexerAction::Emit)),
+        DfaState::default().accepting(accept("RPAREN", RPAREN, LexerAction::Emit)),
+        DfaState::default().accepting(accept("SEMICOLON", SEMICOLON, LexerAction::Emit)),
+        DfaState::default().accepting(accept("COMMA", COMMA, LexerAction::Emit)),
+        DfaState::default().accepting(accept("LBRACE", LBRACE, LexerAction::Emit)),
+        DfaState::default().accepting(accept("RBRACE", RBRACE, LexerAction::Emit)),
+        DfaState::default()
+            .transition(CharacterMatcher::Categories(letters), 13)
+            .transition(CharacterMatcher::Exact(b' '.into()), 13)
+            .transition(CharacterMatcher::Exact(b'"'.into()), 14),
+        DfaState::default().accepting(accept("STRING", STRING, LexerAction::Emit)),
+        DfaState::default().accepting(accept("EQUAL", EQUAL, LexerAction::Emit)),
+    ];
+    let mut table = YyLexer::new(ErrorHandler::default()).expect("lexer table");
+    table.using_eof = true;
+    table
+        .set_start_dfa("YYINITIAL", Dfa::from_states(states, 0).expect("dfa"))
+        .expect("start state");
+    table
+        .set_reserved_words(
+            "keywords",
+            ResWds::from_pairs(
+                [
+                    ("default", definition("DEFAULT", DEFAULT)),
+                    ("state_entry", definition("STATE_ENTRY", STATE_ENTRY)),
+                ],
+                false,
+            )
+            .expect("reserved words"),
+        )
+        .expect("reserved table");
+    let mut lexer = Lexer::new(table).expect("lexer");
+    lexer.start_with_string(source.to_owned()).expect("source");
+    lexer
+}
+
+fn add_common_terminals(grammar: &mut Grammar) {
+    for (name, number) in [
+        ("EOF", EOF),
+        ("ID", ID),
+        ("INTEGER", INTEGER),
+        ("PLUS", PLUS),
+        ("STAR", STAR),
+        ("MINUS", MINUS),
+        ("LPAREN", LPAREN),
+        ("RPAREN", RPAREN),
+        ("SEMICOLON", SEMICOLON),
+        ("COMMA", COMMA),
+        ("STRING", STRING),
+        ("LBRACE", LBRACE),
+        ("RBRACE", RBRACE),
+        ("EQUAL", EQUAL),
+        ("DEFAULT", DEFAULT),
+        ("STATE_ENTRY", STATE_ENTRY),
+        ("Error", 0),
+    ] {
+        grammar.add_symbol(name, number, true).expect("terminal");
+    }
+}
+
+fn expression_parser(source: &str) -> Parser {
+    const EXPRESSION: i32 = 100;
+    let mut grammar = Grammar::new(EXPRESSION, EOF).expect("grammar");
+    add_common_terminals(&mut grammar);
+    grammar
+        .add_symbol("Expression", EXPRESSION, false)
+        .expect("nonterminal");
+    grammar
+        .set_precedence(PLUS, 1, Associativity::Left)
+        .expect("plus precedence");
+    grammar
+        .set_precedence(STAR, 2, Associativity::Left)
+        .expect("star precedence");
+    grammar
+        .set_precedence(MINUS, 3, Associativity::Right)
+        .expect("minus precedence");
+    grammar
+        .add_production(EXPRESSION, vec![EXPRESSION, PLUS, EXPRESSION])
+        .expect("plus");
+    grammar
+        .add_production(EXPRESSION, vec![EXPRESSION, STAR, EXPRESSION])
+        .expect("multiply");
+    grammar
+        .add_production_with_precedence(EXPRESSION, vec![MINUS, EXPRESSION], Some(MINUS))
+        .expect("unary");
+    grammar
+        .add_production(EXPRESSION, vec![LPAREN, EXPRESSION, RPAREN])
+        .expect("parentheses");
+    grammar
+        .add_production(EXPRESSION, vec![ID])
+        .expect("identifier");
+    Parser::new(grammar.build().expect("table"), parser_lexer(source)).expect("parser")
+}
+
+fn tree(symbol: &libremetaverse_lsl_tools::SYMBOL) -> &ParseTree {
+    symbol
+        .m_dollar
+        .downcast_ref::<ParseTree>()
+        .expect("parse tree")
+}
+
+#[test]
+fn precedence_and_associativity_choose_the_reference_tree() {
+    let mut parser = expression_parser("a + b * c + d");
+    let result = parser
+        .parse_with_string("a + b * c + d".to_owned())
+        .expect("parse");
+    let root = tree(&result);
+    assert_eq!(root.name, "Expression");
+    assert_eq!(root.children[1].number, PLUS);
+    assert_eq!(root.children[0].children[1].number, PLUS);
+    assert_eq!(root.children[0].children[2].children[1].number, STAR);
+    assert!(!parser.m_symbols.conflicts().is_empty());
+}
+
+#[test]
+fn right_associative_unary_precedence_is_deterministic() {
+    let mut parser = expression_parser("- - a");
+    let result = parser.parse_with_string("- - a".to_owned()).expect("parse");
+    let root = tree(&result);
+    assert_eq!(root.children[0].number, MINUS);
+    assert_eq!(root.children[1].children[0].number, MINUS);
+}
+
+#[test]
+fn nonassociative_conflict_rejects_a_chained_operator() {
+    const EXPRESSION: i32 = 100;
+    let mut grammar = Grammar::new(EXPRESSION, EOF).expect("grammar");
+    add_common_terminals(&mut grammar);
+    grammar
+        .add_symbol("Expression", EXPRESSION, false)
+        .expect("nonterminal");
+    grammar
+        .set_precedence(PLUS, 1, Associativity::Nonassoc)
+        .expect("precedence");
+    grammar
+        .add_production(EXPRESSION, vec![EXPRESSION, PLUS, EXPRESSION])
+        .expect("binary");
+    grammar
+        .add_production(EXPRESSION, vec![ID])
+        .expect("identifier");
+    let mut parser =
+        Parser::new(grammar.build().expect("table"), parser_lexer("a+a+a")).expect("parser");
+    assert!(matches!(
+        parser.parse_with_string("a+a+a".to_owned()),
+        Err(Error::Parse {
+            context: "syntax error",
+            ..
+        })
+    ));
+    assert_eq!(
+        parser.diagnostics().expect("diagnostics")[0].category,
+        DiagnosticCategory::Syntax
+    );
+}
+
+#[test]
+fn empty_production_accepts_an_empty_input() {
+    const START: i32 = 100;
+    const ITEMS: i32 = 101;
+    let mut grammar = Grammar::new(START, EOF).expect("grammar");
+    add_common_terminals(&mut grammar);
+    grammar.add_symbol("Start", START, false).expect("start");
+    grammar.add_symbol("Items", ITEMS, false).expect("items");
+    grammar.add_production(START, vec![ITEMS]).expect("root");
+    grammar.add_production(ITEMS, vec![]).expect("empty");
+    let mut parser =
+        Parser::new(grammar.build().expect("table"), parser_lexer("")).expect("parser");
+    let result = parser.parse_with_string(String::new()).expect("parse");
+    assert_eq!(tree(&result).children[0].name, "Items");
+}
+
+#[test]
+fn lalr_lookaheads_avoid_the_classic_slr_assignment_conflict() {
+    const START: i32 = 100;
+    const LEFT: i32 = 101;
+    const RIGHT: i32 = 102;
+    let mut grammar = Grammar::new(START, EOF).expect("grammar");
+    add_common_terminals(&mut grammar);
+    for (name, number) in [("Start", START), ("Left", LEFT), ("Right", RIGHT)] {
+        grammar
+            .add_symbol(name, number, false)
+            .expect("nonterminal");
+    }
+    grammar
+        .add_production(START, vec![LEFT, EQUAL, RIGHT])
+        .expect("assignment");
+    grammar.add_production(START, vec![RIGHT]).expect("value");
+    grammar
+        .add_production(LEFT, vec![STAR, RIGHT])
+        .expect("dereference");
+    grammar.add_production(LEFT, vec![ID]).expect("name");
+    grammar.add_production(RIGHT, vec![LEFT]).expect("right");
+    let table = grammar.build().expect("LALR table");
+    assert!(table.conflicts().is_empty());
+    for source in ["a = b", "* a = b"] {
+        let mut parser = Parser::new(table.clone(), parser_lexer(source)).expect("parser");
+        let result = parser
+            .parse_with_string(source.to_owned())
+            .expect("assignment parse");
+        assert_eq!(tree(&result).name, "Start");
+    }
+}
+
+#[test]
+fn error_token_recovery_discards_input_and_returns_recovered_tree() {
+    const START: i32 = 100;
+    const LIST: i32 = 101;
+    const STATEMENT: i32 = 102;
+    let mut grammar = Grammar::new(START, EOF).expect("grammar");
+    add_common_terminals(&mut grammar);
+    for (name, number) in [("Start", START), ("List", LIST), ("Statement", STATEMENT)] {
+        grammar
+            .add_symbol(name, number, false)
+            .expect("nonterminal");
+    }
+    grammar.add_production(START, vec![LIST]).expect("start");
+    grammar
+        .add_production(LIST, vec![LIST, STATEMENT])
+        .expect("list");
+    grammar
+        .add_production(LIST, vec![STATEMENT])
+        .expect("single");
+    grammar
+        .add_production(STATEMENT, vec![ID, SEMICOLON])
+        .expect("statement");
+    grammar
+        .add_production(STATEMENT, vec![0, ID, SEMICOLON])
+        .expect("recovery");
+    let mut parser =
+        Parser::new(grammar.build().expect("table"), parser_lexer("a; + b;")).expect("parser");
+    let result = parser
+        .parse_with_string("a; + b;".to_owned())
+        .expect("recovered parse");
+    assert_eq!(result.yyname(), "recoveredError");
+    let diagnostics = parser.diagnostics().expect("diagnostics");
+    assert_eq!(diagnostics[0].category, DiagnosticCategory::Syntax);
+    assert_eq!(diagnostics[1].category, DiagnosticCategory::ParserRecovery);
+}
+
+#[test]
+fn representative_lsl_script_parses_to_expected_model() {
+    const SCRIPT: i32 = 100;
+    const EVENT: i32 = 101;
+    const CALL: i32 = 102;
+    let source = r#"default { state_entry() { llSay(0, "hello"); } }"#;
+    let mut grammar = Grammar::new(SCRIPT, EOF).expect("grammar");
+    add_common_terminals(&mut grammar);
+    for (name, number) in [("Script", SCRIPT), ("Event", EVENT), ("Call", CALL)] {
+        grammar
+            .add_symbol(name, number, false)
+            .expect("nonterminal");
+    }
+    grammar
+        .add_production(SCRIPT, vec![DEFAULT, LBRACE, EVENT, RBRACE])
+        .expect("script");
+    grammar
+        .add_production(
+            EVENT,
+            vec![STATE_ENTRY, LPAREN, RPAREN, LBRACE, CALL, SEMICOLON, RBRACE],
+        )
+        .expect("event");
+    grammar
+        .add_production(CALL, vec![ID, LPAREN, INTEGER, COMMA, STRING, RPAREN])
+        .expect("call");
+    let mut parser =
+        Parser::new(grammar.build().expect("table"), parser_lexer(source)).expect("parser");
+    let result = parser
+        .parse_with_string(source.to_owned())
+        .expect("LSL source");
+    let root = tree(&result);
+    assert_eq!(root.name, "Script");
+    assert_eq!(root.children[2].name, "Event");
+    assert_eq!(root.children[2].children[4].name, "Call");
+    assert_eq!(
+        root.children[2].children[4].children[0].text.as_deref(),
+        Some("llSay")
+    );
+}
+
+#[test]
+fn parser_table_emission_is_byte_deterministic() {
+    let parser = expression_parser("a");
+    let first = Arc::new(Mutex::new(Vec::new()));
+    parser
+        .m_symbols
+        .emit(Box::new(SharedWriter(first.clone())))
+        .expect("first");
+    let second = Arc::new(Mutex::new(Vec::new()));
+    parser
+        .m_symbols
+        .emit(Box::new(SharedWriter(second.clone())))
+        .expect("second");
+    assert_eq!(
+        first.lock().expect("first output").as_slice(),
+        second.lock().expect("second output").as_slice()
+    );
+}
+
+struct SharedWriter(Arc<Mutex<Vec<u8>>>);
+
+impl std::io::Write for SharedWriter {
+    fn write(&mut self, bytes: &[u8]) -> std::io::Result<usize> {
+        self.0
+            .lock()
+            .map_err(|_| std::io::Error::other("poisoned output"))?
+            .extend_from_slice(bytes);
+        Ok(bytes.len())
+    }
+
+    fn flush(&mut self) -> std::io::Result<()> {
+        Ok(())
+    }
+}
+
+#[test]
+fn malformed_corpus_terminates_without_panics_or_unbounded_growth() {
+    for source in ["+", "a +", "((a", "a * * b", "-", "a )", "@"] {
+        let mut parser = expression_parser(source);
+        let _ = parser.parse_with_string(source.to_owned());
+        assert!(parser.diagnostics().expect("diagnostics").len() <= 2);
+    }
+}
+
+#[test]
+fn mapped_symbol_set_production_precedence_and_entry_apis_are_live() {
+    let terminal = CSymbol::native("PLUS", PLUS, CSymbolSymType::Terminal).expect("terminal");
+    let nonterminal =
+        CSymbol::native("Expression", 100, CSymbolSymType::Nonterminal).expect("nonterminal");
+    let mut set = SymbolSet::native();
+    assert!(set.check_in(terminal.clone()).expect("insert"));
+    assert!(!set.check_in(terminal.clone()).expect("deduplicate"));
+    assert!(set.contains(terminal.clone()).expect("contains"));
+    assert!(terminal.matches("PLUS".to_owned()).expect("symbol text"));
+
+    let mut followed = nonterminal.clone();
+    assert!(
+        followed
+            .add_follow(terminal.m_first.clone())
+            .expect("first follow insertion")
+    );
+    assert_eq!(nonterminal.m_follow.count(), 1);
+
+    let nullable =
+        CSymbol::native("Optional", 101, CSymbolSymType::Nonterminal).expect("nullable symbol");
+    let _empty = Production::native(8, nullable.clone(), vec![]).expect("empty production");
+    assert!(nullable.is_nullable().expect("registered empty production"));
+
+    let mut production = Production::native(7, nonterminal.clone(), vec![]).expect("production");
+    production.add_to_rhs(nonterminal.clone()).expect("lhs");
+    production.add_to_rhs(terminal.clone()).expect("operator");
+    assert_eq!(production.prefix(1).expect("prefix")[0].m_yynum, 100);
+    let target = CSymbol::native("Target", 102, CSymbolSymType::Nonterminal).expect("target");
+    production
+        .add_first(target.clone(), 1)
+        .expect("FIRST/FOLLOW propagation");
+    assert!(
+        target
+            .m_follow
+            .contains(terminal.clone())
+            .expect("shared follow mutation")
+    );
+
+    let precedence = Precedence::first(PrecedencePrecType::Left, 5).expect("precedence");
+    assert_eq!(
+        Precedence::check_with_precedence_prec_type_int32(precedence, PrecedencePrecType::Left, 0,),
+        Ok(5)
+    );
+
+    let parser = expression_parser("a");
+    let mut lexer = parser.m_lexer.clone();
+    let symbol = lexer.next_token().expect("lexing").expect("token");
+    let mut entry = ParserEntry::default();
+    assert_eq!(
+        symbol.pass_(parser.m_symbols.clone(), 0, &mut entry),
+        Ok(true)
+    );
+    assert_eq!(entry.str(), "shift 1");
+
+    let lsl_error = LslError {
+        state: 4,
+        sym: libremetaverse_lsl_tools::SYMBOL::new_with_lexer(parser.m_lexer.clone())
+            .expect("symbol"),
+    };
+    assert!(lsl_error.to_string().contains("state 4"));
+}

tests/compat/tests/extension_shims.rs

@@ -26,7 +26,7 @@ async fn compile_rlv_calls(
         .await;
 }
 
-fn compile_parser_call(parser: &Parser) {
+fn compile_parser_call(parser: &mut Parser) {
     let _ = parser.parse_with_string(String::new());
 }
 
@@ -65,11 +65,8 @@ fn extension_flows_have_typed_callable_signatures() {
 }
 
 #[test]
-fn extension_constructors_fail_with_catalog_ids() {
+fn extension_constructors_have_expected_native_or_failure_behavior() {
-    assert_eq!(
+    assert!(YyParser::new().is_ok());
-        member_id(YyParser::new()),
-        "M:LibreMetaverse.LslTools.YyParser.#ctor"
-    );
     assert_eq!(
         member_id(RlvActionCallbacksDefault::new()),
         "M:LibreMetaverse.RLV.RlvActionCallbacksDefault.#ctor"

tools/check_milestone_10_issue_82.py

@@ -0,0 +1,138 @@
+#!/usr/bin/env python3
+"""Audit issue 82's native grammar, parser-table, and recovery boundary."""
+
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+
+import generate_api_shims
+
+
+ROOT = Path(__file__).resolve().parents[1]
+SOURCE = ROOT / "crates" / "libremetaverse-lsl-tools" / "src" / "parser.rs"
+GENERATED = ROOT / "crates" / "libremetaverse-lsl-tools" / "src" / "generated.rs"
+TESTS = ROOT / "crates" / "libremetaverse-lsl-tools" / "tests" / "parser_compat.rs"
+EXTENSION_TESTS = ROOT / "tests" / "compat" / "tests" / "extension_shims.rs"
+DOC = ROOT / "crates" / "libremetaverse-lsl-tools" / "README.md"
+WORKFLOW = ROOT / ".gitea" / "workflows" / "rust-workspace.yml"
+CATALOG = ROOT / "api" / "public-api.json"
+STUB_RE = re.compile(
+    r"\b(?:not_implemented|unimplemented_api)\b|\b(?:todo|unimplemented)!\s*\("
+)
+
+TYPES = {
+    "T:LibreMetaverse.LslTools.CSymbol": "crate::parser::CSymbol",
+    "T:LibreMetaverse.LslTools.CSymbol.SymType": "crate::parser::CSymbolSymType",
+    "T:LibreMetaverse.LslTools.Error": "crate::parser::LslError",
+    "T:LibreMetaverse.LslTools.Literal": "crate::parser::Literal",
+    "T:LibreMetaverse.LslTools.ParseStackEntry": "crate::parser::ParseStackEntry",
+    "T:LibreMetaverse.LslTools.ParseState": "crate::parser::ParseState",
+    "T:LibreMetaverse.LslTools.Parser": "crate::parser::Parser",
+    "T:LibreMetaverse.LslTools.ParserAction": "crate::parser::ParserAction",
+    "T:LibreMetaverse.LslTools.ParserEntry": "crate::parser::ParserEntry",
+    "T:LibreMetaverse.LslTools.ParserOldAction": "crate::parser::ParserOldAction",
+    "T:LibreMetaverse.LslTools.ParserReduce": "crate::parser::ParserReduce",
+    "T:LibreMetaverse.LslTools.ParserShift": "crate::parser::ParserShift",
+    "T:LibreMetaverse.LslTools.ParserSimpleAction": "crate::parser::ParserSimpleAction",
+    "T:LibreMetaverse.LslTools.ParsingInfo": "crate::parser::ParsingInfo",
+    "T:LibreMetaverse.LslTools.Precedence": "crate::parser::Precedence",
+    "T:LibreMetaverse.LslTools.Precedence.PrecType": "crate::parser::PrecedencePrecType",
+    "T:LibreMetaverse.LslTools.ProdItem": "crate::parser::ProdItem",
+    "T:LibreMetaverse.LslTools.Production": "crate::parser::Production",
+    "T:LibreMetaverse.LslTools.SymbolSet": "crate::parser::SymbolSet",
+    "T:LibreMetaverse.LslTools.Transition": "crate::parser::Transition",
+    "T:LibreMetaverse.LslTools.YyParser": "crate::parser::YyParser",
+    "T:LibreMetaverse.LslTools.recoveredError": "crate::parser::RecoveredError",
+}
+
+
+def require_markers(path: Path, markers: tuple[str, ...]) -> None:
+    text = path.read_text()
+    missing = [marker for marker in markers if marker not in text]
+    if missing:
+        raise SystemExit(f"{path.name}: audit evidence missing: " + ", ".join(missing))
+
+
+def catalog_member_count() -> int:
+    catalog = json.loads(CATALOG.read_text())
+    assembly = next(
+        value for value in catalog["assemblies"]
+        if value["identity"]["name"] == "LibreMetaverse.LslTools"
+    )
+    return sum(
+        len(api_type["members"])
+        for api_type in assembly["types"]
+        if api_type["doc_id"] in TYPES
+    )
+
+
+def generated_type_block(text: str, doc_id: str) -> str:
+    marker = f"/// C# type: `{doc_id}`."
+    start = text.find(marker)
+    if start < 0:
+        raise SystemExit(f"generated declaration is missing for {doc_id}")
+    next_type = text.find("/// C# type:", start + len(marker))
+    return text[start:] if next_type < 0 else text[start:next_type]
+
+
+def main() -> None:
+    for api_type, declaration in TYPES.items():
+        if generate_api_shims.NATIVE_TYPES.get(api_type) != declaration:
+            raise SystemExit(f"issue 82 native type mapping is missing for {api_type}")
+    if catalog_member_count() != 215:
+        raise SystemExit("issue 82 expected 215 mapped parser members")
+
+    source = SOURCE.read_text()
+    if STUB_RE.search(source):
+        raise SystemExit("issue 82 owned Rust stubs remain in parser.rs")
+    if re.search(r"unsafe\s*\{|unsafe\s+impl|target_os\s*=\s*\"macos\"", source):
+        raise SystemExit("parser.rs contains an unsafe or macOS-only implementation")
+    generated = GENERATED.read_text()
+    for api_type in TYPES:
+        if STUB_RE.search(generated_type_block(generated, api_type)):
+            raise SystemExit(f"issue 82 owned generated stubs remain for {api_type}")
+
+    require_markers(SOURCE, (
+        "pub struct Grammar", "pub struct GrammarProduction", "struct ParserMachine",
+        "fn first_sets", "fn lalr_lookaheads", "fn resolve_shift_reduce", "TableAction::Reject",
+        "pub struct ParserConflict", "MAX_PARSER_STATES", "MAX_PARSER_STACK",
+        "MAX_PARSER_STEPS", "MAX_RECOVERY_ERRORS", "pub struct ParseTree",
+        "pub struct CSymbol", "pub struct SymbolSet", "pub struct Production",
+        "pub struct Precedence", "pub struct ParseState", "pub struct Transition",
+        "pub struct YyParser", "pub struct Parser", "fn parse_started", "fn recover",
+        "DiagnosticCategory::ParserRecovery", "Error::InvalidOperation",
+    ))
+    require_markers(TESTS, (
+        "precedence_and_associativity_choose_the_reference_tree",
+        "right_associative_unary_precedence_is_deterministic",
+        "nonassociative_conflict_rejects_a_chained_operator",
+        "empty_production_accepts_an_empty_input",
+        "lalr_lookaheads_avoid_the_classic_slr_assignment_conflict",
+        "error_token_recovery_discards_input_and_returns_recovered_tree",
+        "representative_lsl_script_parses_to_expected_model",
+        "parser_table_emission_is_byte_deterministic",
+        "malformed_corpus_terminates_without_panics_or_unbounded_growth",
+        "mapped_symbol_set_production_precedence_and_entry_apis_are_live",
+        "shared follow mutation", "registered empty production",
+    ))
+    if TESTS.read_text().count("#[test]") != 10:
+        raise SystemExit("issue 82 expected 10 focused parser fixtures")
+    require_markers(EXTENSION_TESTS, ("assert!(YyParser::new().is_ok())",))
+    require_markers(DOC, (
+        "canonical LR(0)", "deterministic LALR(1)", "Shift/reduce conflicts",
+        "nonassociative", "1,048,576", "16,777,216", "37 focused",
+        "issue 83", "ubuntu-latest",
+    ))
+    require_markers(WORKFLOW, ("python3 tools/check_milestone_10_issue_82.py",))
+    print(
+        "issue 82 audit: 22 native mapped parser types and 215 members, deterministic "
+        "LALR tables, precedence conflicts, bounded shifts/reductions, error-token "
+        "recovery, shared compatibility grammar state, 10 focused fixtures, docs, and "
+        "ubuntu-only CI are present"
+    )
+
+
+if __name__ == "__main__":
+    main()

tools/generate_api_shims.py

@@ -43,23 +43,45 @@ NATIVE_TYPES = {
     "T:LibreMetaverse.LslTools.CSToolsStopException": "crate::lexer::CSToolsStopException",
     "T:LibreMetaverse.LslTools.CatTest": "crate::lexer::CatTest",
     "T:LibreMetaverse.LslTools.Charset": "crate::lexer::Charset",
+    "T:LibreMetaverse.LslTools.CSymbol": "crate::parser::CSymbol",
+    "T:LibreMetaverse.LslTools.CSymbol.SymType": "crate::parser::CSymbolSymType",
     "T:LibreMetaverse.LslTools.CommentList": "crate::lexer::CommentList",
     "T:LibreMetaverse.LslTools.CsReader": "crate::lexer::CsReader",
     "T:LibreMetaverse.LslTools.Dfa": "crate::lexer::Dfa",
     "T:LibreMetaverse.LslTools.Dfa.Action": "crate::lexer::DfaAction",
     "T:LibreMetaverse.LslTools.EOF": "crate::lexer::EOF",
     "T:LibreMetaverse.LslTools.ErrorHandler": "crate::lexer::ErrorHandler",
+    "T:LibreMetaverse.LslTools.Error": "crate::parser::LslError",
     "T:LibreMetaverse.LslTools.Lexer": "crate::lexer::Lexer",
     "T:LibreMetaverse.LslTools.Lexer._Enumerator": "crate::lexer::LexerEnumerator",
     "T:LibreMetaverse.LslTools.LineList": "crate::lexer::LineList",
     "T:LibreMetaverse.LslTools.LineManager": "crate::lexer::LineManager",
+    "T:LibreMetaverse.LslTools.Literal": "crate::parser::Literal",
     "T:LibreMetaverse.LslTools.Null": "crate::lexer::Null",
     "T:LibreMetaverse.LslTools.ObjectList": "crate::lexer::ObjectList",
+    "T:LibreMetaverse.LslTools.ParseStackEntry": "crate::parser::ParseStackEntry",
+    "T:LibreMetaverse.LslTools.ParseState": "crate::parser::ParseState",
+    "T:LibreMetaverse.LslTools.Parser": "crate::parser::Parser",
+    "T:LibreMetaverse.LslTools.ParserAction": "crate::parser::ParserAction",
+    "T:LibreMetaverse.LslTools.ParserEntry": "crate::parser::ParserEntry",
+    "T:LibreMetaverse.LslTools.ParserOldAction": "crate::parser::ParserOldAction",
+    "T:LibreMetaverse.LslTools.ParserReduce": "crate::parser::ParserReduce",
+    "T:LibreMetaverse.LslTools.ParserShift": "crate::parser::ParserShift",
+    "T:LibreMetaverse.LslTools.ParserSimpleAction": "crate::parser::ParserSimpleAction",
+    "T:LibreMetaverse.LslTools.ParsingInfo": "crate::parser::ParsingInfo",
+    "T:LibreMetaverse.LslTools.Precedence": "crate::parser::Precedence",
+    "T:LibreMetaverse.LslTools.Precedence.PrecType": "crate::parser::PrecedencePrecType",
+    "T:LibreMetaverse.LslTools.ProdItem": "crate::parser::ProdItem",
+    "T:LibreMetaverse.LslTools.Production": "crate::parser::Production",
     "T:LibreMetaverse.LslTools.ResWds": "crate::lexer::ResWds",
     "T:LibreMetaverse.LslTools.SYMBOL": "crate::lexer::SYMBOL",
     "T:LibreMetaverse.LslTools.SourceLineInfo": "crate::lexer::SourceLineInfo",
+    "T:LibreMetaverse.LslTools.SymbolSet": "crate::parser::SymbolSet",
     "T:LibreMetaverse.LslTools.TOKEN": "crate::lexer::TOKEN",
+    "T:LibreMetaverse.LslTools.Transition": "crate::parser::Transition",
     "T:LibreMetaverse.LslTools.YyLexer": "crate::lexer::YyLexer",
+    "T:LibreMetaverse.LslTools.YyParser": "crate::parser::YyParser",
+    "T:LibreMetaverse.LslTools.recoveredError": "crate::parser::RecoveredError",
     "T:LibreMetaverse.RLV.AttachmentRequest": "crate::service::AttachmentRequest",
     "T:LibreMetaverse.RLV.RlvActionCallbacksDefault": "crate::service::RlvActionCallbacksDefault",
     "T:LibreMetaverse.RLV.RlvCallbacksDefault": "crate::service::RlvCallbacksDefault",