99 lines
4.7 KiB
Markdown
99 lines
4.7 KiB
Markdown
# Native data generation
|
|
|
|
`libremetaverse-codegen` is the Rust-only framework for the six source
|
|
generators present at LibreMetaverse commit
|
|
`2aa70bb68513b39795da5d13c88f31b86e85a3ba`. Normal builds compile only the
|
|
checked-in Rust outputs. They do not run this tool, invoke .NET or Roslyn, read
|
|
an environment-dependent path, access the network, or open the sibling
|
|
reference checkout.
|
|
|
|
## Provenance inventory
|
|
|
|
[`sources.json`](sources.json) is the machine-readable inventory. It records
|
|
the upstream repository and commit, source-generator hashes, every actual
|
|
`AdditionalFile`, its vendored path, SHA-256, format, and BSD-3-Clause license.
|
|
|
|
| Golden generator | Pinned data inputs |
|
|
| --- | --- |
|
|
| PacketSourceGenerator | `message_template.msg` |
|
|
| VisualParamGenerator | `avatar_lad.xml` |
|
|
| SkeletonGenerator | `avatar_skeleton.xml` |
|
|
| AttentionsGenerator | `attentions.xml`, `attentionsN.xml` |
|
|
| TreesGenerator | `trees.xml`, `grass.xml` |
|
|
| GenepoolGenerator | `genepool.xml` |
|
|
|
|
The C# project does not provide the optional `visualparamtemplate.cs` file.
|
|
The pinned VisualParam generator therefore uses the template embedded in its
|
|
own source; the inventory records that generator source and hash as provenance.
|
|
|
|
The files under `inputs/` are exact, unmodified snapshots. Their license and
|
|
copyright attribution are covered by the repository
|
|
[`LICENSE.md`](../LICENSE.md). The generated source manifest repeats each hash
|
|
and license in its header and data table.
|
|
|
|
## Packet generation
|
|
|
|
The packet generator tokenizes the pinned `message_template.msg` directly in
|
|
Rust. It preserves source order, all 483 packet names, the 434 low/17 medium/32
|
|
high frequency assignments, 16-bit IDs (including the low-frequency `Fixed`
|
|
IDs), trust and zerocode flags, optional protocol flags, 905 blocks, repetition
|
|
rules, and every field width. Braces, identifiers, numeric values, field kinds,
|
|
prefix widths, duplicate names, and duplicate frequency/ID pairs are validated
|
|
with stable source-located `PGxxx` diagnostics.
|
|
|
|
[`packet_catalog.rs`](../crates/libremetaverse/src/packet_catalog.rs) is the
|
|
checked-in native output. It contains the public `PacketType` discriminants,
|
|
ordered immutable schema descriptors, name/type/frequency-ID dispatch tables,
|
|
and native construction and sizing implementations for every mapped packet and
|
|
block. Variable blocks start empty, fixed-repeat blocks receive their declared
|
|
number of independently constructed elements, and scalar/vector/UUID fields
|
|
receive the same zero defaults as the golden generator. The generated `Length`
|
|
metadata intentionally preserves the golden generator's variable-block count
|
|
semantics.
|
|
|
|
The same output contains the native wire implementation for every generated
|
|
packet and block. It preserves high-, medium-, and low-frequency headers,
|
|
network-order packet IDs, sequence numbers and appended ACKs; the reference
|
|
field endianness and widths; one- and two-byte variable prefixes; fixed-field
|
|
padding and truncation; fixed and variable block counts; normalized quaternion
|
|
encoding; zerocoding boundaries; and the reference `ToBytesMultiple` splitting
|
|
rules. Readers use checked bounds and allocation arithmetic and reject unknown
|
|
IDs or truncated payloads without panicking. Generated tests round-trip all 483
|
|
packet types and exercise every truncation boundary, while focused fixtures
|
|
assert exact reference bytes and malformed-input behavior.
|
|
|
|
Generation also compares all packet, block, field, field-type, and `PacketType`
|
|
entries against the pinned compiled public API catalog. This rejects a template
|
|
parser or naming change even if the emitted Rust would otherwise compile. The
|
|
Rust API compile fixture then compiles every mapped generated member, while the
|
|
packet catalog test exercises representative low/medium/high, fixed, multiple,
|
|
variable, and unknown dispatch behavior.
|
|
|
|
## Commands
|
|
|
|
Regenerate every currently registered Rust output using only checked-in data:
|
|
|
|
```console
|
|
cargo run -p libremetaverse-codegen -- generate
|
|
```
|
|
|
|
Verify hashes and reject stale output without writing:
|
|
|
|
```console
|
|
cargo run -p libremetaverse-codegen -- check
|
|
```
|
|
|
|
Maintainers can refresh the vendored snapshots from an explicitly supplied
|
|
checkout. The command verifies all six golden generator hashes and all input
|
|
hashes before writing, so a different upstream revision is rejected:
|
|
|
|
```console
|
|
cargo run -p libremetaverse-codegen -- vendor ../libremetaverse
|
|
```
|
|
|
|
Generation uses UTF-8 with an optional BOM, normalizes CRLF/CR to LF, rejects
|
|
NUL input, reports stable `path:line:column: severity[code]` diagnostics, sorts
|
|
manifest records by stable IDs, emits provenance-rich generated headers, and
|
|
always terminates output with one LF. Tests generate both outputs twice in
|
|
memory and require byte identity before comparing with the checked-in files.
|