# Native data generation

`libremetaverse-codegen` is the Rust-only framework for the six source
generators present at LibreMetaverse commit
`2aa70bb68513b39795da5d13c88f31b86e85a3ba`. Normal builds compile only the
checked-in Rust outputs. They do not run this tool, invoke .NET or Roslyn, read
an environment-dependent path, access the network, or open the sibling
reference checkout.

## Provenance inventory

[`sources.json`](sources.json) is the machine-readable inventory. It records
the upstream repository and commit, source-generator hashes, every actual
`AdditionalFile`, its vendored path, SHA-256, format, and BSD-3-Clause license.

| Golden generator | Pinned data inputs |
| --- | --- |
| PacketSourceGenerator | `message_template.msg` |
| VisualParamGenerator | `avatar_lad.xml` |
| SkeletonGenerator | `avatar_skeleton.xml` |
| AttentionsGenerator | `attentions.xml`, `attentionsN.xml` |
| TreesGenerator | `trees.xml`, `grass.xml` |
| GenepoolGenerator | `genepool.xml` |

The C# project does not provide the optional `visualparamtemplate.cs` file.
The pinned VisualParam generator therefore uses the template embedded in its
own source; the inventory records that generator source and hash as provenance.

The files under `inputs/` are exact, unmodified snapshots. Their license and
copyright attribution are covered by the repository
[`LICENSE.md`](../LICENSE.md). The generated source manifest repeats each hash
and license in its header and data table.

## Packet generation

The packet generator tokenizes the pinned `message_template.msg` directly in
Rust. It preserves source order, all 483 packet names, the 434 low/17 medium/32
high frequency assignments, 16-bit IDs (including the low-frequency `Fixed`
IDs), trust and zerocode flags, optional protocol flags, 905 blocks, repetition
rules, and every field width. Braces, identifiers, numeric values, field kinds,
prefix widths, duplicate names, and duplicate frequency/ID pairs are validated
with stable source-located `PGxxx` diagnostics.

[`packet_catalog.rs`](../crates/libremetaverse/src/packet_catalog.rs) is the
checked-in native output. It contains the public `PacketType` discriminants,
ordered immutable schema descriptors, name/type/frequency-ID dispatch tables,
and native construction and sizing implementations for every mapped packet and
block. Variable blocks start empty, fixed-repeat blocks receive their declared
number of independently constructed elements, and scalar/vector/UUID fields
receive the same zero defaults as the golden generator. The generated `Length`
metadata intentionally preserves the golden generator's variable-block count
semantics.

The same output contains the native wire implementation for every generated
packet and block. It preserves high-, medium-, and low-frequency headers,
network-order packet IDs, sequence numbers and appended ACKs; the reference
field endianness and widths; one- and two-byte variable prefixes; fixed-field
padding and truncation; fixed and variable block counts; normalized quaternion
encoding; zerocoding boundaries; and the reference `ToBytesMultiple` splitting
rules. Readers use checked bounds and allocation arithmetic and reject unknown
IDs or truncated payloads without panicking. Generated tests round-trip all 483
packet types and exercise every truncation boundary, while focused fixtures
assert exact reference bytes and malformed-input behavior.

Generation also compares all packet, block, field, field-type, and `PacketType`
entries against the pinned compiled public API catalog. This rejects a template
parser or naming change even if the emitted Rust would otherwise compile. The
Rust API compile fixture then compiles every mapped generated member, while the
packet catalog test exercises representative low/medium/high, fixed, multiple,
variable, and unknown dispatch behavior.

## Visual parameters and foliage

The visual-parameter generator parses the pinned `avatar_lad.xml` with explicit
root, attribute, numeric, vector, color, relationship, and duplicate-ID
validation. Its checked-in
[`visual_catalog.rs`](../crates/libremetaverse/src/visual_catalog.rs) output
contains all 672 public parameters, including ranges and defaults, groups,
wearable associations, alpha/color data, driven relationships, skeletal
distortions, and volume morphs. `VisualParams::params()` exposes a
`BTreeMap`-backed sorted list keyed by parameter ID. The 253 group-0/group-3
transmitted IDs are kept in ascending numeric order because that order is
observable in `AvatarAppearance` bytes; no other XML order is retained.

The foliage generator validates `trees.xml` and `grass.xml`, including UUIDs,
all numeric properties, unique IDs, and the contiguous source-order species
index required by the reference `Get` implementation. Its checked-in
[`foliage_catalog.rs`](../crates/libremetaverse/src/foliage_catalog.rs) output
uses source-ordered vectors, so tree and grass enum discriminants provide
constant-time indexed lookup. Golden fixtures cover all catalog counts,
representative complete records, public getters/setters, appearance decoding,
and every generated relationship category.

## Skeleton, attentions, and genepool

The skeleton generator validates the pinned hierarchy root, declared counts,
finite three-component transforms, unique joint names and aliases, collision
volumes, support categories, and child element schema. Its checked-in
[`skeleton_catalog.rs`](../crates/libremetaverse/src/skeleton_catalog.rs) output
constructs all 133 joints and 26 collision volumes in source hierarchy order.
The native skeleton records also implement depth-first enumeration, canonical
and alias lookup, expanded mesh-joint lists, the built-in loader, and validated
custom XML loading without a serializer or managed runtime.

The attention generator validates both pinned attention inputs independently,
requiring masculine and feminine sets with each of the nine named parameters
exactly once. The generated
[`attention_catalog.rs`](../crates/libremetaverse/src/attention_catalog.rs)
keeps the eleven `LookAtType` slots in enum-discriminant order, including the
observable zero-initialized `None` and `Clear` entries.

The genepool generator validates unique archetype names and parameter IDs,
finite values, and each ignored-but-schema-relevant texture UUID and slot. The
generated [`genepool_catalog.rs`](../crates/libremetaverse/src/genepool_catalog.rs)
preserves source archetype and parameter order for all 24 archetypes and 3,360
parameter values; exact-name lookup returns the original source index. Separate
generated headers retain the input hashes and BSD-3-Clause provenance for the
skeleton, both attention files, and genepool asset.

## Commands

Regenerate every currently registered Rust output using only checked-in data:

```console
cargo run -p libremetaverse-codegen -- generate
```

Verify hashes and reject stale output without writing:

```console
cargo run -p libremetaverse-codegen -- check
```

Maintainers can refresh the vendored snapshots from an explicitly supplied
checkout. The command verifies all six golden generator hashes and all input
hashes before writing, so a different upstream revision is rejected:

```console
cargo run -p libremetaverse-codegen -- vendor ../libremetaverse
```

Generation uses UTF-8 with an optional BOM, normalizes CRLF/CR to LF, rejects
NUL input, reports stable `path:line:column: severity[code]` diagnostics, sorts
manifest records by stable IDs, emits provenance-rich generated headers, and
always terminates output with one LF. Tests generate every registered output
twice in memory and require byte identity before comparing with the checked-in
files.