StructuredData: implement JSON and Protobuf OSD formats #36

New Issue

hugo · 2026-08-08T08:16:58Z

hugo commented

2026-08-08 08:16:58 +00:00

Objective

Implement the JSON and Protobuf adapters with explicit conversions to the OSD model.

Deliverables

Preserve LibreMetaverse JSON tagging/coercion behavior rather than relying on default Serde mappings.
Determine and document Protobuf stability, field tags, numeric widths, unknown fields, and compatibility policy.
Keep schema/wire details private unless they are public contracts.

Validation

All translated JSON/Protobuf tests and golden fixtures pass.
Cross-language golden bytes/JSON match the pinned reference.
Unknown and malformed data behavior is explicit and bounded.

Prerequisite

Milestone 04 supplies implemented value types and the global API/test gates are complete.

Project constraints

Native Rust only; no LibreMetaverse/.NET bridge or fallback.
Keep mapped signatures fixed and preserve translated assertions exactly.
Enforce bounded parsing for untrusted input and update docs and parity status.

## Objective Implement the JSON and Protobuf adapters with explicit conversions to the OSD model. ## Deliverables - Preserve LibreMetaverse JSON tagging/coercion behavior rather than relying on default Serde mappings. - Determine and document Protobuf stability, field tags, numeric widths, unknown fields, and compatibility policy. - Keep schema/wire details private unless they are public contracts. ## Validation - All translated JSON/Protobuf tests and golden fixtures pass. - Cross-language golden bytes/JSON match the pinned reference. - Unknown and malformed data behavior is explicit and bounded. ## Prerequisite Milestone 04 supplies implemented value types and the global API/test gates are complete. ## Project constraints - Native Rust only; no LibreMetaverse/.NET bridge or fallback. - Keep mapped signatures fixed and preserve translated assertions exactly. - Enforce bounded parsing for untrusted input and update docs and parity status.

hugo added this to the 05 - StructuredData implementation milestone 2026-08-08 08:16:58 +00:00

hugo added the enhancement label 2026-08-08 08:16:58 +00:00

hugo commented

2026-08-09 01:31:25 +00:00

Implemented in commit 6d02db8 (pushed to origin/main).

Implementation:

Added explicit native JSON token-to-OSD conversion for bool, numeric-width coercion, empty strings/null, arrays, maps, and duplicate-property errors; no derived/default Serde mapping is used.
Implemented C# preserveDefaults behavior, including root/array null defaults, omitted map defaults, binary byte arrays, typed values as strings, stable map ordering, and System.Text.Json-compatible escaping.
Added the private OSD Protobuf schema directly in Rust: fixed field tags, ZigZag int32, little-endian fixed64 real/date values, 16-byte UUIDs, length-delimited strings/binary/maps/arrays, optional exact header, deterministic maps, and wire-aware unknown-field skipping.
Enforced input/output, depth, node, allocation, varint, and length bounds with positioned malformed-input errors.
Added checked-in JSON and Protobuf golden fixtures, native round-trip/malformed/unknown-field tests, stream overload coverage, native generator mappings, docs, dependency locks, dispatch integration, coverage, and parity baseline updates.
StructuredData now has native implementations for all 295 mapped members and no generated failure stubs.

Verification:

All 12 pre-created Protobuf compatibility tests pass in isolation.
The complete pre-created StructuredData suite passes: 78/78.
All 31 libremetaverse-structured-data unit tests pass.
Controlled red-suite audit is current: 179 pass, 1,203 expected standardized failures, no unrelated failures.
cargo fmt, mapping/shim regeneration, API coverage, parity/surface, workspace/all-target check, locked API fixture, workspace test compilation, all-target/all-feature clippy with warnings denied, and workspace rustdoc all pass.

Reviewed against issue #36 and the pinned OSDJson.cs and OSDProtobuf.cs golden references. The implementation is native Rust only and contains no bridge, fallback, or stubbed codec paths.

Implemented in commit 6d02db8 (pushed to origin/main). Implementation: - Added explicit native JSON token-to-OSD conversion for bool, numeric-width coercion, empty strings/null, arrays, maps, and duplicate-property errors; no derived/default Serde mapping is used. - Implemented C# preserveDefaults behavior, including root/array null defaults, omitted map defaults, binary byte arrays, typed values as strings, stable map ordering, and System.Text.Json-compatible escaping. - Added the private OSD Protobuf schema directly in Rust: fixed field tags, ZigZag int32, little-endian fixed64 real/date values, 16-byte UUIDs, length-delimited strings/binary/maps/arrays, optional exact header, deterministic maps, and wire-aware unknown-field skipping. - Enforced input/output, depth, node, allocation, varint, and length bounds with positioned malformed-input errors. - Added checked-in JSON and Protobuf golden fixtures, native round-trip/malformed/unknown-field tests, stream overload coverage, native generator mappings, docs, dependency locks, dispatch integration, coverage, and parity baseline updates. - StructuredData now has native implementations for all 295 mapped members and no generated failure stubs. Verification: - All 12 pre-created Protobuf compatibility tests pass in isolation. - The complete pre-created StructuredData suite passes: 78/78. - All 31 libremetaverse-structured-data unit tests pass. - Controlled red-suite audit is current: 179 pass, 1,203 expected standardized failures, no unrelated failures. - cargo fmt, mapping/shim regeneration, API coverage, parity/surface, workspace/all-target check, locked API fixture, workspace test compilation, all-target/all-feature clippy with warnings denied, and workspace rustdoc all pass. Reviewed against issue #36 and the pinned OSDJson.cs and OSDProtobuf.cs golden references. The implementation is native Rust only and contains no bridge, fallback, or stubbed codec paths.

hugo closed this issue

2026-08-09 01:31:31 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/MetaCrate#36