Align file serialization with bDS2 canonical output (frontmatter, sidecars, meta JSON, menu.opml) #99
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
RuDS's file serializers were written to byte-match the old TypeScript bDS output (golden tests in
crates/bds-core/src/util/frontmatter.rsassert againstfixtures/compatibility-projects/rfc1437-sample). bDS2 is the canonical baseline and its serializers (../bDS2/lib/bds/frontmatter.ex,../bDS2/lib/bds/sidecar.ex,../bDS2/lib/bds/metadata.ex,../bDS2/lib/bds/menu.ex) produce different bytes. RuDS must be changed to write exactly what bDS2 writes. Existing files on disk in old formats do not matter; only RuDS write behaviour vs bDS2 write behaviour matters. Both parsers are lenient, so reading stays compatible — every change below is write-side.1. Post frontmatter (
crates/bds-core/src/util/frontmatter.rs,PostFrontmatter::to_yaml)Align with
../bDS2/lib/bds/posts/file_sync.exserialize_post_file/2+../bDS2/lib/bds/frontmatter.ex:id, title, slug, excerpt, status, author, language, doNotTranslate, templateSlug, createdAt, updatedAt, publishedAt, tags, categories. (RuDS currently: id, title, slug, status, createdAt, updatedAt, tags, categories, excerpt, author, language, doNotTranslate, templateSlug, publishedAt.)^[\p{L}\p{N} ._/-]+$unquoted; otherwise double-quotes with\\,\",\nescaped. RuDS currently single-quotes on a character blacklist. Note bDS2 writes a title liketrueor123unquoted (it re-parses as bool/int — that is canonical behaviour).tags:line with no items (frontmatter serializer list clause), nottags: [].createdAt: '2024-03-30T21:20:00.000Z') — already matches.doNotTranslateonly when true).\nto the file (serialize_documentjoins with a final empty element). RuDS must normalize identically instead of writing the body as-is.Reference expected output:
../bDS2/test/bds/compatibility_serializer_parity_test.exs(first test).2. Translation frontmatter (
TranslationFrontmatter::to_yaml)Field order already matches bDS2 (
id, translationFor, language, title, excerpt, status, createdAt, updatedAt, publishedAt). Apply the same quoting rules and one-trailing-newline rule as above.3. Template & script frontmatter (
TemplateFrontmatter::to_yaml,ScriptFrontmatter::to_yaml)bDS2 (
../bDS2/lib/bds/templates.exserialize_template_file/2,../bDS2/lib/bds/scripts.exserialize_script_file/2) routes through the same frontmatter serializer as posts. RuDS currently double-quotes every string including timestamps (TS style). Must change to:id, projectId, slug, title, kind, [entrypoint,] enabled, version, createdAt, updatedAt).4. Media sidecars (
crates/bds-core/src/util/sidecar.rs)Align with
../bDS2/lib/bds/media/sidecars.exwrite_sidecar/2+../bDS2/lib/bds/sidecar.ex:linkedPostIdsbeforetags(RuDS currently writes tags first).linkedPostIdsmust be written aslinkedPostIds: [](RuDS currently omits the line).\n->\\n(RuDS currently only escapes\\and"; a caption containing a newline produces a broken sidecar).originalName/title/alt/caption/author, maybe-quoted rule for other strings, width/height omitted when absent, no trailing newline after closing---.translationFor, language, title, alt, caption) already matches in order; apply the escaping fix.Reference expected output:
../bDS2/test/bds/compatibility_serializer_parity_test.exs(sidecar tests).5. meta/ JSON files (
crates/bds-core/src/engine/meta.rs,crates/bds-core/src/model/metadata.rs,crates/bds-core/src/model/generation.rs)bDS2 writes via
Jason.encode!(payload, pretty: true)over Elixir maps, which emits keys in sorted (alphabetical, byte-order) order. RuDS serializes serde structs in declaration order. To match bytes:blogLanguages, blogmarkCategory, defaultAuthor, description, imageImportConcurrency, mainLanguage, maxPostsPerPage, name, picoTheme, publicUrl, semanticSimilarityEnabled(nil keys omitted).sshHost, sshMode, sshRemotePath, sshUser(nil keys omitted, sshMode always present, defaultscp).color, name, postTemplateSlug(color/postTemplateSlug omitted when blank). Sorting by lowercased name already matches.HashMap, so top-level category order is non-deterministic — even two consecutive RuDS writes of unchanged data can differ. Use a sorted map (BTreeMap) to match bDS2's sorted key output. Per-category key order must be sorted too:listTemplateSlug, postTemplateSlug, renderInLists, showTitle, title(nil omitted; renderInLists/showTitle always written).Enum.sort); RuDS sorts case-insensitively. Match bDS2.Simplest approach for key order: either rename/reorder struct fields alphabetically, or serialize via
serde_json::Mapbuilt in sorted order. Verify empty-map/empty-array rendering matches Jason ({}/[]).6. meta/menu.opml (
crates/bds-core/src/engine/menu.rs,serialize_opml)Align with
../bDS2/lib/bds/menu.exserialize_opml/2:<title>must be the project name (RuDS hardcodesBlog Menu), plus<dateCreated>and<dateModified>set to the project's updated_at/created_at ISO timestamp. RuDS currently drops these on every write.<outline ... />with a space before/>; quick_xml emits<outline .../>. Match bDS2.\n; RuDS does not.text,type,pageSlug/categoryName),homehandling, indentation (2 spaces per level) already match.Test fallout
frontmatter.rs/sidecar.rsand therfc1437-samplefixtures assert the old TS format; they must be updated (or fixtures regenerated with bDS2) to assert the bDS2 format. bDS2'scompatibility_serializer_parity_test.exsis the authoritative expected-output reference.specs/post.allium,specs/media_processing.allium,specs/schema.allium) and validate the spec if it needs tending.Implemented in
7b8e539. All shared filesystem serializers now byte-match bDS2 write behavior: post/translation/template/script frontmatter field order, Unicode scalar quoting, empty-list form, timestamp quoting, and terminal-newline normalization; media/translation sidecar escaping plus linkedPostIds ordering and empty emission; deterministic project/publishing/tags/category metadata JSON key order, omission/default behavior, case-sensitive category sorting, and compact empty collections; and canonical menu.opml project head metadata, spaced self-closing outlines, and trailing newline. Readers remain compatible with legacy TypeScript fixtures and now normalize terminal body newlines/scalar values so unchanged published entities remain unchanged. Default project metadata writers and the desktop menu-save path use the same canonical serializers. README and Allium contracts were updated. Neutral review checked every numbered requirement against issue #99, bDS2's frontmatter/sidecar/metadata/menu implementations, and the Allium rules. Verification: exact-byte parity tests for every format and empty/default variants; legacy fixture and metadata-diff integration matrix; cargo test --workspace (all green), cargo build --workspace, strict cargo clippy --workspace --all-targets -D warnings, cargo machete, cargo fmt --check, git diff --check, and allium check specs/*.allium.