hugo/RuDS
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Implement WordPress WXR analysis, review, and import #17

New Issue
Closed
opened 2026-07-18 20:48:25 +00:00 by hugo · 1 comment
hugo commented 2026-07-18 20:48:25 +00:00
Owner
Copy Link

Goal

Implement the complete bDS2-compatible WordPress migration workflow from saved import definition through review and recoverable execution.

Current state

  • The import_definitions SQLite table exists.
  • The Import sidebar and workspace tab are placeholders.
  • No WXR parser, analysis engine, execution engine, or repositories exist in Rust.
  • specs/import.allium and ImportAnalysisSurface in specs/editor_misc.allium are normative.

Required behavior

  • Create, rename, persist, reload, and delete project-scoped import definitions containing WXR path, optional uploads folder, and last analysis.
  • Parse untrusted WXR safely; require an RSS channel and extract site data, posts, pages, attachments, categories, and tags without creating unbounded identifiers.
  • Convert post HTML to Markdown and WordPress shortcodes such as gallery into bDS macros.
  • Analyze asynchronously without mutating project content. Classify posts/pages by slug and checksum as new, update, conflict, or content duplicate. Classify media by original filename/checksum and mark absent upload files missing.
  • Show counts, conflicts, missing media, years, macros, and taxonomy in a localized review UI.
  • Default conflicts to ignore; allow overwrite or import-as-new with a unique slug. Report same-content updates and duplicates but skip them during execution.
  • Match taxonomy case-insensitively and support manual or airplane-gated AI mapping.
  • Execute taxonomy, posts, media, and pages in that order, using 500-item transactions. Roll back the failing batch, retain earlier batches, and provide progress plus ETA.
  • Preserve source author/default author, timestamps, status, taxonomy, media alt text, and parent links through normal core engines.

Implementation notes

  • Read docs/UI_STYLE_GUIDE.md before UI work.
  • Reuse post, publishing, media, tag, sidecar, search, embedding, task, and event paths. Do not add import-only persistence paths.
  • Follow red/green TDD.
  • Use ../bDS2/lib/bds/wxr_parser.ex, import_analysis.ex, import_execution.ex, import_definitions.ex, import editor modules, and their tests when the spec is silent.

Acceptance criteria

  • Parser tests cover malformed XML, missing channel, unknown elements, HTML conversion, shortcodes, all item types, and upload resolution.
  • Analysis tests cover every classification and case-insensitive taxonomy matching without project mutation.
  • Execution tests cover ignore/overwrite/import, unique slugs, phases, 500-item batch rollback, progress callback failure, timestamps, status, media links, and AI airplane gating.
  • Saved analysis survives reopening the app.
  • The Import placeholders are replaced with a localized working workflow.
  • Imported metadata round-trips through filesystem, metadata diff, and rebuild.
  • allium check specs, cargo fmt --all -- --check, cargo build --workspace, and cargo test --workspace pass.
## Goal Implement the complete bDS2-compatible WordPress migration workflow from saved import definition through review and recoverable execution. ## Current state - The import_definitions SQLite table exists. - The Import sidebar and workspace tab are placeholders. - No WXR parser, analysis engine, execution engine, or repositories exist in Rust. - specs/import.allium and ImportAnalysisSurface in specs/editor_misc.allium are normative. ## Required behavior - Create, rename, persist, reload, and delete project-scoped import definitions containing WXR path, optional uploads folder, and last analysis. - Parse untrusted WXR safely; require an RSS channel and extract site data, posts, pages, attachments, categories, and tags without creating unbounded identifiers. - Convert post HTML to Markdown and WordPress shortcodes such as gallery into bDS macros. - Analyze asynchronously without mutating project content. Classify posts/pages by slug and checksum as new, update, conflict, or content duplicate. Classify media by original filename/checksum and mark absent upload files missing. - Show counts, conflicts, missing media, years, macros, and taxonomy in a localized review UI. - Default conflicts to ignore; allow overwrite or import-as-new with a unique slug. Report same-content updates and duplicates but skip them during execution. - Match taxonomy case-insensitively and support manual or airplane-gated AI mapping. - Execute taxonomy, posts, media, and pages in that order, using 500-item transactions. Roll back the failing batch, retain earlier batches, and provide progress plus ETA. - Preserve source author/default author, timestamps, status, taxonomy, media alt text, and parent links through normal core engines. ## Implementation notes - Read docs/UI_STYLE_GUIDE.md before UI work. - Reuse post, publishing, media, tag, sidecar, search, embedding, task, and event paths. Do not add import-only persistence paths. - Follow red/green TDD. - Use ../bDS2/lib/bds/wxr_parser.ex, import_analysis.ex, import_execution.ex, import_definitions.ex, import editor modules, and their tests when the spec is silent. ## Acceptance criteria - Parser tests cover malformed XML, missing channel, unknown elements, HTML conversion, shortcodes, all item types, and upload resolution. - Analysis tests cover every classification and case-insensitive taxonomy matching without project mutation. - Execution tests cover ignore/overwrite/import, unique slugs, phases, 500-item batch rollback, progress callback failure, timestamps, status, media links, and AI airplane gating. - Saved analysis survives reopening the app. - The Import placeholders are replaced with a localized working workflow. - Imported metadata round-trips through filesystem, metadata diff, and rebuild. - allium check specs, cargo fmt --all -- --check, cargo build --workspace, and cargo test --workspace pass.
hugo added the enhancement label 2026-07-18 20:48:25 +00:00
hugo commented 2026-07-19 11:18:54 +00:00
Author
Owner
Copy Link

Implemented and pushed in commit 429f471.

The implementation adds project-scoped saved import definitions and reopening, safe streaming WXR parsing, HTML5-to-Markdown and shortcode conversion, non-mutating async classification, localized review and conflict/taxonomy controls, airplane-aware AI taxonomy mapping, and recoverable taxonomy/posts/media/pages execution through the normal core engines. Execution preserves source status, authors, timestamps, taxonomy, media alt text, and parent links; skips updates/duplicates/missing items; supports ignore/overwrite/import-as-new; and rolls back database plus filesystem artifacts per 500-item batch while retaining earlier batches.

Verification completed: 12 focused WordPress import integration tests, saved-analysis UI reopening and locale completeness tests, metadata diff/rebuild round trips, allium check specs, allium analyse specs, cargo fmt --all -- --check, cargo build --workspace, strict workspace Clippy, cargo test --workspace, and visual inspection of the rebuilt macOS app bundle. The neutral review compared the result with issue #17, bDS2, and specs/import.allium and found no remaining gap.

Implemented and pushed in commit 429f471. The implementation adds project-scoped saved import definitions and reopening, safe streaming WXR parsing, HTML5-to-Markdown and shortcode conversion, non-mutating async classification, localized review and conflict/taxonomy controls, airplane-aware AI taxonomy mapping, and recoverable taxonomy/posts/media/pages execution through the normal core engines. Execution preserves source status, authors, timestamps, taxonomy, media alt text, and parent links; skips updates/duplicates/missing items; supports ignore/overwrite/import-as-new; and rolls back database plus filesystem artifacts per 500-item batch while retaining earlier batches. Verification completed: 12 focused WordPress import integration tests, saved-analysis UI reopening and locale completeness tests, metadata diff/rebuild round trips, allium check specs, allium analyse specs, cargo fmt --all -- --check, cargo build --workspace, strict workspace Clippy, cargo test --workspace, and visual inspection of the rebuilt macOS app bundle. The neutral review compared the result with issue #17, bDS2, and specs/import.allium and found no remaining gap.
hugo closed this issue 2026-07-19 11:19:13 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 wontfix
This won't be fixed
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
hugo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: hugo/RuDS#17
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?