Files
RuDS/RUST_EXECUTION_BACKLOG.md

23 KiB
Raw Blame History

bDS Rust Rewrite — Execution Backlog

Purpose

This document converts the core and extension plans into execution order. It is organized by milestone first and by crate second.

Rules:

  1. Do not pull work forward from a later milestone unless it directly unblocks the current milestone.
  2. Keep tasks vertically sliced where possible: engine + UI + tests.
  3. Compatibility tasks outrank polish tasks.
  4. The allium specs in specs/ (synced from ../bDS2/specs/ on 2026-07-18) are the behaviour contract. When a task references a spec file, implement what the spec says and cover its invariants with unit tests.

Status (2026-07-18)

  • M0M2: complete (items marked DONE below).
  • M3: implemented (all views and the editor exist in crates/bds-ui/src/views/); items below are not individually ticked — verify against tests before re-doing anything.
  • M4: in progress per git history ("M4 closing" commits). bds-core/src/render/ exists (generation, macros, markdown, page renderer, routes, site, template lookup). bds-core/src/ai/ does not exist yet — the one-shot AI client is unstarted.
  • M5: not started (bds-core/src/scripting/ is an empty stub; no PublishEngine).
  • M6 (below): new — code-vs-spec gaps introduced by the bDS2 spec sync. Fold these in as the milestones they touch come up; none of them should wait for "later" if the code they touch is being written now.

Milestone M0: Compatibility Baseline

bds-core

  • create workspace and crate boundaries (bds-core, bds-editor, bds-ui, bds-cli) DONE
  • add typed SQLite connection and query layer via diesel DONE
  • add embedded migration loader via diesel_migrations DONE
  • define initial shared model modules with serde derives DONE
  • add checksum (sha2) and slug (deunicode) utilities DONE
  • establish error handling conventions: thiserror for bds-core, anyhow for bds-ui/bds-cli DONE
  • add tokio runtime as workspace dependency (used by bds-ui and bds-cli, not directly by bds-core engine code) DONE

bds-editor

  • create crate with ropey, syntect, cosmic-text dependencies DONE
  • implement basic rope buffer wrapper with edit operations DONE
  • implement syntect integration for markdown highlighting DONE
  • implement cosmic-text layout for rendering highlighted text DONE (measured monospace font metrics via cosmic-text FontSystem/Buffer; OnceLock-cached MonoMetrics replaces hardcoded constants)
  • implement basic cursor model (position, move, place by click) DONE (up/down/left/right/home/end + click placement)
  • implement basic text insertion and deletion DONE (insert, backspace, delete forward, enter)
  • implement Iced custom widget that composes buffer + highlight + layout + cursor DONE (CodeEditor widget with gutter, text, cursor rendering + keyboard/mouse events)
  • implement basic vertical scrolling with viewport-aware rendering DONE (scroll_by, ensure_cursor_visible, mouse wheel, viewport-clipped rendering)
  • verify IME composition events work through winit (early risk check) DONE (verified: winit/Iced delivers composed chars via Key::Character; pre-edit display deferred to M3)

bds-ui

  • create app entry point with Iced Application impl DONE
  • wire Iced window creation DONE
  • wire muda menu bar with skeleton App/File/Edit/View/Window/Help menus DONE (stub)
  • wire macOS lifecycle shim via objc2 (application:openFile:, application:openURLs:) behind cfg gate DONE (stub)
  • wire muda MenuEvent receiver as Iced Subscription DONE (platform/menu.rs menu_subscription + app.rs subscription)

fixtures and docs

  • collect representative fixture projects from current app DONE (fixtures/compatibility-projects/rfc1437-sample/)
  • capture golden generated output for those fixtures DONE (fixtures/golden-generated-sites/rfc1437-sample/ — subset from live TypeScript-generated site: 3 post pages × 2 languages, structural files, assets, feeds, sitemap, category page)
  • create initial compatibility inventory from the matrix template (must include mediaTranslations, postLinks, FTS5 tables) DONE (docs/COMPATIBILITY_INVENTORY.md)
  • create Liquid feature inventory from default templates DONE (docs/LIQUID_FEATURE_INVENTORY.md)
  • determine whether current app uses Pagefind or another client-side search index — if Pagefind, plan for pagefind crate library integration (not CLI binary) DONE (confirmed: TypeScript app uses pagefind_extended CLI; Rust will use pagefind crate library API)
  • create slug compatibility test suite comparing deunicode vs transliteration output on fixture content DONE
  • create Iced architecture patterns document (message design, subscription model, custom widget patterns) DONE (docs/ICED_ARCHITECTURE_PATTERNS.md)

Validation

  • DB readability tests (all tables including mediaTranslations, postLinks, FTS5, and AI/catalog tables) DONE (59 unit tests + 29 fixture integration tests)
  • app launch smoke test (Iced window + muda menus) DONE (tests/app_smoke.rs — type-level tests; muda requires main thread so full launch tested via cargo run)
  • bds-editor PoC test: renders highlighted markdown, accepts keyboard input, cursor moves DONE (tests/editor_poc.rs — 11 integration tests covering highlight + input + cursor + scroll)
  • fixture harness test DONE (tests/fixture_readability.rs)
  • slug compatibility tests DONE

Milestone M1: Data Fidelity

bds-core/db

  • verify schema compatibility against existing projects DONE
  • add FTS5 virtual tables (posts_fts, media_fts) for in-app full-text search DONE (multi-column schema: title/excerpt/content/tags/categories with per-language snowball stemming, 24 languages)
  • verify mediaTranslations table read/write DONE
  • verify postLinks table read/write DONE

bds-core/engine

  • implement ProjectEngine DONE
  • implement PostEngine DONE
  • implement MediaEngine DONE
  • implement PostMediaEngine DONE
  • implement TagEngine DONE
  • implement MetaEngine DONE
  • implement metadata diff flow (posts, media, scripts, templates) DONE
  • implement rebuild-from-filesystem flow DONE

bds-core/util

  • frontmatter parser/writer via serde_yaml DONE
  • translation file parser/writer DONE
  • sidecar parser/writer DONE
  • content hashing support via sha2 DONE
  • thumbnail generation via image crate (resize, WEBP encoding) DONE
  • recursive directory traversal via walkdir (for rebuild-from-filesystem) DONE

Validation

  • round-trip persistence tests DONE (38 m1_validation integration tests)
  • rebuild tests DONE
  • metadata diff tests DONE (full coverage matrix: 13 post fields, 6 media, 4 template, 5 script)
  • golden-file comparisons for written files DONE

Milestone M2: Native Workspace

bds-ui/platform

  • muda menu bar: full menu tree with accelerators DONE (platform/menu.rs with muda MenuBar, menu_subscription, accelerators)
  • menu enable/disable validation synced to app state DONE (sync_menu_state() evaluates has_project/has_tab/offline_mode after every state change)
  • menu event routing as Message variants DONE (MenuEvent → Message mapping in app.rs subscription)
  • rfd integration for file/folder open and save dialogs DONE (platform/dialog.rs with i18n-aware pick_folder/pick_media_files)
  • macOS lifecycle shim: file-open and URL-open events forwarded as Message variants DONE (objc2 BdsAppDelegate with application:openFile: and application:openURLs: → mpsc channel → Message::FileOpenRequested/UrlOpenRequested)

bds-ui/app

  • root Message enum and update() dispatcher DONE (Message enum with menu, lifecycle, sidebar, tab, editor variants + update dispatcher)
  • app state model DONE (AppState with project, sidebar, editor, output panel state; sidebar post list with 500-post limit)
  • task surface model DONE (engine/task.rs TaskManager with concurrency limit, FIFO queue, progress reporting, cancel support)

bds-ui/views

  • workspace layout DONE (three-panel layout: sidebar + editor + optional right panel)
  • sidebar DONE (post list with search, status filter, category/tag counts)
  • activity bar DONE (VS Code-style 50px bar with SVG icons for Posts/Pages/Media/Scripts/Templates/Tags/Chat/Import/Git/Settings)
  • tab bar DONE (editor tab bar with open/close/switch)
  • status bar DONE (bottom bar with project name, task progress)
  • project selector DONE (project open dialog and recent projects)
  • toast notifications DONE (overlay toast stack with auto-dismiss, 4 severity levels, i18n messages; replaces output-only notifications)

Validation

  • menu event → Message routing integration tests DONE (tests/menu_routing.rs — 4 tests: i18n key prefix, all-locale coverage, action count, no duplicates)
  • keyboard shortcut integration tests DONE (tests/m2_validation.rs — 15 accelerator-bound actions verified)
  • rfd dialog invocation tests DONE (tests/m2_validation.rs — dialog i18n keys in all 5 locales)
  • fixture project open flow test DONE (tests/project_flow.rs — 6 tests: create, switch, delete, directory, meta files)
  • toast notification tests DONE (tests/m2_validation.rs — toast id monotonicity, levels, expiry, message preservation, i18n keys)
  • menu enable/disable rule tests DONE (tests/m2_validation.rs — project-gated 11 items, tab-gated 4 items, offline-gated 2 items)

Milestone M3: Authoring

bds-editor (maturation)

  • full cursor movement (arrows, word, line, page, home/end)
  • selection (shift+movement, click-and-drag, double-click word select)
  • system clipboard integration (copy/cut/paste)
  • undo/redo with edit grouping
  • line numbers gutter
  • incremental syntax rehighlighting on edits
  • IME input handling for CJK and other input methods
  • configurable soft wrap
  • additional syntax grammars: Liquid/HTML, Lua, YAML, JSON

bds-core/engine

  • expose editor-safe save APIs
  • expose publish/unpublish/discard flows
  • expose template validation APIs
  • expose script validation APIs

bds-ui/views

  • sidebar post filtering: text search box, status filter, tag/category filter dropdowns, language filter, year/month selectors, date range picker — wired to existing PostSearchFilters / search_posts_filtered() in bds-core. The 500 post limit applies AFTER filtering, not before.
  • dashboard
  • post editor (bds-editor with markdown + YAML frontmatter highlighting)
  • translation editor (bds-editor)
  • media browser
  • media editor
  • tags view
  • settings view
  • templates view and editor (bds-editor with Liquid/HTML highlighting)
  • scripts view and editor (bds-editor with Lua highlighting)

bds-ui/components

  • inputs
  • modals
  • reusable list rows and panels

Validation

  • entity create/edit/save flows
  • template validation flow
  • script validation flow
  • editor widget tests: cursor movement, selection, undo/redo, clipboard, IME

Milestone M4: Rendering Parity

bds-core/render

  • markdown render pipeline via pulldown-cmark
  • Liquid render pipeline via liquid crate (scoped to the feature subset documented in the Liquid inventory: if/elsif/else, for, assign, render, whitespace stripping, plus filters: default, escape, url_encode, append; .size is property access on arrays, not a pipe filter)
  • custom Liquid filter: i18n (translation lookup by key and language)
  • custom Liquid filter: markdown (markdown-to-HTML with macro expansion, URL rewriting, media path canonicalization)
  • built-in macro renderers: gallery, youtube, vimeo, photo_archive, tag_cloud (native Rust, not Lua)
  • template resolution rules
  • URL rewriting
  • RSS/Atom feed generation via quick-xml
  • sitemap generation via quick-xml
  • generated file hash tracking (generatedFileHashes table for skip-unchanged-writes)
  • Pagefind search index generation via pagefind crate library API (PagefindIndex::add_html_file() + get_files()) — if required for parity (determine in M0 inventory)

bds-core/ai

  • one-shot AI client: reqwest + serde_json against OpenAI-compatible Chat Completions endpoint
  • two-endpoint configuration: online endpoint (URL, API key, model) + airplane mode endpoint (URL, model)
  • AI translate post operation (title, excerpt, content to target language)
  • AI translate media metadata operation (title, alt, caption to target language)
  • AI image description / alt text generation operation
  • AI post analysis operation (title, excerpt, slug suggestion)
  • AI taxonomy analysis operation (tag, category suggestions)
  • AI language detection operation
  • API key storage via OS keychain (macOS Keychain, Windows DPAPI, Linux libsecret)
  • airplane mode gating: block online endpoint, use airplane endpoint or show toast
  • error handling: surface failures as user-visible feedback, never silent

bds-core/engine

  • TemplateEngine
  • PageRenderer (parallelize rendering via rayon)
  • BlogGenerationEngine
  • PreviewServer (localhost HTTP via axum on tokio)
  • SearchIndexEngine only if required for parity

bds-ui/views

  • internal Markdown/Preview switch backed by Wry, plus external Open in Browser command
  • generation progress display
  • render errors and diagnostics
  • AI operation triggers in post editor (analysis, taxonomy), translation editor (translate), and media editor (alt text, translate)
  • AI endpoint configuration in settings view (online + airplane mode endpoints)

Validation

  • golden generated-site comparisons
  • preview route coverage
  • template compatibility suite
  • one-shot AI client tests (mocked endpoint: all 6 operations)
  • AI endpoint configuration persistence tests
  • AI airplane mode gating tests

Milestone M5: Operate And Ship

bds-core/engine

  • PublishEngine (SSH/SCP via ssh2, rsync via child process)
  • validation and integrity services
  • filesystem watcher via notify (if needed for detecting external changes to open content)

bds-core/scripting

  • ScriptEngine
  • LuaRuntime via mlua (Lua 5.4, vendored) — user-authored scripts only; built-in macros are native Rust in bds-core/render
  • LuaApi (expose post, media, tag, and project data to Lua scripts via mlua::UserData trait)
  • user-authored Lua macro execution at render time
  • user-authored transform and utility script execution
  • scripting docs generator

bds-ui/views

  • publish workflow screens
  • publish progress and failure surfaces
  • script docs access points if needed for core usability

Validation

  • publish end-to-end tests
  • Lua API bridge tests
  • built-in macro compatibility tests
  • docs sync tests

Milestone M6: bDS2 Spec Parity

Code-vs-spec gaps from the 2026-07-18 spec sync. Each task names its authoritative spec. Schema changes go through generated Diesel migrations, and every new metadata field must be wired into publishing, metadata-diff, and rebuild-from-filesystem together.

bds-core/util

  • slug uniqueness: unbounded numeric suffix {slug}-2, -3, … — remove the 999 cap and timestamp fallback in util/slug.rs (post.allium)

bds-core/db

  • migration: add cache_read_tokens and cache_write_tokens (nullable integers) to ai_usage (schema.allium)
  • migration: rename npmpackage_ref and align provider_npm naming (schema.allium)
  • media FTS search: accept structured filters alongside the query (search.allium SearchMediaRequested(query, filters))

bds-core/engine

  • project open: discover data_path from the location of meta/project.json; never persist it in project.json; keep a machine-local project registry under the OS app-data dir (project.allium DiscoverProjectDataPath, DataPathNotPersistedInProjectJson)
  • verify public/private data placement matches project.allium (PublicContentLivesInProjectFolder, PrivateArtifactsLiveInOsAppDir): DB, embeddings index, model cache, registry, and window state in the OS app-data dir; everything user-owned in the project folder
  • menu: file-only model (no DB table), HomeAlwaysFirst normalization, SyncMenuFromFilesystem flow (menu.allium)
  • media: optional import metadata (title, alt, caption, author, language, tags), ReplaceMediaFile flow, batch-import linked-image processing (media.allium)
  • metadata diff: compare only translation-specific fields for translation files; per-item repair direction; embedding entity type (metadata_diff.allium)
  • translations: ReopenPublishedTranslation on edit; TranslationFilesCarryFullMetadata (translation.allium)
  • auto-translation pipeline: ScheduleAutoTranslation, AutoTranslatePost, AutoTranslateMediaCascade, FillMissingTranslations — gated by configured AI endpoint and airplane mode, skips doNotTranslate, fills only missing languages (translation.allium; depends on M4 AI client)
  • task manager: allow cancelling pending (queued) tasks, not only running ones (task.allium)

bds-core/render (fold into M4)

  • implement rendering.allium as the shared render-assigns contract for preview and generation; content language only, never UI locale
  • expose slugify as a custom Liquid filter alongside i18n and markdown (rendering.allium)
  • generation task structure: task group with one task per section (Site Core, Single Posts, Category/Tag/Date Archives) + final Build Search Index task; stream one progress message per written page — "Generated /url (n/total)" or "Rewrote …" for validation apply (generation.allium)
  • enforce GenerationPublishedOnly and the specified language variant selection (generation.allium)

bds-core/ai (fold into M4)

  • honour ai.allium invariants in the one-shot client and settings: endpoint add/remove rules, advisory model catalog with conditional refresh, provider detection, vision capability gate, airplane-mode model swap
  • record cache token usage in ai_usage (schema.allium)

bds-core/scripting (fold into M5)

  • configurable script_extension (default "lua"): file path scripts/{slug}.{extension}; entrypoint default "render" for macros, "main" otherwise (script.allium, frontmatter.allium)
  • sandboxed Lua runtime: no filesystem mutation, process control, or package loading; host capabilities only via explicit bds.* API (script.allium SandboxedExecution, ExplicitHostCapabilities)
  • enforce config limits: macro timeout 10s; transform toasts max 5/script, 20 total, 300 chars; blogmark title ≤ 200, URL ≤ 2048 (script.allium)

bds-ui

  • settings: technology section (semantic similarity toggle), data section (rebuild buttons: posts/media/scripts/templates/links/thumbnails/embedding + Open Data Folder), agent integrations (Claude Code + GitHub Copilot; others disabled with "not supported yet" note), image import concurrency (18, default 4), default editor mode, settings in a separate tab (editor_settings.allium)
  • media search filter UI wired to the filtered media search
  • auto-translation and repair-direction UI hooks for the engine flows above

Validation

  • unit tests for every invariant and rule named above, per AGENTS.md spec-coverage requirement
  • re-run allium check specs/*.allium after any spec edit

Extension Backlog

Bucket A: Git And Validation

bds-core

  • GitEngine via git2 (shell out for LFS operations)
  • richer site validation service support

bds-ui

  • Git sidebar
  • diff view
  • richer metadata diff UI

Bucket B: Import

bds-core

  • WXR parser
  • import analysis
  • import execution
  • import definitions

bds-ui

  • import wizard
  • import progress and result views

Bucket C: AI Chat And Tool Use

bds-core

  • streaming client extension (SSE parsing on top of core reqwest client)
  • tool execution framework (tool definitions, call routing, result formatting)
  • multi-turn conversation management

bds-ui

  • chat sidebar
  • chat panel
  • provider settings UI (extends core AI endpoint configuration)

Bucket D: Embeddings And Duplicates

bds-core

  • embedding engine via ort (ONNX Runtime)
  • vector index via usearch (HNSW)
  • duplicate detection logic

bds-ui

  • semantic search UI
  • duplicates view

Bucket E: Translation QA And Docs UX

bds-core

  • translation validation engine

bds-ui

  • translation validation view
  • documentation browser

bds-core

  • menu editing services
  • deep-link flow support beyond core shell hooks

bds-ui

  • menu editor UI

Bucket G: MCP And Automation

bds-cli

  • CLI command set per cli.allium: rebuild | repair | render | upload | push | pull | post | media | gallery | config | project | lua — same database, settings, and cache as the GUI; no listeners, no window; logging redirected to the rotating log file
  • MCP server surface per mcp.allium (incl. translation tools, stats/media resources, proposal status)
  • CLI-to-app synchronization via the domain event bus (events.allium): entity mutations broadcast (entity, entity_id, action) on one topic shape covering in-app and external writes

bds-core

  • domain event bus (events.allium): broadcast after every successful mutation of posts, media, tags, templates, scripts; SettingsChangedEvent for global settings

Bucket K: Headless Server Mode (server.allium)

bds-core / bds-cli

  • boot modes desktop | server | tui resolved from an env var at start
  • headless server: no window; SSH transport for remote TUI/GUI clients
  • SSH key material generated on first boot in the private app-data dir (host key + empty authorized_keys, mode 600) — never in the repo or project folder

Bucket L: Terminal UI (tui.allium)

new crate bds-tui (or feature in bds-cli)

  • TUI as a second renderer over the same shared UI core: sidebar views (posts/media/templates/scripts/tags/settings/git), sidebar/editor focus model, vim-style + arrow navigation skipping section headers
  • post editor drives the same workflow as the GUI: canonical-language edits update the post, other languages update translations; publish routes through the same pipeline
  • subscribes to the domain event bus (Bucket G) for live updates

Bucket H: Blogmark And Transforms

bds-core

  • BlogmarkTransformService
  • transform script chain execution

bds-ui

  • external content capture workflow

Bucket I: Rich Editor

bds-editor (evolution)

  • inline bold/italic/header rendering via cosmic-text mixed font styles
  • inline image preview via Iced image rendering within the custom widget
  • macro block preview (render macro output inline)
  • clickable links

bds-ui

  • rich editor mode toggle
  • image insert dialog from linked media

Bucket J: A2UI Surfaces

bds-ui

  • A2UI component renderer
  • A2UI surface manager
  • AI assistant dynamic surface integration

Exit Rule

Do not start extension buckets until Milestones M5 and M6 are complete and the Rust app is already a credible replacement for the baseline authoring and publishing workflow.