From a594b99e90f4a09fc785de2f008681561f3c2fed Mon Sep 17 00:00:00 2001 From: Chili Palmer Date: Sat, 18 Jul 2026 10:16:30 +0200 Subject: [PATCH] chore: brought RuDS up to bDS2 alignment, as that is the new baseline we want to hit --- AGENTS.md | 6 +- RUST_COMPATIBILITY_MATRIX_TEMPLATE.md | 4 +- RUST_EXECUTION_BACKLOG.md | 91 ++++++- RUST_PLAN.md | 51 ++-- RUST_PLAN_CORE.md | 53 +++- RUST_PLAN_EXTENSION.md | 56 +++- SPECIFICATION_INDEX.md | 10 + docs/RESEARCH_FINDINGS.md | 2 +- specs/action_patterns.allium | 2 +- specs/ai.allium | 238 +++++++++++++++-- specs/bds.allium | 25 +- specs/cli.allium | 175 +++++++++++++ specs/cli_sync.allium | 12 +- specs/editor_chat.allium | 80 ++++++ specs/editor_media.allium | 19 +- specs/editor_misc.allium | 23 +- specs/editor_post.allium | 28 +- specs/editor_script.allium | 31 ++- specs/editor_settings.allium | 78 ++++-- specs/editor_template.allium | 17 +- specs/embedding.allium | 87 ++++++- specs/engine_side_effects.allium | 6 +- specs/events.allium | 59 +++++ specs/frontmatter.allium | 176 ++++++++----- specs/generation.allium | 86 ++++++- specs/git.allium | 2 +- specs/layout.allium | 7 +- specs/mcp.allium | 108 ++++++-- specs/media.allium | 76 +++++- specs/media_processing.allium | 55 +++- specs/menu.allium | 55 ++-- specs/metadata.allium | 22 ++ specs/metadata_diff.allium | 41 ++- specs/modals.allium | 2 +- specs/post.allium | 138 ++++++++-- specs/preview.allium | 33 ++- specs/project.allium | 89 ++++++- specs/publishing.allium | 10 +- specs/rendering.allium | 219 ++++++++++++++++ specs/schema.allium | 358 ++++++++++++++++++++++++-- specs/script.allium | 220 +++++++++++++--- specs/search.allium | 127 ++++++--- specs/server.allium | 90 +++++++ specs/sidebar_views.allium | 5 +- specs/task.allium | 23 +- specs/template.allium | 102 ++++++-- specs/template_context.allium | 18 +- specs/translation.allium | 157 ++++++++++- specs/tui.allium | 211 +++++++++++++++ specs/ui_data_flow.allium | 6 +- 50 files changed, 3140 insertions(+), 449 deletions(-) create mode 100644 specs/cli.allium create mode 100644 specs/events.allium create mode 100644 specs/rendering.allium create mode 100644 specs/server.allium create mode 100644 specs/tui.allium diff --git a/AGENTS.md b/AGENTS.md index b1ce82c..d5fec88 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,9 +1,9 @@ # Agents Instructions for Blogging Desktop Server (bDS) -This is the Rust rewrite of an existing project bDS written in Typescript and living in ../bDS - if -in doubt about behaviour, look at the original code to verify. +This is the Rust rewrite of an existing project bDS. The baseline implementation is bDS2, written in Elixir and living in ../bDS2 - if +in doubt about behaviour, look at the baseline code to verify. -This project has an allium spec in the folder specs/ - use it to verify behaviour against expected behaviour. It is based on the typescript implementation. The command line utility is installed. +This project has an allium spec in the folder specs/ - use it to verify behaviour against expected behaviour. It is synced from the bDS2 specs (../bDS2/specs). The command line utility is installed. Invariants and behaviours in the allium spec should be covered by unit tests of the application code, to make sure the spec is followed. diff --git a/RUST_COMPATIBILITY_MATRIX_TEMPLATE.md b/RUST_COMPATIBILITY_MATRIX_TEMPLATE.md index cbc9697..2e69d6a 100644 --- a/RUST_COMPATIBILITY_MATRIX_TEMPLATE.md +++ b/RUST_COMPATIBILITY_MATRIX_TEMPLATE.md @@ -9,7 +9,7 @@ One row per persisted field or compatibility-sensitive artifact. ## How To Use 1. Start with posts, translations, media, templates, project metadata, menus, and generated outputs. -2. Record the current TypeScript behavior first. +2. Record the baseline (bDS2, `../bDS2/`) behavior first; the allium specs in `specs/` are the contract. 3. Record the intended Rust behavior only if it is identical or explicitly approved as a divergence. 4. Link every row to tests or golden fixtures once available. @@ -73,7 +73,7 @@ Use this for compatibility-sensitive behaviors that are not a single persisted f | rendering | built-in macros | JS server-side (gallery, youtube, vimeo, photo_archive, tag_cloud) | Rust-native in bds-core/render | implementation language changes, output must match | golden-generated-site comparisons | not-started | These are NOT Python macros | | rendering | Liquid feature subset | liquidjs 10.25 (full spec available) | Rust Liquid (scoped to used subset) | implementation may differ for unused features | template compatibility suite | not-started | Only ~35% of spec used by default templates | | slugs | slug generation | `transliteration` npm package | `deunicode` Rust crate | possible edge-case differences | slug corpus tests in M0 fixtures | not-started | Verify against real content | -| editor | content editing | Milkdown WYSIWYG (default) | plain-text syntax-highlighting editor (bds-editor: ropey + syntect + cosmic-text) + live preview | yes | n/a | approved-divergence | Rich editor deferred to extension Bucket I, builds on bds-editor foundation | +| editor | content editing | rich editor (default) | plain-text syntax-highlighting editor (bds-editor: ropey + syntect + cosmic-text) + live preview | yes | n/a | approved-divergence | Rich editor deferred to extension Bucket I, builds on bds-editor foundation | | preview | asset sourcing | local assets only | identical | no | HTML assertions | not-started | | | runtime | JS dependency | Electron (Chromium + Node.js) | no JavaScript anywhere — pure Rust + native APIs | yes (intentional) | build verification: no JS in dependency tree | approved-divergence | Supply-chain security constraint | | runtime | async executor | Node.js event loop | tokio | yes (internal) | n/a | approved-divergence | Used for preview server, publish, file watching | diff --git a/RUST_EXECUTION_BACKLOG.md b/RUST_EXECUTION_BACKLOG.md index c1b025d..7d68173 100644 --- a/RUST_EXECUTION_BACKLOG.md +++ b/RUST_EXECUTION_BACKLOG.md @@ -9,6 +9,15 @@ Rules: 1. Do not pull work forward from a later milestone unless it directly unblocks the current milestone. 2. Keep tasks vertically sliced where possible: engine + UI + tests. 3. Compatibility tasks outrank polish tasks. +4. The allium specs in `specs/` (synced from `../bDS2/specs/` on 2026-07-18) are the behaviour contract. When a task references a spec file, implement what the spec says and cover its invariants with unit tests. + +## Status (2026-07-18) + +- M0–M2: complete (items marked DONE below). +- M3: implemented (all views and the editor exist in `crates/bds-ui/src/views/`); items below are not individually ticked — verify against tests before re-doing anything. +- M4: in progress per git history ("M4 closing" commits). `bds-core/src/render/` exists (generation, macros, markdown, page renderer, routes, site, template lookup). `bds-core/src/ai/` does **not** exist yet — the one-shot AI client is unstarted. +- M5: not started (`bds-core/src/scripting/` is an empty stub; no PublishEngine). +- M6 (below): new — code-vs-spec gaps introduced by the bDS2 spec sync. Fold these in as the milestones they touch come up; none of them should wait for "later" if the code they touch is being written now. ## Milestone M0: Compatibility Baseline @@ -263,6 +272,60 @@ Rules: - built-in macro compatibility tests - docs sync tests +## Milestone M6: bDS2 Spec Parity + +Code-vs-spec gaps from the 2026-07-18 spec sync. Each task names its authoritative spec. Schema changes go through proper refinery migrations, and every new metadata field must be wired into publishing, metadata-diff, and rebuild-from-filesystem together. + +### `bds-core/util` + +- slug uniqueness: unbounded numeric suffix `{slug}-2, -3, …` — remove the 999 cap and timestamp fallback in `util/slug.rs` (post.allium) + +### `bds-core/db` + +- migration: add `cache_read_tokens` and `cache_write_tokens` (nullable integers) to `ai_usage` (schema.allium) +- migration: rename `npm` → `package_ref` and align `provider_npm` naming (schema.allium) +- media FTS search: accept structured `filters` alongside the query (search.allium `SearchMediaRequested(query, filters)`) + +### `bds-core/engine` + +- project open: discover `data_path` from the location of `meta/project.json`; never persist it in project.json; keep a machine-local project registry under the OS app-data dir (project.allium `DiscoverProjectDataPath`, `DataPathNotPersistedInProjectJson`) +- verify public/private data placement matches project.allium (`PublicContentLivesInProjectFolder`, `PrivateArtifactsLiveInOsAppDir`): DB, embeddings index, model cache, registry, and window state in the OS app-data dir; everything user-owned in the project folder +- menu: file-only model (no DB table), `HomeAlwaysFirst` normalization, `SyncMenuFromFilesystem` flow (menu.allium) +- media: optional import metadata (title, alt, caption, author, language, tags), `ReplaceMediaFile` flow, batch-import linked-image processing (media.allium) +- metadata diff: compare only translation-specific fields for translation files; per-item repair direction; `embedding` entity type (metadata_diff.allium) +- translations: `ReopenPublishedTranslation` on edit; `TranslationFilesCarryFullMetadata` (translation.allium) +- auto-translation pipeline: `ScheduleAutoTranslation`, `AutoTranslatePost`, `AutoTranslateMediaCascade`, `FillMissingTranslations` — gated by configured AI endpoint and airplane mode, skips `doNotTranslate`, fills only missing languages (translation.allium; depends on M4 AI client) +- task manager: allow cancelling `pending` (queued) tasks, not only running ones (task.allium) + +### `bds-core/render` (fold into M4) + +- implement rendering.allium as the shared render-assigns contract for preview and generation; content language only, never UI locale +- expose `slugify` as a custom Liquid filter alongside `i18n` and `markdown` (rendering.allium) +- generation task structure: task group with one task per section (Site Core, Single Posts, Category/Tag/Date Archives) + final Build Search Index task; stream one progress message per written page — "Generated /url (n/total)" or "Rewrote …" for validation apply (generation.allium) +- enforce `GenerationPublishedOnly` and the specified language variant selection (generation.allium) + +### `bds-core/ai` (fold into M4) + +- honour ai.allium invariants in the one-shot client and settings: endpoint add/remove rules, advisory model catalog with conditional refresh, provider detection, vision capability gate, airplane-mode model swap +- record cache token usage in `ai_usage` (schema.allium) + +### `bds-core/scripting` (fold into M5) + +- configurable `script_extension` (default "lua"): file path `scripts/{slug}.{extension}`; entrypoint default "render" for macros, "main" otherwise (script.allium, frontmatter.allium) +- sandboxed Lua runtime: no filesystem mutation, process control, or package loading; host capabilities only via explicit `bds.*` API (script.allium `SandboxedExecution`, `ExplicitHostCapabilities`) +- enforce config limits: macro timeout 10s; transform toasts max 5/script, 20 total, 300 chars; blogmark title ≤ 200, URL ≤ 2048 (script.allium) + +### `bds-ui` + +- settings: technology section (semantic similarity toggle), data section (rebuild buttons: posts/media/scripts/templates/links/thumbnails/embedding + Open Data Folder), agent integrations (Claude Code + GitHub Copilot; others disabled with "not supported yet" note), image import concurrency (1–8, default 4), default editor mode, settings in a separate tab (editor_settings.allium) +- media search filter UI wired to the filtered media search +- auto-translation and repair-direction UI hooks for the engine flows above + +### Validation + +- unit tests for every invariant and rule named above, per AGENTS.md spec-coverage requirement +- re-run `allium check specs/*.allium` after any spec edit + ## Extension Backlog ### Bucket A: Git And Validation @@ -345,9 +408,29 @@ Rules: #### `bds-cli` -- headless commands -- MCP server surface -- CLI-to-app notification mechanism (`db_notifications` / `NotificationWatcher`) +- CLI command set per cli.allium: `rebuild | repair | render | upload | push | pull | post | media | gallery | config | project | lua` — same database, settings, and cache as the GUI; no listeners, no window; logging redirected to the rotating log file +- MCP server surface per mcp.allium (incl. translation tools, stats/media resources, proposal status) +- CLI-to-app synchronization via the domain event bus (events.allium): entity mutations broadcast `(entity, entity_id, action)` on one topic shape covering in-app and external writes + +#### `bds-core` + +- domain event bus (events.allium): broadcast after every successful mutation of posts, media, tags, templates, scripts; `SettingsChangedEvent` for global settings + +### Bucket K: Headless Server Mode (server.allium) + +#### `bds-core` / `bds-cli` + +- boot modes desktop | server | tui resolved from an env var at start +- headless server: no window; SSH transport for remote TUI/GUI clients +- SSH key material generated on first boot in the private app-data dir (host key + empty authorized_keys, mode 600) — never in the repo or project folder + +### Bucket L: Terminal UI (tui.allium) + +#### new crate `bds-tui` (or feature in `bds-cli`) + +- TUI as a second renderer over the same shared UI core: sidebar views (posts/media/templates/scripts/tags/settings/git), sidebar/editor focus model, vim-style + arrow navigation skipping section headers +- post editor drives the same workflow as the GUI: canonical-language edits update the post, other languages update translations; publish routes through the same pipeline +- subscribes to the domain event bus (Bucket G) for live updates ### Bucket H: Blogmark And Transforms @@ -384,4 +467,4 @@ Rules: ## Exit Rule -Do not start extension buckets until Milestone M5 is complete and the Rust app is already a credible replacement for the current authoring and publishing workflow. \ No newline at end of file +Do not start extension buckets until Milestones M5 and M6 are complete and the Rust app is already a credible replacement for the baseline authoring and publishing workflow. \ No newline at end of file diff --git a/RUST_PLAN.md b/RUST_PLAN.md index b9bc30c..04a0853 100644 --- a/RUST_PLAN.md +++ b/RUST_PLAN.md @@ -1,10 +1,19 @@ # bDS Rust Rewrite Plan -bDS is a blogging desktop system that is currently in the following folder: +bDS is a blogging desktop system. The behavioural baseline is **bDS2**, an Elixir (Phoenix LiveView) implementation living in: -../bDS/ +../bDS2/ -this is to be rewritten with Rust and this is the plan. For any implementation detail, look into the ../bDS/ folder into the typescript code there. Assume the typescript code as "the truth" about the current implementation. +This project (RuDS) rewrites it in Rust. For any implementation detail, look into ../bDS2/ — the Elixir code there is "the truth" about the current implementation. The old TypeScript app at ../bDS is historical and no longer the reference. + +## Spec Baseline + +The authoritative behaviour contract is the allium spec set in `specs/`, synced wholesale from `../bDS2/specs/` on 2026-07-18 (46 files, all pass `allium check`). Rules: + +- When spec and Rust code disagree, the spec wins; when the spec is ambiguous, the bDS2 Elixir code wins. +- Five specs are new with the bDS2 sync and have **no Rust implementation yet**: `rendering.allium` (core scope — render assigns/filters/macros shared by preview and generation), `cli.allium`, `events.allium`, `server.allium`, `tui.allium` (all extension scope). +- The concrete code-vs-spec gaps are tracked in RUST_EXECUTION_BACKLOG.md under "Milestone M6: bDS2 Spec Parity". +- When re-syncing specs from bDS2, re-run `allium check specs/*.allium` and refresh the M6 gap list. This plan is split into multiple documents: @@ -69,9 +78,8 @@ bds-rust/ ## Distribution Characteristics - **Single static binary.** No external runtime dependencies (no GTK, no Electron, no Node.js). Lua and SQLite are compiled into the binary. -- **Binary size:** ~15–25 MB (versus 150–200 MB for the current Electron app). -- **Memory usage:** ~50–80% less RAM than the Electron app (no Chromium process). -- **Startup:** Significantly faster (no V8/Chromium initialization). +- **Binary size:** ~15–25 MB. +- **Memory usage / startup:** no BEAM VM, no browser engine — a single native process. - **Zero install dependencies:** users download one binary. No Homebrew, no system packages. ## Split Rationale @@ -80,7 +88,7 @@ The old single-file plan mixed three categories of work: - work required to ship a real replacement for the current app - work required for eventual feature parity -- work that is valuable but not on the critical path to replacing TypeScript +- work that is valuable but not on the critical path to replacing the baseline app The new split keeps the shipping path narrow. Core is only the work needed to replace the current app for everyday authoring and publishing. Extensions carry the rest. @@ -88,7 +96,7 @@ The new split keeps the shipping path narrow. Core is only the work needed to re The Rust rewrite is not considered successful until the core release can do all of the following with existing bDS project data: -1. Open a real project created by the TypeScript app. +1. Open a real project created by the baseline app. 2. Create and edit posts, translations, media, tags, templates, and settings. 3. Preview drafts and published content locally. 4. Generate a complete site whose output matches the current app modulo approved normalization differences. @@ -101,31 +109,10 @@ The Rust rewrite is not considered successful until the core release can do all Both plan documents assume the same verification baseline: 1. Fixture projects exported from the current app are the compatibility corpus. -2. Golden-file tests compare Rust-written files against TypeScript-written files. +2. Golden-file tests compare Rust-written files against baseline-written files. 3. Golden-output tests compare generated sites between the current app and the Rust app. 4. No feature is complete until both UI behavior and underlying engine behavior are covered. -## Recommended Repository Strategy +## Repository Strategy -Build the Rust rewrite as a separate parallel project, not inside the current TypeScript application tree. - -Why: - -1. The rewrite has a different language stack, build chain, packaging model, runtime model, and test harness. -2. The compatibility target is the current app's behavior and data, not shared source code. -3. Keeping the rewrite isolated avoids contaminating this repo with long-lived dual-toolchain complexity. -4. The current app remains the stable reference implementation while the Rust app catches up. - -Recommended structure: - -- keep this repository as the reference implementation and fixture source -- create a sibling repository such as `bds-rust` -- pull compatibility fixtures from this repo into the Rust repo on a controlled cadence -- compare generated output across repos in the Rust repo's CI - -Only use the same repository if you explicitly want a temporary umbrella monorepo and are willing to accept: - -- slower CI -- mixed Node and Rust release pipelines -- more complex contributor onboarding -- higher risk of plan drift between the legacy app and the rewrite +This repository (RuDS) is the Rust rewrite, kept separate from the baseline. `../bDS2` remains the stable reference implementation and fixture source; compatibility fixtures are pulled from it on a controlled cadence and generated output is compared here. diff --git a/RUST_PLAN_CORE.md b/RUST_PLAN_CORE.md index f492f55..7908340 100644 --- a/RUST_PLAN_CORE.md +++ b/RUST_PLAN_CORE.md @@ -2,7 +2,7 @@ ## Goal -Ship a native desktop Rust application (macOS primary, cross-platform ready) that fully replaces the current TypeScript app for the primary blogging workflow: +Ship a native desktop Rust application (macOS primary, cross-platform ready) that fully replaces the baseline app (bDS2, Elixir, at `../bDS2/`) for the primary blogging workflow: 1. open an existing project 2. create and edit content @@ -52,6 +52,43 @@ Core is not a toy MVP. Core must already be a production-capable replacement for User content and project data are compatible. User-authored Python scripts are not compatible and are treated as legacy artifacts. +## bDS2 Spec Sync — Contract Updates (2026-07-18) + +The specs in `specs/` were synced wholesale from `../bDS2/specs/`. The following contracts changed or were made explicit relative to what this plan and the existing Rust code assumed. Each item names the spec file that is authoritative; the executable gap list lives in RUST_EXECUTION_BACKLOG.md M6. + +### Data placement (project.allium) + +- `public_dir` = the project folder (the directory containing `meta/project.json`): all user-owned, portable, webserver-bound content — posts, media + thumbnails, templates/, scripts/, meta/, tags.json, menu.opml, and generated `html/` output. +- `private_dir` = the OS per-user app-data dir (`~/Library/Application Support/bds`, `$XDG_CONFIG_HOME/bds`, `%APPDATA%\bds`): SQLite database, per-project embeddings index, model cache, project registry, window/UI state. Never inside the repo or the project folder. +- `data_path` is **discovered** from the on-disk location of `meta/project.json` and never persisted inside project.json — projects must stay movable (invariant `DataPathNotPersistedInProjectJson`). +- Creating a project does not seed `public_dir/templates` with bundled defaults (`ProjectTemplatesDirectoryReservedForUserTemplates`). + +### Behaviour deltas vs. old plan assumptions + +- **Slug uniqueness** (post.allium): unbounded numeric suffix `{slug}-2, -3, …`. The old `-999` cap and timestamp fallback are gone. +- **Menu** (menu.allium): file-only model — `meta/menu.opml` is the sole store, no DB table. Normalization guarantees home is always the first item (`HomeAlwaysFirst`); `UpdateMenu` strips incoming home entries and prepends one. A `SyncMenuFromFilesystem` flow reloads it from disk. +- **Scripts** (script.allium, schema.allium): file path is `scripts/{slug}.{extension}` with configurable `script_extension` (default `"lua"`); entrypoint default is `"render"` for macros, `"main"` otherwise. Execution guarantees: sandboxed runtime (no filesystem mutation, process control, or package loading) with host capabilities exposed only through an explicit `bds.*` API. Config limits: macro timeout 10s, transform toast limits (5/script, 20 total, 300 chars), blogmark title ≤200 / URL ≤2048. +- **Schema** (schema.allium): `ai_usage` gains `cache_read_tokens` / `cache_write_tokens` (nullable); the legacy `npm` / `provider_npm` columns are the generic `package_ref` / provider package reference. Statuses are named enums (PostStatus, PostTranslationStatus, TemplateStatus, ScriptStatus). +- **Media** (media.allium): import accepts optional caller-supplied metadata (title, alt, caption, author, language, tags); a `ReplaceMediaFile` flow swaps the underlying file for an existing media record; batch import processes linked images. Media search takes `filters`, not just a query (search.allium). +- **Search** (search.allium): FTS5 tables use per-field stemmed columns (title/excerpt/content/tags/categories); translations are stemmed with their own language stemmer and appended per-field. Languages without a Snowball algorithm pass through unstemmed. (Already implemented in `bds-core/db/fts.rs`.) +- **Generation** (generation.allium): only published content generates (`GenerationPublishedOnly`); language variant selection is specified; generation/validation run as a task group — one task per section (Site Core, Single Posts, Category/Tag/Date Archives) plus a final Build Search Index task — streaming one progress message per written page. +- **Rendering** (rendering.allium — NEW, core scope): the render-assigns contract shared by preview and generation. Rendering language is the content language, never the UI locale. Custom Liquid filters are `i18n`, `markdown`, and `slugify` (slugify was not in the old plan's filter list). +- **Metadata diff** (metadata_diff.allium): translation files carry only translation-specific metadata — shared status/timestamps come from the canonical post and must not be reported missing; repair supports a direction per item; `embedding` is a diffable entity type. +- **Translations** (translation.allium): translation files carry full metadata (`TranslationFilesCarryFullMetadata`); editing a published translation reopens it to draft (`ReopenPublishedTranslation`); an auto-translation pipeline (`ScheduleAutoTranslation`, `AutoTranslatePost`, `AutoTranslateMediaCascade`, `FillMissingTranslations`) fills missing languages, skips `doNotTranslate`, and is gated by the configured AI endpoint and airplane mode. +- **AI** (ai.allium): endpoint add/remove rules, advisory model catalog with conditional refresh, provider detection, vision capability gating, chat context truncation, airplane-mode model swap. One-shot operations remain core; chat remains extension. +- **Settings UI** (editor_settings.allium): settings gains a technology section (semantic similarity toggle; scripting runtime is fixed, no switch exposed), a data section (rebuild buttons for posts/media/scripts/templates/links/thumbnails/embedding + Open Data Folder), agent integrations (Claude Code and GitHub Copilot supported; others render disabled with a "not supported yet" note), image import concurrency (1–8, default 4), and default editor mode (markdown | preview). Settings opens in a separate tab. +- **Tasks** (task.allium): `pending → cancelled` is a legal transition (cancel queued tasks, not only running ones). +- **Frontmatter** (frontmatter.allium): timestamps are Unix **milliseconds**; keys are camelCase; `doNotTranslate` only written when true, `templateSlug`/`publishedAt` only when present. (Already implemented in `bds-core/util/frontmatter.rs`.) + +### New subsystems specified by bDS2 (extension scope) + +- `cli.allium` — workspace CLI (`rebuild | repair | render | upload | push | pull | post | media | gallery | config | project | tui | lua`) sharing the same database and settings as the GUI, no listeners/window. +- `events.allium` — domain event bus: every entity mutation broadcasts `(entity, entity_id, action)` on the same topic/payload shape as the CLI sync watcher, so one subscription covers in-app and external changes. +- `server.allium` — headless server mode with SSH transport (boot modes desktop | server | tui via env var; SSH key material in the private app-data dir). +- `tui.allium` — terminal UI as a second renderer over the same shared UI core and post-editor workflow as the GUI. + +These map to extension Buckets G (CLI + events) and K/L (server, TUI) in RUST_PLAN_EXTENSION.md. + ## Architecture Decisions - **Single process Rust app**: no Electron, no IPC boundary. @@ -62,7 +99,7 @@ User-authored Python scripts are not compatible and are treated as legacy artifa - **Custom editor widget (bds-editor crate)**: syntax-highlighting markdown/Liquid/Lua editor built on ropey (rope buffer), syntect (syntax highlighting), and cosmic-text (font shaping and text layout). This is the highest-risk custom component and gets a proof-of-concept in Wave 0. - **rusqlite + embedded SQL migrations**: no ORM. SQL stays explicit. Migrations managed via `refinery`. - **tokio as the async runtime**: preview server (axum), SSH publishing, file watching, and rfd async dialogs all require an async executor. tokio is the standard choice and is used workspace-wide. Synchronous engine code in bds-core does not use tokio directly — it remains callable from both async (bds-ui) and sync (bds-cli) contexts. -- **Plain-text markdown editor first**: no WYSIWYG in core. Live preview is required. This is an intentional regression from the current app's Milkdown WYSIWYG editor. A rich editor can be added as an extension bucket after core ships — and the bds-editor widget provides a natural foundation for it. +- **Plain-text markdown editor first**: no WYSIWYG in core. Live preview is required. This is an intentional regression from the baseline app's rich editor. A rich editor can be added as an extension bucket after core ships — and the bds-editor widget provides a natural foundation for it. - **Lua for user-authored scripting**: `mlua` with Lua 5.4. Only user-authored macros, transforms, and utility scripts run through Lua. Built-in macros are native Rust — see the macro architecture section below. - **One-shot AI operations are core scope**: six operations use a simple OpenAI-compatible HTTP client (`reqwest` + `serde_json`): (1) translate post, (2) translate media metadata, (3) image description / alt text, (4) post analysis (title + excerpt + slug suggestion), (5) taxonomy analysis (tag + category suggestions), (6) language detection. Two configurable endpoints: one for online use, one for airplane/offline mode (local model). These are fire-and-forget request/response calls — no streaming, no tool use, no chat history. The AI chat UI, streaming responses, and tool execution remain in Extension Bucket C. @@ -256,8 +293,8 @@ This means the Rust Liquid implementation only needs roughly 35% of the full spe The current app has two distinct macro systems that must not be conflated: -1. **Built-in macros** (`gallery`, `youtube`, `vimeo`, `photo_archive`, `tag_cloud`) — implemented in JavaScript, rendered server-side during generation. Each has a corresponding `.liquid` template in `src/main/engine/templates/macros/`. These are **not** Python scripts and never were. -2. **User-authored script macros** — implemented in Python (Pyodide), invoked via the `PythonMacroWorkerRuntime` worker thread. These are replaced by Lua in the Rust app. +1. **Built-in macros** (`gallery`, `youtube`, `vimeo`, `photo_archive`, `tag_cloud`) — implemented in the host language of the baseline app, rendered server-side during generation, each with a corresponding `.liquid` macro template. These are **not** user scripts and never were. +2. **User-authored script macros** — Lua in bDS2 and in the Rust app (the old TypeScript app used Python/Pyodide; those scripts are legacy artifacts). See script.allium for the sandbox and `bds.*` host API contract. In the Rust app: @@ -268,7 +305,7 @@ Macro invocation syntax in content: `[[macro_name param1="value1" param2="value2 ### Slug generation compatibility -The current app uses the `transliteration` npm package for Unicode-to-ASCII conversion in slugs. The plan specifies `deunicode` for Rust. These libraries may produce different output for edge cases (CJK characters, Cyrillic, accented Latin, etc.). Add a slug compatibility test suite to M0 fixtures that runs both libraries against the project's actual content corpus and documents any divergences. +The Rust app uses `deunicode` for Unicode-to-ASCII conversion in slugs; it must match the established bDS transliteration behaviour (especially German umlauts). The slug compatibility test suite in M0 fixtures covers this. Uniqueness per post.allium: try the base slug, then `{slug}-2`, `{slug}-3`, … with an **unbounded** numeric suffix (no 999 cap, no timestamp fallback). ### Two search systems @@ -280,13 +317,13 @@ The app has two independent search systems — do not conflate them: ### Client-side search index (Pagefind) -If the current TypeScript app generates a Pagefind search index as part of site generation, the Rust app must produce the same artifact. Pagefind publishes a Rust library (`pagefind` crate) with a programmatic API (`pagefind::api::PagefindIndex`). The generation pipeline feeds rendered HTML to `PagefindIndex::add_html_file()` and writes the resulting index files via `PagefindIndex::get_files()`. No external binary, no npm — this is a pure Rust library dependency, fully compatible with the no-JavaScript constraint. +The baseline app generates a Pagefind search index as part of site generation, and the Rust app must produce the same artifact. Pagefind publishes a Rust library (`pagefind` crate) with a programmatic API (`pagefind::api::PagefindIndex`). The generation pipeline feeds rendered HTML to `PagefindIndex::add_html_file()` and writes the resulting index files via `PagefindIndex::get_files()`. No external binary, no npm — this is a pure Rust library dependency, fully compatible with the no-JavaScript constraint. Determine during the compatibility inventory (M0) whether Pagefind or another client-side search solution is used. If Pagefind: add it to the generation pipeline in Wave 4 via the `pagefind` crate. If a different solution: document it and plan accordingly. ### Image processing -Media import requires thumbnail generation and potentially format conversion (e.g., to WEBP). The current TypeScript app uses `sharp` (libvips bindings). The Rust equivalent is the `image` crate for decoding/encoding and basic transforms (resize, crop). If WEBP encoding performance or advanced processing (EXIF handling, ICC profiles) proves insufficient with `image` alone, `libvips` Rust bindings (`libvips-rs`) are the fallback. Start with `image` — it covers the common cases and has no system dependencies. +Media import requires thumbnail generation and potentially format conversion (e.g., to WEBP). The Rust choice is the `image` crate for decoding/encoding and basic transforms (resize, crop). If WEBP encoding performance or advanced processing (EXIF handling, ICC profiles) proves insufficient with `image` alone, `libvips` Rust bindings (`libvips-rs`) are the fallback. Start with `image` — it covers the common cases and has no system dependencies. ## Planned Crate Registry @@ -565,7 +602,7 @@ Not all of these are needed in Wave 0 — this is the full foundation set. Wave- - round-trip tests for posts, translations, media, templates, and project metadata - rebuild tests using fixture projects - metadata diff tests proving no false negatives for known field changes -- file-by-file golden comparisons against TypeScript output +- file-by-file golden comparisons against baseline output ### Done when diff --git a/RUST_PLAN_EXTENSION.md b/RUST_PLAN_EXTENSION.md index e474e2a..d6aabc2 100644 --- a/RUST_PLAN_EXTENSION.md +++ b/RUST_PLAN_EXTENSION.md @@ -134,10 +134,9 @@ Core must read menu documents for rendering compatibility, but editing them can #### Scope -- headless CLI maturation -- MCP server -- remote automation contracts -- `NotificationWatcher` / `db_notifications` mechanism for CLI-to-app synchronization +- workspace CLI per `cli.allium`: `rebuild | repair | render | upload | push | pull | post | media | gallery | config | project | lua` — boots the same repo, settings, and database as the desktop app, with no listeners and no window; logging to the rotating log file +- MCP server per `mcp.allium` (translation tools, stats and media resources, proposal workflow) +- domain event bus per `events.allium`: every successful entity mutation (posts, media, tags, templates, scripts) broadcasts `(entity, entity_id, action)`; global settings changes broadcast a settings event. One topic/payload shape covers in-app mutations and external CLI writes, replacing the old `db_notifications` polling design. The bus also feeds the TUI (Bucket L) and any future multi-client sync. #### Why extension @@ -146,7 +145,7 @@ Useful ecosystem surface, not required to replace the desktop app itself. #### Done when - automation consumers can drive the Rust app safely and consistently -- CLI changes are detected and surfaced to the running desktop app +- CLI changes are detected and surfaced to the running desktop app via the event bus ### Bucket H: Blogmark And Transform Pipeline @@ -170,13 +169,13 @@ Secondary content-capture workflow. Not required for core authoring with existin #### Scope -- WYSIWYG or hybrid markdown editing (similar to current Milkdown editor) +- WYSIWYG or hybrid markdown editing (comparable to the baseline app's rich editor) - macro syntax preview in editor - image insert dialog from linked media #### Why extension -Core ships with the bds-editor syntax-highlighting plain-text editor and live preview. The current app defaults to a Milkdown WYSIWYG editor, so this is a user-facing regression that should be addressed after core stabilizes. +Core ships with the bds-editor syntax-highlighting plain-text editor and live preview. The baseline app defaults to a rich editor, so this is a user-facing regression that should be addressed after core stabilizes. #### Architecture advantage @@ -191,7 +190,44 @@ This is an evolution of the existing editor widget, not a separate technology de #### Done when -- users can edit content with a rich editor comparable to the current app's Milkdown experience +- users can edit content with a rich editor comparable to the baseline app's editing experience + +### Bucket K: Headless Server Mode + +#### Scope + +Per `server.allium`: + +- boot modes `desktop | server | tui` resolved from an environment variable at application start +- headless server: full engine, no window, SSH transport for remote TUI/GUI clients +- SSH key material (host key + authorized_keys, mode 600) generated on first boot into the private OS app-data dir — never the repo or the project folder +- GUI "connect to server" flow + +#### Why extension + +Client/server split is new bDS2 capability, not needed to replace the desktop app for local use. + +#### Done when + +- the app can run headless and accept an SSH-transported client session against real project data + +### Bucket L: Terminal UI + +#### Scope + +Per `tui.allium` — a second renderer over the same shared UI core: + +- sidebar views (posts, media, templates, scripts, tags, settings, git) with sidebar/editor focus model and vim-style + arrow navigation that skips section headers +- post editor driving the same workflow as the GUI: canonical-language edits update the post, other languages update translations, publish routes through the same pipeline +- live updates via the domain event bus (Bucket G) + +#### Why extension + +Second renderer; requires the shared UI core and (for remote use) Bucket K. + +#### Done when + +- everyday authoring (browse, edit, publish) works in a terminal against the same project data as the GUI ### Bucket J: A2UI Server-Driven Surfaces @@ -230,6 +266,10 @@ Bucket H Blogmark + Transforms ↓ Bucket I Rich Editor (builds on bds-editor from core) ↓ +Bucket K Headless Server Mode + ↓ +Bucket L Terminal UI (after Buckets G + K) + ↓ Bucket J A2UI Surfaces (after Bucket C) ``` diff --git a/SPECIFICATION_INDEX.md b/SPECIFICATION_INDEX.md index bb12a21..82c7b48 100644 --- a/SPECIFICATION_INDEX.md +++ b/SPECIFICATION_INDEX.md @@ -45,6 +45,16 @@ Diese Datei dient als Index zu allen Allium-Spezifikationen und Schema-Inventari | `cli_sync.allium` | Core (Wave 5) | ✅ Existiert | CLI-zu-App Notification Sync | | `metadata_diff.allium` | Core (Wave 1) | ✅ Existiert | DB/Dateisystem-Diff und Rebuild | +### Von bDS2 übernommene Spezifikationen (Sync 2026-07) + +| Datei | Scope | Status | Beschreibung | +|-------|-------|--------|--------------| +| `rendering.allium` | Core | ✨ Neu | Render-Subsystem (Assigns, Filter, Makros für Preview + Generation) | +| `cli.allium` | Extension G | ✨ Neu | Workspace-CLI-Tool (rebuild, repair, render, upload, …) | +| `events.allium` | Extension | ✨ Neu | Domain Event Bus (Multi-Client-Synchronisation) | +| `server.allium` | Extension | ✨ Neu | Headless Server Mode (SSH-Transport für TUI/GUI) | +| `tui.allium` | Extension | ✨ Neu | Terminal UI (zweiter Renderer über gemeinsamem UI-Core) | + ## Neuerstellte Spezifikationen (diese Session) ### 1. `schema.allium` diff --git a/docs/RESEARCH_FINDINGS.md b/docs/RESEARCH_FINDINGS.md index 538f685..dc8c40e 100644 --- a/docs/RESEARCH_FINDINGS.md +++ b/docs/RESEARCH_FINDINGS.md @@ -1,6 +1,6 @@ # bDS Rust Rewrite — Research Findings -Research completed during M0 from the TypeScript codebase at `../bDS/`. +Research completed during M0 from the TypeScript codebase at `../bDS2/`. ## Database Schema (16 migrations, 0000–0015) diff --git a/specs/action_patterns.allium b/specs/action_patterns.allium index d216e1e..ee4adc4 100644 --- a/specs/action_patterns.allium +++ b/specs/action_patterns.allium @@ -106,7 +106,7 @@ rule MediaMetadataTranslationCascade { -- Four distinct patterns used across the application: --- Pattern 1: Native confirm dialog (rfd crate) +-- Pattern 1: System confirm dialog -- Simple yes/no system dialog, no custom UI -- Used by: PostDelete, PostDiscard, TemplateDelete (when references exist) diff --git a/specs/ai.allium b/specs/ai.allium index 2a8a501..6923319 100644 --- a/specs/ai.allium +++ b/specs/ai.allium @@ -3,9 +3,8 @@ -- Scope: core (one-shot operations), extension Bucket C (chat + streaming) -- Distilled from: src/main/engine/ChatEngine.ts, ai/providers.ts, -- ai/chat.ts, ai/tasks.ts, SecureKeyStore.ts --- Note: Rust rewrite simplifies providers to two configurable --- OpenAI-compatible endpoints (online + airplane mode), --- replacing the TypeScript app's 4 named providers. +-- The rewrite models AI access as two configurable OpenAI-compatible +-- endpoints (online + airplane mode) instead of a fixed named-provider set. use "./post.allium" as post use "./media.allium" as media @@ -19,6 +18,61 @@ entity AiEndpoint { -- airplane: local model (Ollama, LM Studio, etc.) } +entity AiCatalogProvider { + id: String + name: String + env_keys: Set + package_ref: String? + api_url: String? + doc_url: String? + updated_at: Timestamp +} + +entity AiModel { + provider: AiCatalogProvider + model_id: String + name: String + family: String? + supports_attachment: Boolean + supports_reasoning: Boolean + supports_tool_calls: Boolean + supports_structured_output: Boolean + supports_temperature: Boolean + knowledge: String? + release_date: String? + last_updated_date: String? + open_weights: Boolean + input_price: Integer? + output_price: Integer? + cache_read_price: Integer? + cache_write_price: Integer? + context_window: Integer + max_input_tokens: Integer + max_output_tokens: Integer + interleaved: String? + status: String? + updated_at: Timestamp + + -- Relationships + modalities: AiModelModality with provider = this.provider and model_id = this.model_id + + -- Derived + input_modalities: modalities where direction = input -> modality + output_modalities: modalities where direction = output -> modality +} + +entity AiModelModality { + provider: AiCatalogProvider + model_id: String + direction: input | output + modality: text | image | audio | file | tool +} + +entity AiCatalogMeta { + key: String + value: String +} + surface AiEndpointSurface { context endpoint: AiEndpoint @@ -29,10 +83,40 @@ surface AiEndpointSurface { endpoint.model } + surface AiModelSurface { + context catalog_model: AiModel + + exposes: + catalog_model.provider + catalog_model.model_id + catalog_model.name + catalog_model.family when catalog_model.family != null + catalog_model.supports_attachment + catalog_model.supports_reasoning + catalog_model.supports_tool_calls + catalog_model.supports_structured_output + catalog_model.supports_temperature + catalog_model.knowledge when catalog_model.knowledge != null + catalog_model.release_date when catalog_model.release_date != null + catalog_model.last_updated_date when catalog_model.last_updated_date != null + catalog_model.open_weights + catalog_model.input_price when catalog_model.input_price != null + catalog_model.output_price when catalog_model.output_price != null + catalog_model.cache_read_price when catalog_model.cache_read_price != null + catalog_model.cache_write_price when catalog_model.cache_write_price != null + catalog_model.context_window + catalog_model.max_input_tokens + catalog_model.max_output_tokens + catalog_model.interleaved when catalog_model.interleaved != null + catalog_model.status when catalog_model.status != null + catalog_model.input_modalities + catalog_model.output_modalities + catalog_model.updated_at + } + entity SecureKeyStore { - -- Encrypts API keys using OS keychain - -- macOS: Keychain, Windows: DPAPI, Linux: libsecret - -- Stored as base64 in settings table with __encrypted_ prefix + -- Encrypts API keys using the host operating system's secure storage. + -- Stored in application settings in encrypted form. -- No plain-text fallback } @@ -64,8 +148,12 @@ entity ChatMessage { conversation: ChatConversation role: system | user | assistant | tool content: String + tool_call_id: String? + tool_calls: String? token_usage_input: Integer? token_usage_output: Integer? + cache_read_tokens: Integer? + cache_write_tokens: Integer? created_at: Timestamp } @@ -76,11 +164,24 @@ surface ChatMessageSurface { message.conversation message.role message.content + message.tool_call_id when message.tool_call_id != null + message.tool_calls when message.tool_calls != null message.token_usage_input when message.token_usage_input != null message.token_usage_output when message.token_usage_output != null + message.cache_read_tokens when message.cache_read_tokens != null + message.cache_write_tokens when message.cache_write_tokens != null message.created_at } +surface AiConfigurationSurface { + facing _: AiOperator + + provides: + SetAiEndpointRequested(kind, url, api_key, model) + RemoveAiEndpointRequested(kind) + RefreshModelCatalogRequested(source) +} + surface OneShotAiSurface { facing _: AiOperator @@ -99,7 +200,31 @@ surface AiChatSurface { provides: StartChatRequested(model) SendChatMessageRequested(conversation, content) - RefreshModelCatalogRequested(endpoint) + CancelChatRequested(conversation) +} + +config { + model_catalog_ttl: Duration = 5.minutes + chat_max_tool_rounds: Integer = 10 + default_max_output_tokens: Integer = 16384 +} + +rule SetAiEndpoint { + when: SetAiEndpointRequested(kind, url, api_key, model) + ensures: + let endpoint = AiEndpoint.created( + kind: kind, + url: url, + api_key: api_key, + model: model + ) + endpoint.kind = kind +} + +rule RemoveAiEndpoint { + when: RemoveAiEndpointRequested(kind) + for endpoint in AiEndpoints where endpoint.kind = kind: + ensures: not exists endpoint } -- One-shot AI tasks (core scope, no streaming) @@ -174,18 +299,25 @@ rule SendChatMessage { ) ensures: conversation.updated_at = now ensures: AiStreamingResponse(conversation) - -- AI SDK v6 streamText() with tool-call loop (max 10 rounds) - -- Blog data tools for post/media querying during chat - -- Token usage tracking (input, output, cache read/write) + -- Streaming response with bounded tool-call loop. + -- Blog data tools for post/media querying and mutation during chat. + -- Render tools may emit structured chart/table/form payloads. + -- Token usage tracking includes input, output, cache read, cache write. +} + +rule CancelChat { + when: CancelChatRequested(conversation) + ensures: AiStreamingResponseCancelled(conversation) } -- Model catalog rule RefreshModelCatalog { - when: RefreshModelCatalogRequested(endpoint) - -- Queries the endpoint's model list API - -- 5-minute cache TTL - ensures: ModelCatalogUpdated(endpoint) + when: RefreshModelCatalogRequested(source) + -- Refreshes advisory provider/model metadata used for capability checks, + -- default token budgeting, and model selection UX. + -- Uses conditional GET with ETag where supported. + ensures: ModelCatalogUpdated() } invariant AirplaneModeGating { @@ -198,12 +330,86 @@ invariant AirplaneModeGating { -- show toast "AI unavailable — configure {online|airplane} endpoint in Settings" } +invariant AirplaneModeModelSwap { + -- In airplane mode, cloud models are never contacted. + -- Chat uses the configured offline chat model when needed. + -- Image analysis uses the configured offline vision-capable model when needed. + -- If no suitable offline model is configured, the operation fails with + -- actionable guidance instead of silently falling back to the online endpoint. +} + invariant TwoEndpointModel { -- Two configurable OpenAI-compatible endpoints: -- online: for cloud providers (requires API key) -- airplane: for local models (no API key required) -- Both use the OpenAI Chat Completions wire format. - -- This replaces the TypeScript app's 4 named provider model. + -- Endpoint selection is configurable rather than tied to hard-coded providers. +} + +invariant AdvisoryModelCatalog { + -- Model metadata is stored separately from runtime endpoint configuration. + -- It supplies capability hints such as context window, tool-call support, + -- structured output support, vision/input modalities, and pricing metadata. + -- The catalog remains usable offline after the last successful refresh. +} + +invariant ConditionalCatalogRefresh { + -- Model catalog refresh uses conditional HTTP requests when possible. + -- The latest ETag and fetch timestamp are persisted in AiCatalogMeta. + -- A 304 response updates freshness metadata without rewriting model rows. + exists meta in AiCatalogMeta where meta.key = "etag" or meta.key = "last_fetched_at" +} + +invariant ProviderDetection { + -- Runtime provider selection may be inferred from model identifiers, + -- local-endpoint registration, or explicit endpoint configuration. + -- The system does not rely on a single hard-coded provider list for routing. +} + +invariant VisionCapabilityGate { + -- AnalyzeImage only runs against models that accept image input. + -- Local/offline models must advertise or be configured with image capability + -- before the runtime sends multimodal requests to them. +} + +invariant ChatContextTruncation { + -- Chat requests are trimmed to fit within the selected model's context window. + -- Oldest user/assistant pairs are dropped first. + -- The system prompt, tool schema budget, and output-token reserve are preserved. +} + +invariant BoundedToolLoop { + -- Chat tool execution is bounded by config.chat_max_tool_rounds. + -- Tool-capable models may call blog-domain tools and render tools. + -- Non-tool-capable models skip tool exposure entirely. +} + +invariant TokenUsageAccounting { + -- Chat turn accounting tracks input, output, cache-read, and cache-write tokens. + -- Usage is reported per turn and accumulated per conversation. + -- Cache token accounting is surfaced when the underlying provider reports it. +} + +invariant ChatCancellation { + -- Each in-flight chat turn can be aborted independently. + -- Cancellation stops streaming and tool execution for that request only. +} + +invariant StructuredRenderTools { + -- Chat may emit structured render payloads via tool calls prefixed "render_". + -- 9 inline surface types: card, chart, form, list, metric, mindmap, table, tabs, text, json. + -- Render tool names: render_card, render_chart, render_form, render_list, + -- render_metric, render_mindmap, render_table, render_tabs. + -- Unrecognized render tool names render as JSON (raw dump). + -- Tab content may nest any surface type including plain text. + -- These payloads are data contracts, not arbitrary HTML strings. + -- Surface data contracts are defined in editor_chat.allium InlineSurface value. +} + +invariant BlogStatsPromptAugmentation { + -- The base system prompt may be augmented with current blog statistics + -- such as post counts, media counts, tag/category totals, and date ranges + -- so long as the augmentation reflects current project state. } invariant AiSpecPartitioning { @@ -215,5 +421,5 @@ invariant AiSpecPartitioning { invariant SecureKeyStorage { -- API keys are never stored in plain text - -- Always encrypted via OS keychain before DB storage + -- Always encrypted via host secure storage before persistence } diff --git a/specs/bds.allium b/specs/bds.allium index 4a5f70c..c55e7d3 100644 --- a/specs/bds.allium +++ b/specs/bds.allium @@ -1,7 +1,7 @@ -- allium: 1 -- bDS (Blogging Desktop Server) — Axiom Specification --- Distilled from TypeScript implementation at ../bDS/ --- This is the behavioural baseline for the Rust rewrite (RuDS) +-- Distilled from the existing bDS application at ../bDS2/ +-- This is the behavioural baseline for the rewrite effort. -- An offline-first desktop application for blog authoring with -- static site generation, SSH publishing, AI integration, and @@ -20,6 +20,7 @@ use "./metadata.allium" as metadata -- Project config, categories, publi -- Infrastructure use "./search.allium" as search -- FTS5 full-text search with Snowball stemming +use "./rendering.allium" as rendering -- Template render assigns, filters, macros (preview + generation) use "./generation.allium" as generation -- Static site generation (sections, routes, hashing) use "./preview.allium" as preview -- Local HTTP preview server use "./publishing.allium" as publishing -- SSH upload (SCP / rsync) @@ -53,23 +54,24 @@ use "./embedding.allium" as embedding -- Semantic similarity (HNSW vectors use "./cli_sync.allium" as cli_sync -- CLI-to-app notification sync use "./metadata_diff.allium" as metadata_diff -- DB/filesystem diff and rebuild --- Compatibility contract (from RUST_PLAN.md) +-- Compatibility contract -- -- MUST stay identical: --- SQLite schema semantics, post markdown frontmatter, +-- persistence semantics, post markdown frontmatter, -- translation file naming, media sidecars, thumbnail conventions, -- template file formats, menu OPML, generated routes/feeds/sitemaps, --- FTS5 search behaviour, slug generation, metadata diff, rebuild-from-filesystem +-- full-text search behaviour, slug generation, metadata diff, +-- rebuild-from-filesystem -- -- MAY change intentionally: --- Implementation language (TS -> Rust), editor (WYSIWYG -> plain text + preview), --- scripting runtime (Lua), process model (Electron -> native), --- UI framework (React -> Iced) +-- implementation language, desktop container, UI framework, +-- editor implementation, internal process model, runtime libraries -- Resolved questions: -- -- 1. Slug generation scope: only German and English letters are used. --- Verify deunicode handles ä/ö/ü/ß/ÄÖÜ correctly against transliteration npm. +-- Verify transliteration preserves the established bDS behaviour for +-- ä/ö/ü/ß/ÄÖÜ. -- -- 2. Liquid subset: see template.allium for the exact subset. -- Only 5 tags, 4 standard filters, 2 custom filters, 5 operators. @@ -78,7 +80,6 @@ use "./metadata_diff.allium" as metadata_diff -- DB/filesystem diff and rebuil -- 3. Macro calling convention: [[macroslug param1=value1 ...]] -- Double-bracket syntax, not Liquid tags. Identical to current app. -- --- 4. AI provider model: Rust rewrite uses two configurable OpenAI-compatible --- endpoints (online + airplane mode) instead of the TypeScript app's --- 4 named providers (OpenCode Zen, Mistral, Ollama, LM Studio). +-- 4. AI provider model: the rewrite uses two configurable OpenAI-compatible +-- endpoints (online + airplane mode) rather than a fixed named-provider set. -- See ai.allium for details. diff --git a/specs/cli.allium b/specs/cli.allium new file mode 100644 index 0000000..07672e8 --- /dev/null +++ b/specs/cli.allium @@ -0,0 +1,175 @@ +-- allium: 1 +-- Workspace CLI tool (issue #25) +-- Scope: extension (Bucket G — MCP + Automation) +-- Distilled from: lib/bds/cli.ex, lib/bds/cli/commands.ex, lib/bds/cli/install.ex, +-- rel/overlays/cli/bin/bds-cli + +entity CliInvocation { + command: rebuild | repair | render | upload | push | pull | post | media + | gallery | config | project | tui | lua + incremental: Boolean + force: Boolean + exit_code: Integer + -- Parsed by Optimus: subcommands, options, flags, and auto-generated + -- help/version output. Unknown commands and invalid options exit 1 + -- with formatted errors on stderr. +} + +surface CliSurface { + facing _: CliRuntime + + provides: + CliCommandExecuted(command) + CliInstallRequested() +} + +invariant SharedDatabase { + -- The CLI boots the application in BDS_MODE=cli: the same repo, + -- settings, and cache database as the GUI/TUI app — but with no + -- HTTP listener, no SSH daemon, no window, and no sync watcher. + -- Console logging is redirected to the rotating log file so stdout + -- carries only command output. +} + +invariant CliWritesNotify { + -- Every CLI mutation (post/media/gallery creation, config set, + -- project add/switch, bulk rebuild/repair) inserts DbNotification + -- rows (from_cli: true) so a concurrently running app picks the + -- change up through its sync watcher (see cli_sync.allium). Bulk + -- maintenance uses wildcard entity ids per entity type. +} + +rule ExitCode { + when: CliCommandExecuted(command) + -- Success prints the result (stdout) and exits 0; any error prints + -- to stderr and exits 1. + requires: CliInvocation.command = command + ensures: CliInvocation.exit_code.updated() +} + +rule RebuildFull { + when: CliCommandExecuted(command: rebuild) + requires: not CliInvocation.incremental + -- The same step sequence as the GUI "Rebuild Database" + -- (BDS.Maintenance.full_rebuild_steps — posts, media, scripts, + -- templates, post links, thumbnails, embedding index), run + -- synchronously with progress on stdout. + ensures: CacheDatabaseRebuilt() +} + +rule RebuildIncremental { + when: CliCommandExecuted(command: rebuild) + requires: CliInvocation.incremental + -- Metadata diff, then auto-apply file→db for every difference and + -- import every orphan file. + ensures: CacheDatabaseRebuilt() +} + +rule Repair { + when: CliCommandExecuted(command: repair) + -- Subcommand argument selects the repair part: post-links, + -- media-links, thumbnails, embeddings, or search — the standard + -- rebuild tasks outside the full rebuild. + ensures: RepairTaskCompleted() +} + +rule Render { + when: CliCommandExecuted(command: render) + -- Default: render all site sections plus the search index. + -- --incremental: validate the generated output and apply only the + -- differences (targeted render + extra-file deletion + calendar). + -- --force: full re-render ignoring (but updating) content hashes. + requires: not (CliInvocation.incremental and CliInvocation.force) + ensures: SiteRendered() +} + +rule Upload { + when: CliCommandExecuted(command: upload) + -- Uses the project publishing preferences (ssh host/user/path/mode) + -- exactly like the app's upload, and waits for the publish job. + ensures: SiteUploaded() +} + +rule GitSync { + when: CliCommandExecuted(command: push | pull) + -- push: git push of the project repository to origin. + -- pull: git pull --ff-only, then the incremental cache update + -- (metadata diff auto-apply + orphan import) so the database + -- reflects the pulled files. + ensures: RepositorySynchronized() +} + +rule CreatePost { + when: CliCommandExecuted(command: post) + -- Post data from parameters or JSON on stdin (LLM-friendly). + -- Language is auto-detected when missing: the configured AI + -- endpoint (airplane mode routes to the local model), falling back + -- to the offline heuristic with a printed notice — the CLI + -- equivalent of the airplane-mode toast. After creation the same + -- auto-translation as the GUI is scheduled and awaited; when + -- nothing can be scheduled the user is told. + ensures: PostCreated() +} + +rule CreateMedia { + when: CliCommandExecuted(command: media) + -- Imports the image, then best-effort AI enrichment: generated + -- title, alt text, caption, and translations to all configured + -- blog languages (the gallery pipeline without post linking). + ensures: MediaImported() +} + +rule CreateGallery { + when: CliCommandExecuted(command: gallery) + -- Creates the post, then runs the shared gallery import pipeline + -- for every referenced image: import, mandatory post link, AI + -- enrichment, translations; finally the post auto-translation. + ensures: PostCreated() + ensures: MediaImported() +} + +rule Preferences { + when: CliCommandExecuted(command: config) + -- get/set/list of the global settings store shared with the app. + ensures: PreferencesAccessed() +} + +rule Projects { + when: CliCommandExecuted(command: project) + -- list, add (register a folder in the cache database), and + -- switch (change the active project). + ensures: ProjectRegistryUpdated() +} + +rule Tui { + when: CliCommandExecuted(command: tui) + -- Handled by the launcher script: exec the release in BDS_MODE=tui + -- so the interactive terminal UI owns the terminal. + ensures: TuiStarted() +} + +rule RunLuaTask { + when: CliCommandExecuted(command: lua) + -- Runs a utility ("task", long-running) script from the database by + -- slug in the active project, with the managed-job execution budget + -- (unlimited time/reductions) but synchronously. Macro and + -- transform scripts are rejected. + ensures: ScriptExecuted() +} + +rule InstallLauncher { + when: CliInstallRequested() + -- Install buttons in the GUI settings (Data section) and the TUI + -- settings (data form action field) both call + -- BDS.UI.SettingsForm.run_action("install_cli"): a shim exec'ing + -- the release's cli/bin/bds-cli launcher is written to + -- ~/.local/bin/bds-cli. Outside a packaged release the action + -- reports that installing requires the packaged application. + ensures: LauncherInstalled() +} + +invariant LauncherArgv { + -- bin/bds eval cannot forward arguments, so the launcher passes the + -- argv in BDS_CLI_ARGV joined with the ASCII unit separator (0x1f) + -- and the release evaluates BDS.CLI.main(). +} diff --git a/specs/cli_sync.allium b/specs/cli_sync.allium index 6390a14..f519bf5 100644 --- a/specs/cli_sync.allium +++ b/specs/cli_sync.allium @@ -15,6 +15,14 @@ entity DbNotification { is_processed: seen_at != null } +surface CliSyncRuntimeSurface { + facing _: CliSyncRuntime + + provides: + CliMutationPerformed(entity_type, entity_id, action) + DbFileChangeDetected() +} + rule CliWriteNotification { when: CliMutationPerformed(entity_type, entity_id, action) -- CLI inserts notification row; app watches for it @@ -29,7 +37,7 @@ rule CliWriteNotification { rule AppWatchNotifications { when: DbFileChangeDetected() - -- Watches SQLite DB file + WAL via filesystem watcher + -- Watches the persisted notification store for external mutations -- Debounced at 100ms let unseen = DbNotifications where seen_at = null and from_cli = true for n in unseen: @@ -54,7 +62,7 @@ rule PruneUnprocessedNotifications { } invariant AppNoopNotifier { - -- The Electron app uses a no-op notifier for its own writes + -- The desktop application uses a no-op notifier for its own writes -- It already knows about its own mutations -- Only CLI writes produce notification rows } diff --git a/specs/editor_chat.allium b/specs/editor_chat.allium index 8df4567..7d612ed 100644 --- a/specs/editor_chat.allium +++ b/specs/editor_chat.allium @@ -23,9 +23,72 @@ value ChatMessage { role: String -- user | assistant | system content: String -- user: plain text; assistant: GFM markdown tool_markers: List + inline_surfaces: List is_streaming: Boolean -- true while accumulating } +value InlineSurface { + id: String + type: "card" | "chart" | "form" | "list" | "metric" | "mindmap" | "table" | "tabs" | "text" | "json" + title: String? + subtitle: String? -- card only + body: String? -- card, text + actions: List -- card only + columns: List -- table + rows: List> -- table + chart_type: String? -- chart: bar/pie/line + series: List -- chart + max_value: Integer? -- chart + label: String? -- metric + value: String? -- metric + items: List -- list + nodes: List -- mindmap + fields: List -- form + submit_label: String? -- form + submit_action: String? -- form + tabs: List -- tabs + selected_index: Integer? -- tabs + raw: Map? -- json catch-all +} + +value SurfaceAction { + label: String + action: String + payload: Map +} + +value ChartSeries { + label: String + value: Integer + segments: List +} + +value MindmapNode { + id: String? + label: String + children: List +} + +value FormField { + key: String + label: String + input_type: String + placeholder: String? + value: String? + options: List + required: Boolean +} + +value FieldOption { + label: String + value: String +} + +value TabPanel { + label: String + content: List -- nested surfaces or text +} + value ToolMarker { tool_name: String args_preview: String -- string args truncated to config.chat_tool_args_max_length @@ -66,6 +129,7 @@ value ModelEntry { config { chat_tool_args_max_length: Integer = 30 chat_input_max_height: Integer = 200 + surface_form_debounce_ms: Integer = 500 } surface ChatPanelSurface { @@ -85,6 +149,9 @@ surface ChatPanelSurface { tm.tool_name tm.args_preview tm.is_complete + for sfc in msg.inline_surfaces: + sfc.type + sfc.id provides: ChatSendMessage(panel.conversation_id, panel.input_text) @@ -138,6 +205,19 @@ surface ChatPanelSurface { -- Actions dispatched through store, same as user clicks. -- Navigation actions open tabs with pin intent. + @guarantee InlineSurfaceRendering + -- Assistant render-tool calls produce structured inline surfaces + -- rendered between message content and tool markers. + -- 9 surface types: card, chart, form, list, metric, mindmap, table, tabs, text, json. + -- Render tool names: render_card, render_chart, render_form, render_list, + -- render_metric, render_mindmap, render_table, render_tabs. + -- Unrecognized render names render as json (raw dump). + -- Tab content nests any surface type including plain text. + -- Form surfaces persist field values in conversation surface_state. + -- Tab surfaces track selected_index in conversation surface_state. + -- Surfaces can be dismissed, persisted in conversation surface_state. + -- Form debounce: config.surface_form_debounce_ms. + @guarantee TokenTracking -- Token usage tracked per conversation. -- Displayed in status bar, not in the chat panel itself. diff --git a/specs/editor_media.allium b/specs/editor_media.allium index ae2cdd1..3fba3e0 100644 --- a/specs/editor_media.allium +++ b/specs/editor_media.allium @@ -116,7 +116,11 @@ surface MediaEditorSurface { -- Translate Metadata (globe icon). @guarantee PreviewArea - -- Images: rendered via bds-media:// protocol with cache-busting timestamp. + -- Images: rendered at full resolution via the /media-file/:id route + -- (serves the original file; falls back to the large thumbnail for + -- formats browsers cannot display inline) with cache-busting + -- timestamp. Never upscaled beyond natural size. The sidebar keeps + -- using small thumbnails. -- Non-images: SVG file icon placeholder + original filename text. @guarantee MetadataForm @@ -130,9 +134,15 @@ surface MediaEditorSurface { @guarantee TranslationsSection -- Shown only when language is set. -- List of existing translations: flag emoji + language name + title. - -- Per-translation actions: click to edit inline, refresh button, delete button. + -- Per-translation actions: click to edit (opens modal), refresh button, delete button. -- "No translations" message when list is empty. + @guarantee TranslationEditModal + -- Editing a translation opens a modal dialog ("Edit Translation"), not + -- an inline form. Hidden language field plus Title, Alt Text, and + -- Caption (textarea) inputs. Footer: Cancel and Save buttons. + -- Save persists the translation; Cancel/close discards. + @guarantee LinkedPostsSection -- "Link to Post" button opens inline post picker overlay. -- List of currently linked posts with document icon. Click navigates to post tab. @@ -214,8 +224,9 @@ rule MediaLinkToPost { rule MediaTranslationEdit { when: MediaTranslationEditClicked(media_id, language) - -- Loads translation fields inline (title, alt, caption) for that language - -- Edit in place, save persists to translated sidecar {path}.{lang}.meta + -- Opens the "Edit Translation" modal pre-filled with the translation's + -- title, alt, and caption for that language + -- Save persists to DB + translated sidecar {path}.{lang}.meta; Cancel discards } rule MediaTranslationRefresh { diff --git a/specs/editor_misc.allium b/specs/editor_misc.allium index 101ab3b..2aaab81 100644 --- a/specs/editor_misc.allium +++ b/specs/editor_misc.allium @@ -148,6 +148,16 @@ surface DashboardSurface { -- Single-click: preview tab. Double-click: pin tab. } +rule ComputeDashboardData { + when: DashboardRequested(project) + -- stats: post counts grouped by status (total = sum of all statuses), + -- plus media/image counts, total media size, tag and category counts. + -- timeline: posts grouped by (year, month) of created_at, newest first, + -- limited to the most recent config.dashboard_timeline_months with data. + -- tag_cloud / category_cloud / recent_posts populated per their guarantees. + ensures: Dashboard +} + -- ─── Menu editor view ──────────────────────────────────────── -- Visual editor for the OPML navigation menu (meta/menu.opml). @@ -448,10 +458,11 @@ rule SiteValidationScan { rule SiteValidationApply { when: SiteValidationApplyRequested(report) -- Classifies affected paths into generation sections (core, single, category, tag, date) - -- Renders only affected sections in parallel + -- Renders only affected routes (a post's own single page plus only its own + -- categories, tags, and date archives); unaffected routes are left untouched -- Deletes extra HTML files, removes empty directories - -- Regenerates calendar if anything changed - -- Rebuilds search index if anything rendered or deleted + -- Regenerates calendar if anything changed (feed/atom/404/sitemap/search + -- index are NOT rebuilt on apply) ensures: ApplyValidationRequested(report.project_id, report.affected_sections) } @@ -697,7 +708,7 @@ value ImportYearDistribution { value ImportConflict { item_type: String -- post | page | media item_name: String - resolution: String -- import | skip | merge + resolution: String -- ignore | overwrite | import } value ImportMacro { @@ -745,8 +756,8 @@ surface ImportAnalysisSurface { -- Year-by-year bar charts for posts + media. @guarantee ConflictsSection - -- Collapsible. Per-item dropdown: Import/Skip/Merge. - -- Default: Import for new items, Skip for existing matches. + -- Collapsible. Per-item dropdown: Ignore/Overwrite/Import. + -- Default: Import for new items, Ignore for existing matches. @guarantee TaxonomySection -- Collapsible. Category + tag pills. diff --git a/specs/editor_post.allium b/specs/editor_post.allium index 80727da..d6fd39b 100644 --- a/specs/editor_post.allium +++ b/specs/editor_post.allium @@ -19,7 +19,7 @@ value PostEditorView { metadata: PostEditorMetadata metadata_expanded: Boolean -- starts expanded when title is empty excerpt_expanded: Boolean - editor_mode: String -- visual | markdown | preview + editor_mode: String -- markdown | preview footer: PostEditorFooter } @@ -137,10 +137,14 @@ surface PostEditorSurface { @guarantee TagAutocomplete -- Tag input with autocomplete. - -- Standard: prefix match on existing tag names (case-insensitive). - -- When semanticSimilarityEnabled: also suggests tags from 10 similar posts, - -- weighted by similarity, top 5 shown. - -- Merged results: prefix matches first, then embedding suggestions. + -- While typing: substring match on existing tag names (case-insensitive), + -- top 8 shown, plus a "create tag" row when the query is new. + -- While the query is empty (on focus): semantic suggestions appear under a + -- "Suggested tags" label — tags drawn from up to 10 similar posts, + -- weighted by similarity, top 5, excluding tags already on the post. + -- Requires semanticSimilarityEnabled and a built embedding index; + -- it is an index read (no model inference) so it works offline and + -- yields nothing when similarity is disabled or unindexed. @guarantee TranslationFlagsBar -- Row of flag emoji buttons inline with metadata toggle. @@ -152,15 +156,15 @@ surface PostEditorSurface { -- Collapsible section with textarea (4 rows). @guarantee EditorBodyToolbar - -- Toolbar: "Content" label, mode toggle (Visual/Markdown/Preview), + -- Toolbar: "Content" label, mode toggle (Markdown/Preview), -- action buttons (markdown mode only): Gallery (with media count), -- Insert Post Link, Insert Media. @guarantee EditorModes - -- Visual: rich-text WYSIWYG editor. -- Markdown: code editor with markdown-with-macros language, -- highlighting [[macro ...]] syntax. Word wrap on, minimap off, 14px font. -- Preview: iframe showing rendered preview. + -- Note: visual/WYSIWYG mode not implemented; "visual" normalizes to markdown. @guarantee DragDropImages -- Drop image file onto editor area triggers import chain. @@ -193,8 +197,8 @@ invariant PostDirtyTracking { } invariant PostEditorModePersistence { - -- Editor mode (visual/markdown/preview) persists per session. - -- Default mode comes from editor settings. + -- Editor mode (markdown/preview) persists per session. + -- Default mode comes from editor settings (markdown). } -- ─── Post editor actions ──────────────────────────────────── @@ -227,6 +231,8 @@ rule PostTranslateAction { rule PostAutoTranslateOnSave { when: PostSaved(post_id) + -- Only an explicit manual save (or publish) triggers this. Auto-saves and + -- post creation (sidebar, deeplink/bookmarklet, import, scripting) never do. -- Gate: airplane mode check + auto_translate not disabled (doNotTranslate=false) -- For each configured blog language missing a translation: -- Enqueue background translation task (title model) @@ -247,7 +253,7 @@ rule PostPublishAction { rule PostDiscardChanges { when: PostDiscardRequested(post_id) -- Only available for published posts with pending draft changes - -- Native confirm dialog (rfd): "Discard changes to this post?" + -- System confirm dialog: "Discard changes to this post?" -- On confirm: reads published version from .md file, -- restores DB to published state (content=null, status=published) -- Editor reloads with restored content @@ -255,7 +261,7 @@ rule PostDiscardChanges { rule PostDeleteAction { when: PostDeleteRequested(post_id) - -- Native confirm dialog (rfd): "Delete this post?" + -- System confirm dialog: "Delete this post?" -- If published: also deletes .md file and all translation files -- If never published: only deletes DB record -- Removes from DB, closes tab, sidebar removes item diff --git a/specs/editor_script.allium b/specs/editor_script.allium index ed9fbf7..01196d8 100644 --- a/specs/editor_script.allium +++ b/specs/editor_script.allium @@ -16,9 +16,10 @@ value ScriptEditorView { title: String -- editable text input slug: String -- editable text input, auto-generated from title kind: String -- select: utility | macro | transform - entrypoint: String -- select: dynamically discovered functions, "main" always first + entrypoint: String -- select: discovered Lua functions available as entrypoints enabled: Boolean -- checkbox content: String -- code editor content + can_publish: Boolean -- true while status = draft created_at: String -- locale-formatted date updated_at: String -- locale-formatted date } @@ -37,20 +38,23 @@ surface ScriptEditorSurface { provides: ScriptSaveRequested(editor.script_id) + ScriptPublishRequested(editor.script_id) + when editor.can_publish ScriptRunRequested(editor.script_id) ScriptCheckSyntaxRequested(editor.script_id) ScriptDeleteRequested(editor.script_id) @guarantee HeaderLayout -- Header bar with script title tab. - -- Actions (right side): Save button, Run button, + -- Actions (right side): Save button, Publish button (shown only when + -- can_publish, i.e. status = draft), Run button, -- Check Syntax button, Delete button (danger style). @guarantee MetadataRow -- Two rows of metadata fields above the editor. -- Row 1: Title (text input), Slug (text input, auto-generated from title). -- Row 2: Kind (select: utility/macro/transform), - -- Entrypoint (select: dynamically discovered functions with "main" always first), + -- Entrypoint (select: discovered Lua functions exposed by the script), -- Enabled (checkbox). @guarantee EditorBody @@ -66,10 +70,19 @@ surface ScriptEditorSurface { rule ScriptSave { when: ScriptSaveRequested(script_id) - -- Lua syntax check first (parse validation via Lua parser/runtime semantics) + -- Lua syntax check first using the configured runtime semantics -- If syntax error: blocks save, shows error inline in editor gutter - -- If valid: bumps version, saves to DB + rewrites .lua file - -- Entrypoint list re-discovered from AST after save + -- If valid: bumps version, saves to DB + rewrites the published script file + -- Entrypoint list re-discovered from Lua source after save +} + +rule ScriptPublish { + when: ScriptPublishRequested(script_id) + -- Only offered while the script is a draft (can_publish gate on the button) + -- Validates Lua syntax first (publish gate); invalid source blocks publish + -- Saves current draft fields, then publishes (status -> published) + -- Writes the published script file to disk + -- See script.allium PublishScript } rule ScriptCheckSyntax { @@ -81,8 +94,8 @@ rule ScriptCheckSyntax { rule ScriptRun { when: ScriptRunRequested(script_id) - -- Executes script in Lua runtime - -- Calls configured entrypoint function (default: main) + -- Executes the script in the configured Lua runtime + -- Calls the configured Lua entrypoint function -- stdout/stderr directed to Output panel tab -- Output panel auto-opens if not already visible -- Errors shown in Output panel with line numbers @@ -91,6 +104,6 @@ rule ScriptRun { rule ScriptDelete { when: ScriptDeleteRequested(script_id) -- No confirmation dialog - -- Deletes DB record + .lua file on disk + -- Deletes DB record + published script file on disk -- Closes script tab, sidebar removes item } diff --git a/specs/editor_settings.allium b/specs/editor_settings.allium index d1db8f8..fb6d703 100644 --- a/specs/editor_settings.allium +++ b/specs/editor_settings.allium @@ -16,8 +16,10 @@ value SettingsView { editor_section: SettingsEditorSection? categories: List ai_section: SettingsAISection? + technology_section: SettingsTechnologySection? publishing_section: SettingsPublishingSection? mcp_section: SettingsMCPSection? + data_section: SettingsDataSection? } value SettingsProjectSection { @@ -29,11 +31,12 @@ value SettingsProjectSection { blog_languages: List -- checkboxes (main language disabled) default_author: String -- text input max_posts_per_page: Integer -- number input (1-500, default 50) + image_import_concurrency: Integer -- number input (1-8, default 4) blogmark_category: String -- select from categories } value SettingsEditorSection { - default_mode: String -- select: wysiwyg | markdown | preview + default_mode: String -- select: markdown | preview diff_view_style: String -- select: inline | side-by-side wrap_long_lines: Boolean -- checkbox hide_unchanged_regions: Boolean -- checkbox @@ -61,6 +64,12 @@ value SettingsAISection { system_prompt: String -- textarea (12 rows) + Save + Reset to Default } +value SettingsTechnologySection { + semantic_similarity_enabled: Boolean -- checkbox: enable duplicate search + related-post embeddings + -- Scripting runtime is fixed at the application layer; no runtime switch is exposed. + -- Saved together with project metadata (settings_project form). +} + value SettingsPublishingSection { ssh_mode: String -- select: scp | rsync ssh_host: String -- text input @@ -69,13 +78,19 @@ value SettingsPublishingSection { } value SettingsMCPSection { - status: String -- port number or "Not running" agents: List } value MCPAgentRow { agent_name: String -- Claude Code, Claude Desktop, GitHub Copilot, etc. - is_installed: Boolean -- Add/Remove toggle + is_supported: Boolean -- false agents show a disabled button + "not supported yet" note + is_installed: Boolean -- toggle label: Remove when installed, else Add + config_path: String? -- agent config file path (nil when unsupported) +} + +value SettingsDataSection { + -- Action-only section: rebuild buttons + Open Data Folder. No persisted fields. + rebuild_targets: List -- posts | media | scripts | templates | links | thumbnails | embedding } invariant SettingsProtectedCategories { @@ -87,11 +102,16 @@ invariant SettingsMCPAgents { -- MCP section has exactly 7 agent rows in this order: -- Claude Code, Claude Desktop, GitHub Copilot, Gemini CLI, -- OpenCode, Mistral Vibe, OpenAI Codex. + -- Only Claude Code and GitHub Copilot are supported; the rest render a + -- disabled toggle and a "not supported in the rewrite yet" note. } config { settings_max_posts_per_page: Integer = 500 settings_default_posts_per_page: Integer = 50 + settings_image_import_concurrency_default: Integer = 4 + settings_image_import_concurrency_min: Integer = 1 + settings_image_import_concurrency_max: Integer = 8 settings_system_prompt_rows: Integer = 12 } @@ -116,9 +136,11 @@ surface SettingsViewSurface { SettingsAISystemPromptReset() SettingsPublishingSaved(publishing_data) SettingsPublishingCleared() + SettingsTechnologySaved(technology_data) SettingsMCPAgentToggled(agent_name) SettingsRebuildRequested(entity_type) SettingsRegenerateThumbnailsRequested() + SettingsEmbeddingIndexRebuildRequested() SettingsOpenDataFolderRequested() StyleThemeSelected(theme_name) StyleApplyRequested(theme_name) @@ -131,14 +153,20 @@ surface SettingsViewSurface { -- Section 1: Project Name, Description (textarea 3 rows), Data Path (text + Browse + Reset), -- Public URL, Main Language (select), Blog Languages (checkbox grid, main disabled), -- Default Author, Max Posts Per Page (number 1-500), + -- Image Import Concurrency (number 1-8, default 4), -- Blogmark Category (select), Blogmark Bookmarklet (copy button), Save button. @guarantee BookmarkletCopy - -- Copy button copies bookmarklet JavaScript to clipboard. - -- Bookmarklet uses project's publicUrl to construct POST endpoint. + -- Copy button copies the generated bookmarklet JavaScript to the system + -- clipboard for pasting into a browser bookmark. + -- The bookmarklet captures the active tab's document.title and + -- location.href (each URI-encoded) and navigates to a + -- bds2://new-post?title=...&url=... deep link. When a project is active + -- its id is appended as &project_id=... so the link targets that + -- project regardless of which one is currently open. @guarantee EditorSection - -- Section 2: Default Editor Mode (select: WYSIWYG/Markdown/Preview), + -- Section 2: Default Editor Mode (select: Markdown/Preview), -- Diff View Style (select: Inline/Side-by-side), -- Wrap Long Lines (checkbox), Hide Unchanged Regions (checkbox). @@ -169,22 +197,27 @@ surface SettingsViewSurface { -- Per-model info: max output tokens, context window (when available). @guarantee TechnologySection - -- Section 5: Lua is the only scripting runtime in the Rust app; - -- there is no scripting language selector. - -- Semantic Similarity toggle. + -- Section 5: Semantic Similarity toggle + -- ("Enable duplicate search and related-post embeddings"). + -- Scripting Runtime row is read-only descriptive text: scripting capabilities + -- are configured at the application layer and expose no runtime switch here. + -- Save button persists with project metadata (settings_project form). @guarantee PublishingSection -- Section 6: SSH Mode (scp/rsync), Host, Username, Remote Path. -- Save + Clear buttons. @guarantee MCPSection - -- Section 7: Status badge (port or "Not running"). - -- 7 agent rows with Add/Remove toggle each. + -- Section 7: 7 agent rows, each with the agent label, its config-file path + -- as a subtitle (or "not supported yet" note), and a toggle button. + -- Supported agents (Claude Code, GitHub Copilot) toggle Add/Remove, + -- writing/removing the bDS server entry in the agent config file. + -- Unsupported agents render a disabled button. @guarantee DataMaintenanceSection - -- Section 8: 6 rebuild buttons (Posts from Files, Media from Files, + -- Section 8: 7 rebuild buttons (Posts from Files, Media from Files, -- Scripts from Files, Templates from Files, Links, - -- Regenerate Missing Thumbnails). + -- Regenerate Missing Thumbnails, Rebuild Embedding Index). -- Open Data Folder button. -- Each rebuild executes immediately (no confirmation). -- Runs as background task with progress in Tasks panel. @@ -198,7 +231,7 @@ surface SettingsViewSurface { rule SettingsRebuild { when: SettingsRebuildRequested(entity_type) - -- entity_type: posts | media | scripts | templates | links | thumbnails + -- entity_type: posts | media | scripts | templates | links | thumbnails | embedding -- Executes immediately (no confirmation dialog) -- Runs as background task with progress visible in Tasks panel -- On completion: wholesale replaces store data for that entity type @@ -207,10 +240,16 @@ rule SettingsRebuild { -- ─── Style view ─────────────────────────────────────────────── +-- The Style view is its OWN singleton tab (tab type `style`, see tabs.allium), +-- NOT one of the SettingsView collapsible sections. It is opened from the +-- Style menu/tab entry and renders the Pico CSS theme editor, distinct from +-- the `settings` tab. Shares the settings_editor code group only because both +-- persist into project metadata. + value StyleView { - themes: List - selected_theme: String? - applied_theme: String? + themes: List -- 20 named Pico themes (see metadata.supported_pico_themes) + selected_theme: String? -- local selection, defaults to applied_theme + applied_theme: String? -- persisted picoTheme (default when none set) preview_mode: String -- auto | light | dark } @@ -240,9 +279,14 @@ surface StyleViewSurface { when style.selected_theme != style.applied_theme StylePreviewModeChanged(mode) + @guarantee SeparateTab + -- Rendered in its own `style` singleton tab (tabs.allium), never inline + -- in the settings view. Requires an active project; no project => no view. + @guarantee ThemePicker -- Grid of theme buttons (one per Pico CSS theme). -- Each button: swatch with 3 colour tones (accent, light bg, dark bg) + theme name. + -- Theme name shown via display transform: "-" -> " ", first letter capitalised. -- Selected theme highlighted with aria-pressed. @guarantee ControlsRow diff --git a/specs/editor_template.allium b/specs/editor_template.allium index be6b3c1..5fc8141 100644 --- a/specs/editor_template.allium +++ b/specs/editor_template.allium @@ -18,6 +18,7 @@ value TemplateEditorView { kind: String -- select: post | list | not_found | partial enabled: Boolean -- checkbox content: String -- code editor content + can_publish: Boolean -- true while status = draft created_at: String -- locale-formatted date updated_at: String -- locale-formatted date } @@ -35,12 +36,15 @@ surface TemplateEditorSurface { provides: TemplateSaveRequested(editor.template_id) + TemplatePublishRequested(editor.template_id) + when editor.can_publish TemplateValidateRequested(editor.template_id) TemplateDeleteRequested(editor.template_id) @guarantee HeaderLayout -- Header bar with template title tab. - -- Actions (right side): Save button, Validate button, + -- Actions (right side): Save button, Publish button (shown only when + -- can_publish, i.e. status = draft), Validate button, -- Delete button (danger style). @guarantee MetadataRow @@ -67,6 +71,15 @@ rule TemplateSave { -- See engine_side_effects.allium UpdateTemplateSideEffects } +rule TemplatePublish { + when: TemplatePublishRequested(template_id) + -- Only offered while the template is a draft (can_publish gate on the button) + -- Validates Liquid first (publish gate); invalid source blocks publish + -- Saves current draft fields, then publishes (status -> published) + -- Writes the published .liquid file to disk + -- See template.allium PublishTemplate +} + rule TemplateValidate { when: TemplateValidateRequested(template_id) -- Validates Liquid syntax without saving @@ -77,7 +90,7 @@ rule TemplateValidate { rule TemplateDelete { when: TemplateDeleteRequested(template_id) -- Checks for references: posts using this template, tags with postTemplateSlug - -- If references exist: native confirm dialog (rfd) + -- If references exist: system confirm dialog -- "This template is used by N posts and M tags. Force delete?" -- Force delete: nulls templateSlug on referencing posts, -- nulls postTemplateSlug on referencing tags diff --git a/specs/embedding.allium b/specs/embedding.allium index c9aeb06..66f007d 100644 --- a/specs/embedding.allium +++ b/specs/embedding.allium @@ -9,15 +9,49 @@ use "./post.allium" as post use "./tag.allium" as tag +surface EmbeddingModelSurface { + context model: EmbeddingModel + + exposes: + model.model_id + model.dimensions +} + +surface EmbeddingRuntimeSurface { + facing _: EmbeddingRuntime + + provides: + PostCreated(post) + PostUpdated(post) + PostDeleted(post) +} + +surface EmbeddingControlSurface { + facing _: EmbeddingOperator + + provides: + ReindexAllRequested(project) + IndexUnindexedRequested(project) + FindSimilarRequested(post, limit) + ComputeSimilaritiesRequested(source_post, target_post_ids) + SuggestTagsRequested(post, input_text) + FindDuplicatesRequested(project) + DismissDuplicatePairRequested(post_a, post_b) +} + -- ─── Model ────────────────────────────────────────────────── value EmbeddingModel { -- multilingual-e5-small: 384-dimensional sentence embeddings - -- Loaded via ONNX runtime (ort crate), model files from Hugging Face Hub + -- Model files are obtained from an external model source and cached locally -- Downloaded on first use, cached in app data directory -- Lazy-loaded: pipeline created on first embedding request, not at startup -- Text preprocessing: prefix all input with "query: " (e5 convention) -- Pooling: mean pooling + L2 normalization + -- Loaded on-device via Bumblebee (EMLX/Apple GPU or EXLA-CPU, see + -- NativeAcceleratedExecution); the canonical e5 weights come from + -- the "intfloat/multilingual-e5-small" repository, surfaced under the + -- "Xenova/multilingual-e5-small" model_id identifier. model_id: String -- "Xenova/multilingual-e5-small" dimensions: Integer -- 384 } @@ -30,7 +64,7 @@ value EmbeddingVector { -- ─── Entities ─────────────────────────────────────────────── entity EmbeddingKey { - label: Integer -- HNSW label for USearch + label: Integer -- HNSW node label / id post: post/Post content_hash: String -- SHA-256 of "{title}\n\n{content}" vector: EmbeddingVector @@ -42,9 +76,11 @@ entity DismissedDuplicatePair { -- IDs stored in canonical order (sorted) for dedup } --- ─── USearch HNSW Index ───────────────────────────────────── +-- ─── HNSW Index ───────────────────────────────────────────── config { + -- HNSW approximate-nearest-neighbour index (hnswlib). USearch has no Elixir + -- binding; hnswlib provides the same HNSW algorithm and parameters. model_id: String = "Xenova/multilingual-e5-small" embedding_dimensions: Integer = 384 hnsw_metric: String = "cosine" @@ -53,7 +89,10 @@ config { hnsw_expansion_search: Integer = 64 -- efSearch debounce_persist: Duration = 5.seconds -- Index file: {userData}/projects/{projectId}/embeddings.usearch - -- Key mapping in SQLite embedding_keys table (label, post_id, content_hash, vector BLOB) + -- Key mapping (label → post_id) persisted in a sidecar (.meta.json) next + -- to the index file, plus the source-of-truth rows in embedding_keys + batch_size: Integer = 16 -- texts per batched inference run + sequence_length: Integer = 256 -- max tokens per input (truncated) } -- ─── Gating ───────────────────────────────────────────────── @@ -77,7 +116,7 @@ rule EmbedPost { let existing = EmbeddingKey{post: post} if not exists existing or existing.content_hash != hash: -- Compute embedding vector via local model - -- Upsert into USearch index + embedding_keys DB table + -- Upsert into HNSW index + embedding_keys DB table -- Debounced index save (5s) ensures: EmbeddingKeyUpdated(post) } @@ -116,9 +155,9 @@ rule IndexUnindexed { rule FindSimilar { when: FindSimilarRequested(post, limit) requires: semantic_similarity_enabled - -- HNSW approximate nearest neighbor search via USearch + -- HNSW approximate nearest neighbor search (hnswlib) -- Searches index for (limit + 1) neighbors, excludes self - -- Converts USearch cosine distance to similarity: max(0, 1 - distance) + -- Converts HNSW cosine distance to similarity: max(0, 1 - distance) -- Returns ranked list sorted by descending similarity ensures: SimilarPostsResult(post, ranked_matches) } @@ -127,7 +166,7 @@ rule ComputeSimilarities { when: ComputeSimilaritiesRequested(source_post, target_post_ids) requires: semantic_similarity_enabled -- Exact pairwise cosine similarity between source vector and each target vector - -- Uses in-memory vector cache, NOT USearch search + -- Uses in-memory vector cache, NOT the HNSW index -- Returns map of post_id -> similarity score -- Used by InsertPostLinkModal to rank FTS search results ensures: SimilarityScoresResult(source_post, scores) @@ -172,7 +211,7 @@ invariant ContentHashSkipsUnchanged { } invariant DebouncedPersistence { - -- USearch index persistence is debounced at 5 seconds + -- HNSW index persistence is debounced at 5 seconds -- Prevents excessive disk I/O during bulk operations -- Index also force-saved on project switch and app shutdown } @@ -183,6 +222,34 @@ invariant VectorCacheInDb { -- Enables instant reload without re-embedding } +invariant RealNeuralModel { + -- Embeddings MUST be produced by the actual ONNX neural model (multilingual-e5-small), + -- not by lexical approximations (TF-IDF, bag-of-words, hash projections). + -- Cross-language semantic similarity is a primary requirement: + -- posts in different languages about the same topic must produce similar vectors. + -- This is only achievable with the trained multilingual transformer model. +} + +invariant NativeAcceleratedExecution { + -- Model execution MUST use the platform's native hardware acceleration + -- where available (GPU/Metal/Neural Engine on Apple Silicon, CUDA on + -- NVIDIA, etc.), and otherwise fall back to optimised native CPU execution. + -- Inference MUST be batched: batch_size inputs are run per compiled + -- inference pass and inputs are truncated to a bounded sequence_length, so + -- (re)indexing many posts is not serialised one document at a time. + -- Current implementation: Bumblebee with a runtime-selected defn compiler. + -- On Apple Silicon the model runs on the Apple GPU via EMLX (MLX/Metal, + -- params placed on the EMLX.Backend GPU device); everywhere else, and as a + -- fallback when EMLX is unavailable, it runs on optimised native CPU via + -- EXLA. Selection is `accelerator: :auto | :emlx | :exla` (default :auto); + -- a request for an unavailable backend degrades to the other one. + -- Packaged releases ship only the backend their platform can run (runtime: + -- conditional deps): macOS releases exclude EXLA, Linux/Windows releases + -- exclude the Apple-only EMLX. The selected backend is started on demand + -- by the Neural backend, never assumed started at boot. + -- Neighbour search is HNSW (hnswlib). +} + invariant ModelCaching { -- Model files (~100 MB) downloaded from Hugging Face Hub on first use -- Cached in app data directory, persists across sessions @@ -190,7 +257,7 @@ invariant ModelCaching { } invariant ProjectIsolation { - -- Each project has its own USearch index file and embedding_keys rows + -- Each project has its own HNSW index file and embedding_keys rows -- On project switch: save current index, load new project's index -- Model pipeline shared across projects (not reloaded) } diff --git a/specs/engine_side_effects.allium b/specs/engine_side_effects.allium index 1e5da20..e9f212a 100644 --- a/specs/engine_side_effects.allium +++ b/specs/engine_side_effects.allium @@ -114,7 +114,7 @@ rule ImportMediaSideEffects { if media.is_image: ensures: ThumbnailsGenerated(media) -- small=150px, medium=400px, large=800px, ai=448x448 - -- Asynchronous, emits thumbnailsGenerated on completion + -- Synchronous (awaited), logged on error ensures: FTSIndexUpdated(media) } @@ -185,7 +185,7 @@ rule DeleteTemplateSideEffects { -- ─── Script operations ─────────────────────────────────── -- Same pattern as templates: --- Create: write .lua file, insert DB +-- Create: write published script file, insert DB -- Update: bump version, rewrite file, update DB -- Publish: write file, clear DB content -- Delete: delete file, delete DB row @@ -299,7 +299,7 @@ rule DeleteMediaTranslationSideEffects { -- updatePost | rename only* | yes | if Δ | no | no | yes | no -- publishPost | .md + trans | yes | yes | no | no | yes | no -- deletePost | delete .md | del | del | no | Δ media | del | no --- importMedia | copy file | yes | no | async | write | no | no +-- importMedia | copy file | yes | no | sync | write | no | no -- updateMedia | no | yes | no | no | rewrite | no | no -- replaceMediaFile | overwrite | no | no | regen | no | no | no -- deleteMedia | delete all | del | no | del | del all | no | no diff --git a/specs/events.allium b/specs/events.allium new file mode 100644 index 0000000..7312a75 --- /dev/null +++ b/specs/events.allium @@ -0,0 +1,59 @@ +-- allium: 1 +-- Domain Event Bus (issue #26, phase 2) +-- Scope: extension (multi-client synchronization over Phoenix.PubSub) +-- Distilled from: lib/bds/events.ex, context broadcast call sites, +-- lib/bds/desktop/shell_live.ex, lib/bds/cli_sync/watcher.ex + +entity EntityChangedEvent { + entity: String -- post, media, tag, template, script + entity_id: String + action: created | updated | deleted + -- Same topic and payload shape as the CLI sync watcher, so one + -- subscription covers in-app mutations and external CLI writes. +} + +entity SettingsChangedEvent { + key: String -- e.g. ui.language +} + +surface EventBusSurface { + facing _: EventBus + + provides: + ContextMutationSucceeded(entity, entity_id, action) + GlobalSettingWritten(key) + ClientSubscribed() +} + +rule BroadcastEntityMutation { + when: ContextMutationSucceeded(entity, entity_id, action) + -- Posts, Media, Tags, Templates, Scripts broadcast after every + -- successful create/update/publish/delete (publish maps to updated). + ensures: EntityChangedEvent.created( + entity: entity, + entity_id: entity_id, + action: action + ) +} + +rule BroadcastSettingsChange { + when: GlobalSettingWritten(key) + ensures: SettingsChangedEvent.created(key: key) +} + +rule ClientSynchronization { + when: ClientSubscribed() + -- Every shell (LiveView GUI or TUI session) subscribes on mount and + -- refreshes affected views on each event; deleted entities close + -- their open tabs. This keeps all connected clients synchronized + -- regardless of which client or pipeline originated the change. + ensures: ClientRefreshOnEvent() +} + +rule ServerSideUiLanguage { + when: GlobalSettingWritten(key: "ui.language") + -- The UI language is a single server-side setting: changing it in any + -- client persists it and re-renders every connected client. The OS + -- locale is only the fallback when the setting is unset. + ensures: AllClientsRelocalized() +} diff --git a/specs/frontmatter.allium b/specs/frontmatter.allium index 190f140..f535efb 100644 --- a/specs/frontmatter.allium +++ b/specs/frontmatter.allium @@ -1,12 +1,12 @@ -- allium: 1 -- bDS Frontmatter Specifications -- Scope: core (Wave 1 — exact file format compatibility) --- Distilled from: ../bDS/src/main/engine/postFileUtils.ts, +-- Distilled from: ../bDS2/src/main/engine/postFileUtils.ts, -- TemplateEngine.ts, ScriptEngine.ts, MediaEngine.ts -- -- This document specifies the exact YAML frontmatter format for all --- file types. The Rust implementation must read and write these --- formats byte-for-byte compatible with the TypeScript implementation. +-- file types. The rewrite must read and write these formats compatibly +-- with existing bDS content. surface FrontmatterPersistenceSurface { facing _: ContentPersistenceRuntime @@ -25,7 +25,7 @@ surface PostFrontmatterSurface { frontmatter.title frontmatter.slug frontmatter.status - frontmatter.published_at + frontmatter.publishedAt frontmatter.tags frontmatter.categories } @@ -35,11 +35,11 @@ surface MediaSidecarSurface { exposes: sidecar.id - sidecar.original_name - sidecar.mime_type + sidecar.originalName + sidecar.mimeType sidecar.width sidecar.height - sidecar.updated_at + sidecar.updatedAt } surface TemplateFrontmatterSurface { @@ -70,21 +70,25 @@ surface MenuOpmlSurface { exposes: document.header.title - document.header.date_created - document.header.date_modified + document.header.dateCreated + document.header.dateModified for item in document.body: item.kind item.label item.slug } +config { + script_extension: String = "lua" +} + -- ============================================================================ -- POST FILE FORMAT -- ============================================================================ value PostFrontmatter { -- File path: posts/{YYYY}/{MM}/{slug}.md - -- For translations: posts/{YYYY}/{MM}/{slug}.{language}.md + -- All keys serialized as camelCase in YAML frontmatter id: String -- UUID v4 title: String slug: String @@ -92,15 +96,46 @@ value PostFrontmatter { status: draft | published | archived author: String? -- Only written if present language: String? -- Only written if present (ISO 639-1) - do_not_translate: Boolean -- Only written when true - template_slug: String? -- Only written if present - created_at: Timestamp -- Unix timestamp in milliseconds - updated_at: Timestamp -- Unix timestamp in milliseconds - published_at: Timestamp? -- Only written if published + doNotTranslate: Boolean -- Only written when true + templateSlug: String? -- Only written if present + createdAt: Timestamp -- Unix timestamp in milliseconds + updatedAt: Timestamp -- Unix timestamp in milliseconds + publishedAt: Timestamp? -- Only written if published tags: List -- Always written, even if empty categories: List -- Always written, even if empty } +value TranslationFrontmatter { + -- File path: posts/{YYYY}/{MM}/{slug}.{language}.md + -- Translation files carry their own publication state and timestamps + -- so that each translation can be rebuilt independently. + -- All keys serialized as camelCase in YAML frontmatter + id: String -- UUID v4 + translationFor: String -- Canonical post UUID + language: String -- ISO 639-1 language code + title: String -- Translated title + excerpt: String? -- Only written when the translated excerpt differs + status: draft | published + createdAt: Timestamp -- Unix timestamp in milliseconds + updatedAt: Timestamp -- Unix timestamp in milliseconds + publishedAt: Timestamp -- Canonical post's publishedAt at time of publish +} + +surface TranslationFrontmatterSurface { + context frontmatter: TranslationFrontmatter + + exposes: + frontmatter.id + frontmatter.translationFor + frontmatter.language + frontmatter.title + frontmatter.excerpt when frontmatter.excerpt != null + frontmatter.status + frontmatter.createdAt + frontmatter.updatedAt + frontmatter.publishedAt +} + invariant PostFileLayout { -- Posts are stored in date-based directory structure -- YYYY and MM derived from created_at (zero-padded) @@ -121,6 +156,14 @@ invariant PostTranslationFileLayout { lang: t.language) } +invariant TranslationFrontmatterRoundtrip { + -- Translation files carry status and timestamps explicitly. + -- On rebuild, these fields are read back directly; fallback to canonical + -- post values applies only when fields are absent (legacy files). + for t in PostTranslations where file_path != "": + parse_frontmatter(read_file(t.file_path)) = translation_frontmatter_fields(t) +} + rule WritePostFile { when: PublishPostRequested(post) ensures: FileWritten( @@ -138,11 +181,12 @@ rule WritePostFile { value MediaSidecar { -- File path: {binary_path}.meta (e.g., media/2024/03/a1b2c3d4.jpg.meta) -- Binary file at: media/{YYYY}/{MM}/{uuid}.{ext} - -- Format: YAML-like key-value (hand-built, not gray-matter frontmatter) + -- Format: YAML-like key-value wrapped in --- delimiters (gray-matter style, hand-built serializer) -- Note: 'filename' is NOT written to sidecar — it is implicit from the binary path + -- All keys serialized as camelCase id: String -- UUID v4 - original_name: String -- Original uploaded filename - mime_type: String + originalName: String -- Original uploaded filename + mimeType: String size: Integer -- Bytes width: Integer? height: Integer? @@ -152,8 +196,9 @@ value MediaSidecar { author: String? -- Only written if present language: String? -- Only written if present tags: List -- Always written, even if empty - created_at: Timestamp - updated_at: Timestamp + linkedPostIds: List -- UUIDs of posts that reference this media + createdAt: Timestamp + updatedAt: Timestamp } invariant MediaSidecarLayout { @@ -167,14 +212,16 @@ invariant MediaSidecarLayout { value TemplateFrontmatter { -- File path: templates/{slug}.liquid + -- All keys serialized as camelCase in YAML frontmatter id: String -- UUID v4 + projectId: String -- Scoped to project slug: String title: String kind: post | list | not_found | partial enabled: Boolean version: Integer - created_at: Timestamp - updated_at: Timestamp + createdAt: Timestamp + updatedAt: Timestamp } rule WriteTemplateFile { @@ -192,24 +239,26 @@ rule WriteTemplateFile { -- ============================================================================ value ScriptFrontmatter { - -- File path: scripts/{slug}.lua + -- File path: scripts/{slug}.{extension} -- YAML frontmatter delimited by --- markers + -- All keys serialized as camelCase in YAML frontmatter id: String -- UUID v4 + projectId: String -- Scoped to project slug: String title: String kind: macro | utility | transform - entrypoint: String -- Default: "render" + entrypoint: String -- Default: "render" for macros, "main" otherwise enabled: Boolean version: Integer - created_at: Timestamp - updated_at: Timestamp + createdAt: Timestamp + updatedAt: Timestamp } rule WriteScriptFile { when: PublishScriptRequested(script) requires: ValidateScript(script.content) = valid ensures: FileWritten( - path: format("scripts/{slug}.lua", slug: script.slug), + path: format("scripts/{slug}.{extension}", slug: script.slug, extension: config.script_extension), content: format_script_file(script) ) ensures: script.content = null @@ -219,24 +268,20 @@ rule WriteScriptFile { -- TAGS FILE FORMAT -- ============================================================================ -value TagsFile { - -- File path: meta/tags.json - -- Portable JSON format (no internal IDs) - tags: List -} - value TagEntry { + -- File path: meta/tags.json + -- Stored as a bare JSON array (no wrapper object) + -- Portable JSON format (no internal IDs), camelCase keys name: String color: String? - post_template_slug: String? + postTemplateSlug: String? } invariant TagsFileFormat { - -- Tags are stored as a sorted JSON array + -- Tags are stored as a bare sorted JSON array -- Sorted alphabetically by name (case-insensitive) - parse_json(read_file("meta/tags.json")) = { - tags: sort_by(Tags, t => lowercase(t.name)) - } + parse_json(read_file("meta/tags.json")) = + sort_by(tags, t => lowercase(t.name)) } -- ============================================================================ @@ -245,16 +290,17 @@ invariant TagsFileFormat { value ProjectJson { -- File path: meta/project.json + -- All keys serialized as camelCase name: String description: String? - public_url: String? - main_language: String? - default_author: String? - max_posts_per_page: Integer - blogmark_category: String? - pico_theme: String? - semantic_similarity_enabled: Boolean - blog_languages: List + publicUrl: String? + mainLanguage: String? + defaultAuthor: String? + maxPostsPerPage: Integer + blogmarkCategory: String? + picoTheme: String? + semanticSimilarityEnabled: Boolean + blogLanguages: List } value CategoriesJson { @@ -270,18 +316,19 @@ value CategoryMetaJson { } value CategorySettings { - render_in_lists: Boolean - show_title: Boolean - post_template_slug: String? - list_template_slug: String? + renderInLists: Boolean + showTitle: Boolean + postTemplateSlug: String? + listTemplateSlug: String? } value PublishingJson { -- File path: meta/publishing.json - ssh_host: String? - ssh_user: String? - ssh_remote_path: String? - ssh_mode: scp | rsync + -- All keys serialized as camelCase + sshHost: String? + sshUser: String? + sshRemotePath: String? + sshMode: scp | rsync } invariant MetadataFileLayout { @@ -292,7 +339,7 @@ invariant MetadataFileLayout { meta/category-meta.json = serialize(CategoryMetaJson) meta/publishing.json = serialize(PublishingJson) meta/menu.opml = serialize(Menu) - meta/tags.json = serialize(TagsFile) + meta/tags.json = serialize(List) } -- ============================================================================ @@ -308,8 +355,8 @@ value MenuOpml { value OpmlHeader { title: String - date_created: Timestamp - date_modified: Timestamp + dateCreated: Timestamp + dateModified: Timestamp } value MenuItem { @@ -343,6 +390,11 @@ invariant YamlFormatting { -- Boolean values are lowercase: true/false } +invariant CamelCaseKeys { + -- All serialized keys in YAML frontmatter and JSON metadata use camelCase. + -- Entity/DB fields use snake_case internally; the mapping happens at serialization. +} + invariant AtomicWrites { -- All file writes are atomic -- Write to temp file first, then rename @@ -357,7 +409,7 @@ invariant RequiredPostFields { -- These fields are ALWAYS written for posts for p in Posts: required_fields(p) = { - id, title, slug, status, created_at, updated_at, + id, title, slug, status, createdAt, updatedAt, tags, categories } } @@ -366,9 +418,9 @@ invariant ConditionalPostFields { -- These fields are ONLY written if truthy for p in Posts: conditional_fields(p) = { - excerpt, author, language, template_slug, published_at + excerpt, author, language, templateSlug, publishedAt } - -- do_not_translate is only written when true + -- doNotTranslate is only written when true } invariant RequiredMediaFields { @@ -376,8 +428,8 @@ invariant RequiredMediaFields { -- Note: 'filename' is NOT a sidecar field — it is the binary path itself for m in Media: required_fields(m) = { - id, original_name, mime_type, size, - created_at, updated_at, tags + id, originalName, mimeType, size, + createdAt, updatedAt, tags } } diff --git a/specs/generation.allium b/specs/generation.allium index bd7119d..41210f1 100644 --- a/specs/generation.allium +++ b/specs/generation.allium @@ -24,7 +24,28 @@ surface GenerationRuntimeSurface { provides: PageRenderRequested(template, context) + GenerationProgressReported(current, total, label) GenerateSiteCompleted(generation) + + @guarantee ProgressReporting + -- Generation, reindex, and site-validation run as background tasks and + -- emit progress via the task progress channel (see task.allium + -- ReportProgress / ProgressThrottled). + -- + -- Rendering (both full generation and validation apply) streams one + -- message per written page — " /url (n/total)" where verb is + -- "Generated" for a full render and "Rewrote" for a validation apply — + -- and advances the bar by the number of URLs written. + -- + -- Full generation and validation apply share the same structuring: a + -- task group containing one task per section (Render Site Core, Render + -- Single Posts, Render Category Archives, Render Tag Archives, Render + -- Date Archives) followed by a final Build Search Index task. The only + -- difference is that full generation renders every URL while apply + -- renders only the affected URLs. + -- + -- Multi-phase work (validation) maps each phase onto a fixed fraction + -- of the 0.0..1.0 bar. } value GenerationSection { @@ -63,6 +84,23 @@ surface GenerationStatusSurface { generation.generated_files.count } +invariant GenerationPublishedOnly { + -- Generation renders the *published* state of the blog, never draft content. + -- + -- Post universe: posts that have a published .md file on disk. + -- This includes status=published posts and status=draft posts that were + -- previously published (they still have a file_path with last-published content). + -- Posts that have never been published (no file_path) are excluded entirely. + -- + -- Content source: always the .md file on disk (the last-published snapshot). + -- The DB content field (which holds draft edits) is never read during generation. + -- Snapshots set content=nil to ensure file-based resolution. + -- + -- Contrast with preview (see preview.allium PreviewDraftOverlay): + -- Preview includes all drafts and prefers DB content over file content, + -- giving the author a live view of unpublished edits. +} + invariant IncrementalByContentHash { -- Files are only written when content_hash changes -- generatedFileHashes table tracks (projectId, relativePath, contentHash) @@ -75,6 +113,17 @@ invariant MultiLanguageRoutes { -- Each language subtree gets its own feeds and archives } +invariant LanguageVariantSelection { + -- Every language tree renders each post in its own language when a + -- matching variant exists, falling back to the post's original language. + -- This applies uniformly to single pages, list pages (home/pagination, + -- category/tag/date archives), and feeds: + -- Main tree: a post whose language differs from main_language renders + -- its main_language translation when one exists (canonical variant). + -- Additional-language trees: posts resolve to that language's + -- translation; do_not_translate posts are excluded from those trees. +} + invariant CanonicalBaseUrlConfigured { for generation in SiteGenerations: generation.base_url != "" @@ -98,16 +147,19 @@ rule GenerateCoreSectionPages { ensures: FileGenerated("index.html") ensures: FileGenerated("sitemap.xml") -- Multi-language sitemap with hreflang alternates - ensures: FileGenerated("feed.xml") - -- RSS 2.0 feed + ensures: FileGenerated("rss.xml") + -- RSS 2.0 feed (same output name as the old bDS app; templates link /rss.xml) ensures: FileGenerated("atom.xml") -- Atom feed ensures: FileGenerated("calendar.json") -- Post dates for calendar widget + ensures: FileGenerated("404.html") + -- Not-found page rendered from the not-found template for lang in generation.blog_languages - {generation.language}: ensures: FileGenerated(format("{lang}/index.html", lang: lang)) - ensures: FileGenerated(format("{lang}/feed.xml", lang: lang)) + ensures: FileGenerated(format("{lang}/rss.xml", lang: lang)) ensures: FileGenerated(format("{lang}/atom.xml", lang: lang)) + ensures: FileGenerated(format("{lang}/404.html", lang: lang)) } -- Single section: one HTML page per published post @@ -143,26 +195,46 @@ rule GenerateTagPages { when: GenerateSiteRequested(generation) requires: tag in generation.sections for t in Tags where post_count > 0: - ensures: FileGenerated(format("tag/{slug}/index.html", slug: slugify(t.name))) + let slug = slugify(t.name) + let page_count = ceil(posts_with_tag(t).count / generation.max_posts_per_page) + ensures: FileGenerated(format("tag/{slug}/index.html", slug: slug)) + for page in page_range(2, page_count): + ensures: FileGenerated(format("tag/{slug}/page/{page}/index.html", + slug: slug, page: page)) } --- Date section: year and month archives +-- Date section: year, month, and day archives rule GenerateDateArchivePages { when: GenerateSiteRequested(generation) requires: date in generation.sections for year in distinct_years(Posts): + let yp = ceil(posts_in_year(year).count / generation.max_posts_per_page) ensures: FileGenerated(format("{year}/index.html", year: year)) + for page in page_range(2, yp): + ensures: FileGenerated(format("{year}/page/{page}/index.html", + year: year, page: page)) for month in distinct_months(Posts, year): + let mp = ceil(posts_in_month(year, month).count / generation.max_posts_per_page) ensures: FileGenerated(format("{year}/{month}/index.html", year: year, month: month)) + for page in page_range(2, mp): + ensures: FileGenerated(format("{year}/{month}/page/{page}/index.html", + year: year, month: month, page: page)) + for day in distinct_days(Posts, year, month): + let dp = ceil(posts_in_day(year, month, day).count / generation.max_posts_per_page) + ensures: FileGenerated(format("{year}/{month}/{day}/index.html", + year: year, month: month, day: day)) + for page in page_range(2, dp): + ensures: FileGenerated(format("{year}/{month}/{day}/page/{page}/index.html", + year: year, month: month, day: day, page: page)) } -- Template rendering context rule RenderPage { when: PageRenderRequested(template, context) - -- LiquidJS rendering with full context: + -- Template rendering with full context: -- posts, pagination, menus, tags, categories, -- project metadata, i18n translations, theme settings -- Macro expansion: [[slug param1=value1 ...]] in post content @@ -196,7 +268,7 @@ invariant ArchiveDayBlocks { -- ============================================================================ -- Pagefind builds a client-side full-text search index from generated HTML. --- Uses the `pagefind` crate (library API), not a CLI subprocess. +-- Uses an embedded Pagefind library integration rather than a CLI subprocess. -- Runs as the final step of the generation pipeline, after all HTML is written. rule BuildSearchIndex { diff --git a/specs/git.allium b/specs/git.allium index 30ef9dc..eab9fdb 100644 --- a/specs/git.allium +++ b/specs/git.allium @@ -69,7 +69,7 @@ rule InitializeRepo { has_lfs: true ) ensures: GitignoreCreated(project) - -- .gitignore manages: thumbnails, generated html, node_modules, etc. + -- .gitignore manages generated artifacts, cached assets, dependency directories, etc. ensures: LfsTrackingConfigured(project) -- Git LFS auto-tracks image patterns (*.jpg, *.png, *.gif, etc.) } diff --git a/specs/layout.allium b/specs/layout.allium index 3c22aa1..f138f09 100644 --- a/specs/layout.allium +++ b/specs/layout.allium @@ -126,8 +126,9 @@ rule ToggleAssistantSidebar { value WindowTitleBar { -- Platform-adaptive - -- menu_bar: rendered only on non-Mac platforms (6 groups: App, File, Edit, View, Window, Help) - -- macOS: native menu bar via muda crate (same 6 groups) + -- menu_bar: rendered only on non-Mac platforms (5 groups: File, Edit, View, Blog, Help) + -- macOS: native menu bar (same groups plus Window between Blog and Help; + -- wx auto Window menu disabled, app supplies its own for a stable order) -- Keyboard: Alt opens mnemonics, Alt+letter opens group -- Arrow keys navigate groups and items, Enter/Space activates -- View group hides devTools toggle when not in dev mode @@ -248,6 +249,8 @@ invariant PanelTabFallback { -- Tasks tab: last 10 tasks, newest first, with progress/cancel. -- Tasks with shared group_id are collapsible groups showing aggregate progress. -- Output tab: log entries with copy-all button. +-- The panel opens on Output automatically when a blogmark transform fails +-- or a transform script writes output. -- Post Links tab: backlinks (posts linking here) + outlinks (posts linked from here). -- Each entry clickable, opens linked post as pinned tab. -- Git Log tab: file-level git history for active post/media (up to 50 entries). diff --git a/specs/mcp.allium b/specs/mcp.allium index 2a91932..d8e6fb9 100644 --- a/specs/mcp.allium +++ b/specs/mcp.allium @@ -8,6 +8,11 @@ use "./media.allium" as media use "./script.allium" as script use "./template.allium" as template +enum ProposalStatus { + pending + expired +} + entity McpServer { transport: http | stdio host: String -- 127.0.0.1 for HTTP @@ -27,7 +32,7 @@ surface McpServerSurface { entity Proposal { kind: draft_post | propose_script | propose_template | propose_media_metadata | propose_post_metadata - status: pending | accepted | discarded | expired + status: ProposalStatus entity_id: String data: String created_at: Timestamp @@ -42,8 +47,6 @@ entity Proposal { is_expired: expires_at <= now transitions status { - pending -> accepted - pending -> discarded pending -> expired } } @@ -68,8 +71,7 @@ surface ProposalSurface { config { http_port: Integer = 4124 - proposal_ttl_app: Duration = 30.minutes - proposal_ttl_cli: Duration = 8.hours + proposal_ttl: Duration = 30.minutes } surface McpAutomationSurface { @@ -80,6 +82,9 @@ surface McpAutomationSurface { McpToolInvoked("search_posts", params) McpToolInvoked("count_posts", params) McpToolInvoked("read_post_by_slug", slug, language) + McpToolInvoked("get_post_translations", post_id) + McpToolInvoked("get_media_translations", media_id) + McpToolInvoked("upsert_media_translation", params) McpToolInvoked("draft_post", params) McpToolInvoked("propose_script", params) McpToolInvoked("propose_template", params) @@ -87,8 +92,6 @@ surface McpAutomationSurface { McpToolInvoked("propose_post_metadata", params) AcceptProposalRequested(proposal) DiscardProposalRequested(proposal) - InstallAgentConfigRequested(agent_kind) - UninstallAgentConfigRequested(agent_kind) } invariant LocalhostOnlyHttp { @@ -161,6 +164,46 @@ surface CategoriesResource { -- bds://categories } +surface StatsResource { + facing viewer: McpClient + context blog: Blog + exposes: + blog.post_count + blog.media_count + blog.tag_count + blog.category_count + @guidance + -- bds://stats +} + +surface PostMediaResource { + facing viewer: McpClient + context post: post/Post + exposes: + for m in post.media: + m.id + m.filename + m.title + m.alt + m.caption + m.tags + @guidance + -- bds://posts/{id}/media + -- Unknown post ids return not_found. +} + +surface MediaImageResource { + facing viewer: McpClient + context media_item: media/Media + exposes: + media_item.mime_type + media_item.file_bytes + @guidance + -- bds://media/{id}/image + -- Returns blob content using the media MIME type. + -- Unknown media ids or missing files return not_found. +} + -- Read-only tools rule CheckTerm { @@ -201,6 +244,30 @@ rule ReadPostBySlug { ensures: FullPostContent(post) } +rule GetPostTranslations { + when: McpToolInvoked("get_post_translations", post_id) + -- Lists all available translations for a post. + ensures: PostTranslations(post_id) +} + +rule GetMediaTranslations { + when: McpToolInvoked("get_media_translations", media_id) + -- Lists all available translated metadata for a media item. + ensures: MediaTranslations(media_id) +} + +rule UpsertMediaTranslation { + when: McpToolInvoked("upsert_media_translation", params) + -- Creates or updates translated media metadata for a language. + ensures: media/UpsertMediaTranslationRequested( + params.media_id, + params.language, + params.title, + params.alt, + params.caption + ) +} + -- Write tools (proposal-based) rule DraftPost { @@ -218,7 +285,7 @@ rule DraftPost { entity_id: new_post.id, data: "", created_at: now, - expires_at: now + config.proposal_ttl_app, + expires_at: now + config.proposal_ttl, draft_post: new_post, proposed_script: null, proposed_template: null, @@ -244,7 +311,7 @@ rule ProposeScript { entity_id: new_script.id, data: "", created_at: now, - expires_at: now + config.proposal_ttl_app, + expires_at: now + config.proposal_ttl, draft_post: null, proposed_script: new_script, proposed_template: null, @@ -270,7 +337,7 @@ rule ProposeTemplate { entity_id: new_template.id, data: "", created_at: now, - expires_at: now + config.proposal_ttl_app, + expires_at: now + config.proposal_ttl, draft_post: null, proposed_script: null, proposed_template: new_template, @@ -289,7 +356,7 @@ rule ProposeMediaMetadata { entity_id: params.media_id, data: serialize(params), created_at: now, - expires_at: now + config.proposal_ttl_app, + expires_at: now + config.proposal_ttl, draft_post: null, proposed_script: null, proposed_template: null, @@ -308,7 +375,7 @@ rule ProposePostMetadata { entity_id: params.post_id, data: serialize(params), created_at: now, - expires_at: now + config.proposal_ttl_app, + expires_at: now + config.proposal_ttl, draft_post: null, proposed_script: null, proposed_template: null, @@ -335,7 +402,6 @@ rule AcceptProposal { media/UpdateMediaRequested(proposal.target_media, deserialize_media_changes(proposal.data)) if proposal.kind = propose_post_metadata: post/UpdatePostRequested(proposal.target_post, deserialize_post_changes(proposal.data)) - proposal.status = accepted not exists proposal } @@ -348,7 +414,6 @@ rule DiscardProposal { script/DeleteScriptRequested(proposal.proposed_script) if proposal.kind = propose_template: template/DeleteTemplateRequested(proposal.proposed_template) - proposal.status = discarded not exists proposal } @@ -374,14 +439,25 @@ surface McpAgentKindSurface { agent_kind.kind } +surface McpSettingsSurface { + provides: + SettingsMCPAgentToggled(agent_kind) + + @guidance + -- Agent configuration install/remove is exposed by the settings UI, + -- not by the MCP automation surface. +} + rule InstallAgentConfig { - when: InstallAgentConfigRequested(agent_kind) + when: SettingsMCPAgentToggled(agent_kind) + requires: not AgentConfigInstalled(agent_kind) -- Writes stdio MCP server config into the agent's config file ensures: AgentConfigInstalled(agent_kind) } rule UninstallAgentConfig { - when: UninstallAgentConfigRequested(agent_kind) + when: SettingsMCPAgentToggled(agent_kind) + requires: AgentConfigInstalled(agent_kind) ensures: AgentConfigRemoved(agent_kind) } diff --git a/specs/media.allium b/specs/media.allium index 4389732..56f378f 100644 --- a/specs/media.allium +++ b/specs/media.allium @@ -9,8 +9,9 @@ surface MediaControlSurface { facing _: MediaOperator provides: - ImportMediaRequested(project, source_file) + ImportMediaRequested(source_path, project, metadata) UpdateMediaRequested(media, changes) + ReplaceMediaFileRequested(media, new_source_path) DeleteMediaRequested(media) UpsertMediaTranslationRequested(media, language, title, alt, caption) RebuildMediaFromFilesRequested(project) @@ -121,22 +122,29 @@ invariant DateBasedMediaLayout { } rule ImportMedia { - when: ImportMediaRequested(project, source_file) - let uuid_name = generate_uuid() + extension(source_file) + when: ImportMediaRequested(source_path, project, metadata) + -- metadata is optional import context: title, alt, caption, author, + -- language, and tags may be supplied by the caller. + let uuid_name = generate_uuid() + extension(source_path) let dest = format("media/{yyyy}/{mm}/{uuid_name}", yyyy: now.year, mm: now.month_padded) ensures: Media.created( project: project, filename: uuid_name, - original_name: source_file.name, - mime_type: detect_mime(source_file), - size: source_file.size, - width: detect_width(source_file), - height: detect_height(source_file), + original_name: basename(source_path), + mime_type: detect_mime(source_path), + size: file_size(source_path), + width: detect_width(source_path), + height: detect_height(source_path), + title: metadata.title, + alt: metadata.alt, + caption: metadata.caption, + author: metadata.author, + language: metadata.language, file_path: dest, - tags: {} + tags: metadata.tags ) - ensures: FileCopied(source_file, dest) + ensures: FileCopied(source_path, dest) ensures: SidecarWritten(media) ensures: ThumbnailsGenerated(media) ensures: SearchIndexUpdated(media) @@ -151,6 +159,28 @@ rule UpdateMedia { ensures: SearchIndexUpdated(media) } +rule ReplaceMediaFile { + when: ReplaceMediaFileRequested(media, new_source_path) + -- Replaces the binary at media.file_path with the new source file, + -- keeping the same path/id. Sidecar metadata (title/alt/etc.) is preserved. + let checksum = md5(read(new_source_path)) + -- Identical content (checksum = media.checksum): no-op, nothing rewritten. + -- Otherwise the old file is backed up to {path}.bak (restored on failure, + -- removed on success) and the row is updated from the new file: + if checksum != media.checksum: + ensures: media.checksum = checksum + ensures: media.size = file_size(new_source_path) + ensures: media.updated_at = now + ensures: MediaDimensionsUpdated(media) + -- width/height re-read from the new image + ensures: SidecarWritten(media) + if media.is_image: + ensures: ThumbnailsRegenerated(media) + -- Synchronous (awaited), not fire-and-forget + ensures: SearchIndexUpdated(media) + -- See engine_side_effects.allium ReplaceMediaFileSideEffects +} + rule DeleteMedia { when: DeleteMediaRequested(media) ensures: not exists media @@ -179,7 +209,7 @@ rule UpsertMediaTranslation { rule RebuildMediaFromFiles { when: RebuildMediaFromFilesRequested(project) -- Scans media directory for .meta sidecars, reimports to DB - for sidecar in scan_directory(project.effective_data_dir + "/media", "*.meta"): + for sidecar in scan_directory(project.public_dir + "/media", "*.meta"): let parsed = parse_sidecar(sidecar) ensures: Media.created(parsed) -- or updated if already exists @@ -196,3 +226,27 @@ invariant SidecarRoundtrip { parse_sidecar(m.sidecar_path).caption = m.caption parse_sidecar(m.sidecar_path).tags = m.tags } + +rule BatchImportProcessLinkImages { + when: BatchImportImagesRequested(project, post, file_paths, language) + requires: not OfflineMode + for source_path in file_paths where is_image(source_path): + let media = ImportMedia(source_path, project) + let analysis = AnalyzeImage(media, language) + ensures: MediaUpdated(media, analysis) + ensures: PostMediaLinked(media, post) + @guidance + -- Triggered from post editor quick action "Add Gallery Images". + -- AI results auto-applied without user confirmation. + -- After metadata is set, media is auto-translated to all configured blog languages. + -- Non-image files skipped entirely. + -- Concurrency limit from project metadata image_import_concurrency (default 4, min 1, max 8). + -- Toast per completed image + final summary toast. + -- On completion: [[gallery]] macro inserted into post content and post editor refreshed. +} + +config { + batch_image_import_concurrency_default: Integer = 4 + batch_image_import_concurrency_min: Integer = 1 + batch_image_import_concurrency_max: Integer = 8 +} diff --git a/specs/media_processing.allium b/specs/media_processing.allium index d380020..688d0c6 100644 --- a/specs/media_processing.allium +++ b/specs/media_processing.allium @@ -1,7 +1,7 @@ -- allium: 1 -- bDS Media Processing Specification -- Scope: core (Wave 1 — media import and processing) --- Distilled from: ../bDS/src/main/engine/MediaEngine.ts, +-- Distilled from: ../bDS2/src/main/engine/MediaEngine.ts, -- mediaProcessing.ts, thumbnail generation logic -- -- This document specifies the exact media processing behavior: @@ -14,10 +14,11 @@ surface MediaProcessingControlSurface { facing _: MediaProcessingOperator provides: - ImportMediaRequested(source_path, project) + ImportMediaRequested(source_path, project, metadata) TagMediaRequested(media, tags) DeleteMediaRequested(media) ValidateMediaRequested(project) + RegenerateMissingThumbnailsRequested(project) } surface MediaProcessingRuntimeSurface { @@ -91,8 +92,10 @@ config { thumbnail_medium_width: Integer = 400 thumbnail_large_width: Integer = 800 thumbnail_ai_size: Integer = 448 -- 448x448 square crop, JPEG - thumbnail_format: String = "webp" -- All sizes except AI (encoder default quality) + thumbnail_format: String = "webp" -- All sizes except AI + thumbnail_quality: Integer = 80 -- WebP quality for small/medium/large thumbnail_ai_format: String = "jpeg" -- AI thumbnail only + thumbnail_ai_quality: Integer = 85 -- JPEG quality for AI thumbnail } rule GenerateThumbnails { @@ -103,36 +106,52 @@ rule GenerateThumbnails { source: media.file_path, destination: media.thumbnails.small, width: config.thumbnail_small_width, - format: config.thumbnail_format + format: config.thumbnail_format, + quality: config.thumbnail_quality ) ensures: ThumbnailGenerated( source: media.file_path, destination: media.thumbnails.medium, width: config.thumbnail_medium_width, - format: config.thumbnail_format + format: config.thumbnail_format, + quality: config.thumbnail_quality ) ensures: ThumbnailGenerated( source: media.file_path, destination: media.thumbnails.large, width: config.thumbnail_large_width, - format: config.thumbnail_format + format: config.thumbnail_format, + quality: config.thumbnail_quality ) ensures: ThumbnailGenerated( source: media.file_path, destination: media.thumbnails.ai, size: config.thumbnail_ai_size, - format: config.thumbnail_ai_format + format: config.thumbnail_ai_format, + quality: config.thumbnail_ai_quality ) } +rule RegenerateMissingThumbnails { + when: RegenerateMissingThumbnailsRequested(project) + -- Maintenance sweep over the project's raster images (images, excluding SVG), + -- oldest first. For each, any thumbnail file absent from disk is regenerated + -- from the original binary; images with a full set are skipped. + -- Runs as a background task reporting progress (current/total). + -- Returns counts: processed, generated, failed. + for media in project.media where is_image(media.mime_type) and not is_svg(media.mime_type): + if any_thumbnail_missing(media): + ensures: ThumbnailsRegenerated(media) +} + -- Thumbnail generation algorithm value ThumbnailGeneration { -- 1. Load source image -- 2. Apply EXIF orientation correction (rotation, flip) so thumbnails display correctly -- 3. Resize: small/medium/large preserve aspect ratio (width-constrained) -- AI thumbnail is a 448x448 center crop (letterboxed on black background) - -- 4. Encode as WebP (encoder default quality) for small/medium/large - -- Encode as JPEG for AI thumbnail + -- 4. Encode as WebP quality 80 for small/medium/large + -- Encode as JPEG quality 85 for AI thumbnail -- 5. Write to bucketed thumbnail path: thumbnails/{id[0:2]}/{id}-{size}.{ext} -- -- No _source copy is made. Thumbnails regenerated from the original binary. @@ -167,8 +186,8 @@ value ImageProcessing { } -- Processing rules: - -- 1. All thumbnails (except AI) are encoded as WebP (encoder default quality) - -- 2. AI thumbnail is encoded as JPEG (for vision model compatibility) + -- 1. All thumbnails (except AI) are encoded as WebP quality 80 + -- 2. AI thumbnail is encoded as JPEG quality 85 (for vision model compatibility) -- 3. Original format is preserved for full-size assets (no conversion) -- 4. EXIF data is not stripped (thumbnails are re-encoded, so EXIF is naturally absent) } @@ -233,11 +252,13 @@ invariant MediaTranslationFileLayout { -- ============================================================================ rule ImportMedia { - when: ImportMediaRequested(source_path, project) + when: ImportMediaRequested(source_path, project, metadata) + -- metadata is optional import context: title, alt, caption, author, + -- language, and tags may be supplied by the caller. -- 1. Validate file type (must be supported image) -- 2. Generate UUID v4 filename -- 3. Copy to media/{YYYY}/{MM}/{uuid}.{ext} - -- 4. Write sidecar {binary_path}.meta + -- 4. Apply optional metadata and write sidecar {binary_path}.meta -- 5. Generate four thumbnail sizes -- 6. Index for search (FTS5) ensures: media/Media.created( @@ -247,6 +268,12 @@ rule ImportMedia { size: file_size(source_path), width: extract_width_from_header(source_path), height: extract_height_from_header(source_path), + title: metadata.title, + alt: metadata.alt, + caption: metadata.caption, + author: metadata.author, + language: metadata.language, + tags: metadata.tags, file_path: format("media/{yyyy}/{mm}/{uuid}.{ext}"), sidecar_path: format("media/{yyyy}/{mm}/{uuid}.{ext}.meta"), checksum: sha256(source_path) @@ -345,7 +372,7 @@ invariant MediaSidecarFormat { -- ============================================================================ config { - -- No file size limit on import (TypeScript app has no max_original_size check) + -- No file size limit on import -- Original files are stored as-is (no compression, no resize) -- Only thumbnails are generated from the original strip_exif: Boolean = false -- Not explicitly stripped; re-encoding naturally omits it diff --git a/specs/menu.allium b/specs/menu.allium index e072955..1a9363d 100644 --- a/specs/menu.allium +++ b/specs/menu.allium @@ -1,28 +1,30 @@ -- allium: 1 -- bDS Navigation Menu -- Scope: core (read for rendering), extension Bucket F (menu editor UI) --- Distilled from: src/main/engine/MenuEngine.ts +-- File-only model: no DB table. Loaded from meta/menu.opml into a +-- transient value, mutated in memory, written back to OPML on save. surface MenuManagementSurface { facing _: MenuOperator provides: - UpdateMenuRequested(menu, items) + MenuLoadRequested(project_id) + UpdateMenuRequested(items) + SyncMenuFromFilesystemRequested(project_id) } value MenuItem { kind: page | submenu | category_archive | home label: String - slug: String? - children: List? -- only for submenu kind + slug: String? -- pageSlug for page/home, categoryName for category_archive + children: List? -- present only for submenu kind } -entity Menu { +value Menu { items: List -- Derived - home_items: items where kind = home - home_entry: home_items.first + home_entry: items.first -- always home after normalization } surface MenuSurface { @@ -30,27 +32,42 @@ surface MenuSurface { exposes: menu.items.count - menu.home_items.count menu.home_entry.label } -invariant HomeAlwaysPresent { - -- The menu always has a Home entry, extracted and prepended +invariant HomeAlwaysFirst { + -- Normalization guarantees home is always the first item. + -- UpdateMenu strips any home entries from input, then prepends one. for menu in Menus: menu.items.first.kind = home } invariant MenuPersistedAsOpml { - -- meta/menu.opml is the canonical storage format - -- Uses OPML with outline elements for each item + -- meta/menu.opml is the sole persistent store (no DB table). + -- OPML outline attributes: text (label), type (kind), + -- pageSlug (slug for page/home), categoryName (slug for category_archive). + -- Nested elements represent submenu children. parse_opml(read_file("meta/menu.opml")) = menu.items } -rule UpdateMenu { - when: UpdateMenuRequested(menu, items) - -- Normalizes Home entry: extracts from items, prepends - let without_home = items where kind != home - let home = MenuItem{kind: home, label: "Home"} - ensures: menu.items = build_menu_items(home, without_home) - ensures: MenuFileWritten(menu) +rule LoadMenu { + when: MenuLoadRequested(project_id) + -- Reads meta/menu.opml; if file missing, returns default (home-only) menu. + -- Normalizes: strips home entries from body, prepends canonical home. + ensures: MenuLoaded(project_id, normalize(parse_opml_or_empty(project_id))) +} + +rule UpdateMenu { + when: UpdateMenuRequested(items) + -- Normalizes Home entry: strips all home items, prepends canonical home. + -- Writes normalized menu back to meta/menu.opml. + let without_home = items where kind != home + ensures: MenuFileWritten(normalize(without_home)) +} + +rule SyncMenuFromFilesystem { + when: SyncMenuFromFilesystemRequested(project_id) + -- Reloads menu from OPML, normalizes, writes back (round-trip repair). + ensures: MenuLoaded(project_id, _) + ensures: MenuFileWritten(_) } diff --git a/specs/metadata.allium b/specs/metadata.allium index 1e875ba..5d6d2ec 100644 --- a/specs/metadata.allium +++ b/specs/metadata.allium @@ -5,6 +5,28 @@ use "./project.allium" as project +surface MetadataControlSurface { + facing _: MetadataOperator + + provides: + UpdateProjectMetadataRequested(project, changes) + AddCategoryRequested(project, name) + RemoveCategoryRequested(project, name) + UpdateCategorySettingsRequested(project, category, settings) + SetPublishingPreferencesRequested(project, prefs) + AppStarted(project) +} + +surface PublishingPreferencesSurface { + context prefs: PublishingPreferences + + exposes: + prefs.ssh_host when prefs.ssh_host != null + prefs.ssh_user when prefs.ssh_user != null + prefs.ssh_remote_path when prefs.ssh_remote_path != null + prefs.ssh_mode +} + value CategoryRenderSettings { render_in_lists: Boolean show_title: Boolean diff --git a/specs/metadata_diff.allium b/specs/metadata_diff.allium index 0b546fb..080c71a 100644 --- a/specs/metadata_diff.allium +++ b/specs/metadata_diff.allium @@ -14,6 +14,7 @@ surface MetadataMaintenanceSurface { provides: MetadataDiffRequested(project) RebuildFromFilesystemRequested(project, entity_type) + RepairMetadataDiffItemRequested(project, direction, item) } value DiffField { @@ -23,7 +24,8 @@ value DiffField { } value DiffReport { - entity_type: String -- post, media, script, template + entity_type: String -- post, post_translation, media, + -- media_translation, script, template, embedding entity_id: String differences: List } @@ -46,13 +48,46 @@ rule RunMetadataDiff { if diffs.count > 0: ensures: DiffReport.created(entity_type: "post", entity_id: post.id, differences: diffs) + -- Translation files only carry language-specific metadata. Shared status and + -- timestamp fields come from the canonical post and must not be reported as + -- missing when they are absent from the translation file. + for translation in project.post_translations: + let translation_file_data = parse_post_file(translation.file_path) + let translation_diffs = compare_translation_specific_fields(translation, translation_file_data) + if translation_diffs.count > 0: + ensures: + DiffReport.created( + entity_type: "post_translation", + entity_id: translation.id, + differences: translation_diffs + ) + -- Detect orphan files (on disk but not in DB) - for file in scan_directory(project.effective_data_dir + "/posts", "*.md"): + for file in scan_directory(project.public_dir + "/posts", "*.md"): let matching = Posts where file_path = file if matching.count = 0: ensures: OrphanReport.created(file_path: file) - -- Same pattern for media sidecar files, scripts, templates + -- Same pattern for media sidecars (media), media translation sidecars + -- (media_translation), scripts, templates, and embeddings. + -- Embedding diffs compare the stored content_hash against the live post + -- content to detect vectors that need recomputation. +} + +rule RepairMetadataDiffItem { + when: RepairMetadataDiffItemRequested(project, direction, item) + -- Resolves a single diff in one direction. + -- direction = file_to_db (filesystem wins) | db_to_file (database wins) + -- Dispatched per item.entity_type: + -- project | categories | category_meta | publishing -> project metadata sync/flush + -- post -> sync_post_from_file / rewrite_published_post + -- post_translation -> sync_post_translation_from_file / rewrite_published_post_translation + -- media -> sync_media_from_sidecar / sync_media_sidecar + -- media_translation -> sync_media_translation_from_sidecar / sync_media_translation_sidecar + -- script -> sync_script_from_file / sync_published_script_file + -- template -> sync_template_from_file / sync_published_template_file + -- embedding -> sync_post (file_to_db) / refresh_snapshot (db_to_file) + -- Unknown entity_type or direction -> unsupported error. } rule RebuildFromFilesystem { diff --git a/specs/modals.allium b/specs/modals.allium index 91ce39d..a661e96 100644 --- a/specs/modals.allium +++ b/specs/modals.allium @@ -192,7 +192,7 @@ surface ConfirmDialogSurface { modal.message } --- Native confirm dialogs (via rfd crate) are NOT modelled as values. +-- System confirm dialogs are NOT modelled as values. -- They are simple yes/no system dialogs with a message string. -- Used by: PostDelete, PostDiscard, TemplateDelete (with references). diff --git a/specs/post.allium b/specs/post.allium index ebe0811..f5c9168 100644 --- a/specs/post.allium +++ b/specs/post.allium @@ -5,14 +5,20 @@ use "./project.allium" as project +enum PostStatus { + draft + published + archived +} + value Slug { value: String -- Generated by: transliterate unicode to ASCII, lowercase, -- replace [^a-z0-9]+ with hyphens, strip leading/trailing hyphens -- Transliteration scope: only German (ä/ö/ü/ß/ÄÖÜ) and English letters used. - -- Verify deunicode handles this set correctly against transliteration npm. - -- Uniqueness: tries base, then {slug}-2 .. {slug}-999, then {slug}-{timestamp} + -- Verify transliteration matches the established bDS behaviour for this set. + -- Uniqueness: tries base, then {slug}-2, {slug}-3, … (unbounded numeric suffix) } value PostFilePath { @@ -40,13 +46,51 @@ value Frontmatter { -- doNotTranslate (only when true), templateSlug, publishedAt } +surface PostControlSurface { + facing _: PostOperator + + provides: + CreatePostRequested(project, title, content, tags, categories, author, language, template_slug) + UpdatePostRequested(post, changes) + PublishPostRequested(post) + DeletePostRequested(post) + ArchivePostRequested(post) + DiscardPostChangesRequested(post) + SyncPostFromFileRequested(post) + ImportOrphanPostFileRequested(project, relative_path) +} + +surface PostFilePathSurface { + context path: PostFilePath + + exposes: + path.base_dir + path.year + path.month + path.slug +} + +surface PostCanonicalUrlSurface { + context url: PostCanonicalUrl + + exposes: + url.year + url.month + url.day + url.slug +} + +surface FrontmatterSurface { + context _: Frontmatter +} + entity Post { project: project/Project title: String slug: Slug excerpt: String? content: String? - status: draft | published | archived + status: PostStatus author: String? language: String? do_not_translate: Boolean @@ -59,6 +103,15 @@ entity Post { updated_at: Timestamp published_at: Timestamp? + -- Published snapshot: copy of title/content/tags/categories/excerpt as of + -- the last publish. Used by changes_affect_published_content to decide when + -- an edit reopens a published post to draft (see ReopenPublishedPost). + published_title: String? + published_content: String? + published_tags: String? + published_categories: String? + published_excerpt: String? + -- Relationships translations: PostTranslation with canonical_post = this linked_media: PostMediaLink with post = this @@ -71,6 +124,10 @@ entity Post { -- Slug changes only allowed before first publish content_location: if status = published: file_path else: content -- Published: body in filesystem. Draft: body in DB field. + editor_body: if content != null: content else: read_markdown_body(file_path) + -- Resolver used by editors: prefer the in-DB draft content, else read + -- the markdown body from the post's file. Empty string when neither + -- exists. The same resolver applies to PostTranslation records. transitions status { draft -> published @@ -104,21 +161,23 @@ rule CreatePost { when: CreatePostRequested(project, title, content, tags, categories, author, language, template_slug) let slug = Slug.generate(title ?? "untitled") let unique_slug = Slug.ensure_unique(slug, project) - ensures: Post.created( - project: project, - title: title ?? "", - slug: unique_slug, - content: content, - status: draft, - author: author, - language: language, - tags: tags ?? {}, - categories: categories ?? {}, - template_slug: template_slug, - do_not_translate: false, - file_path: "" - ) - ensures: SearchIndexUpdated(post) + ensures: + let new_post = Post.created( + project: project, + title: title ?? "", + slug: unique_slug, + content: content, + status: draft, + author: author, + language: language, + tags: tags ?? {}, + categories: categories ?? {}, + template_slug: template_slug, + do_not_translate: false, + file_path: "" + ) + new_post.status = draft + SearchIndexUpdated(new_post) } rule UpdatePost { @@ -133,6 +192,13 @@ rule UpdatePost { -- status auto-transitions back to draft } +rule ReopenPublishedPost { + when: UpdatePostRequested(post, changes) + requires: post.status = published + requires: changes_affect_published_content(changes) + ensures: post.status = draft +} + rule PublishPost { when: PublishPostRequested(post) requires: post.status = draft or post.status = archived @@ -164,9 +230,43 @@ rule DeletePost { rule ArchivePost { when: ArchivePostRequested(post) + requires: post.status = draft or post.status = published ensures: post.status = archived } +rule SyncPostFromFile { + when: SyncPostFromFileRequested(post) + requires: post.file_path != "" + -- Single-post reimport: re-reads the post's own .md file and upserts the DB + -- record from it (the filesystem is treated as truth for that one post). + -- Re-syncs the post's link graph. Errors out if the file is missing. + ensures: PostFieldsUpdated(post, parse_post_file(post.file_path)) + ensures: PostLinksUpdated(post) +} + +rule ImportOrphanPostFile { + when: ImportOrphanPostFileRequested(project, relative_path) + -- Imports a .md file that exists on disk but has no DB record (an orphan, + -- e.g. surfaced by RunMetadataDiff). Rejects non-markdown / missing files. + ensures: + let new_post = Post.created(parse_post_file(relative_path)) + SearchIndexUpdated(new_post) +} + +rule DiscardPostChanges { + when: DiscardPostChangesRequested(post) + requires: post.file_path != "" + -- Only posts with a published file on disk can be discarded; + -- a never-published draft has no file version to restore. + -- Re-reads the published .md file and upserts the DB record from it, + -- discarding unsaved draft edits. + ensures: post.content = null + ensures: post.status = published + ensures: PostLinksUpdated(post) + ensures: SearchIndexUpdated(post) + -- See engine_side_effects.allium DiscardPostChangesSideEffects +} + -- File format axioms invariant FrontmatterRoundtrip { @@ -185,5 +285,5 @@ invariant DateBasedFileLayout { } -- Slug freeze: once published_at is set, the slug is permanently frozen. --- This matches TypeScript behaviour: is_slug_frozen = published_at != null +-- This follows the established bDS rule: is_slug_frozen = published_at != null -- Even if the post reverts to draft, the slug cannot be changed. diff --git a/specs/preview.allium b/specs/preview.allium index cd0bd15..924d947 100644 --- a/specs/preview.allium +++ b/specs/preview.allium @@ -49,18 +49,24 @@ rule StopPreview { } -- Route resolution +-- Preview renders all posts (published + draft) on-demand via Liquid templates. +-- Content priority: DB content (draft edits) over published .md file content. +-- See invariant PreviewDraftOverlay below. rule ServePostPreview { when: PreviewRequest(path) requires: is_post_path(path) -- path matches "/{yyyy}/{mm}/{dd}/{slug}" - -- Renders post via Liquid template with full PageRenderer context + -- Finds post by slug+date regardless of status (published or draft). + -- Content resolved via editor_body: DB content if present, else .md file. + -- Renders via Liquid template with full PageRenderer context. ensures: PreviewResponse(rendered_html) } rule ServeDraftPreview { when: PreviewDraftRequest(path, post_id) - -- Renders draft content (from DB, not filesystem) + -- Explicit draft preview by post_id (used by editor preview pane). + -- Renders draft content (from DB, not filesystem). ensures: PreviewResponse(rendered_html) } @@ -68,6 +74,7 @@ rule ServeArchivePreview { when: PreviewRequest(path) requires: is_archive_path(path) -- Category, tag, date archives with pagination + -- Includes both published and draft posts in listings. ensures: PreviewResponse(rendered_html) } @@ -92,6 +99,28 @@ rule ServeLanguagePrefixedRoute { ensures: PreviewResponse(translated_html) } +invariant PreviewDraftOverlay { + -- Preview is the draft workspace: it shows what the blog *will* look like, + -- not what it currently looks like on the published site. + -- + -- Post universe: all posts with status in {published, draft}. + -- Archived posts are excluded. + -- + -- Content priority (per post): + -- 1. DB content field (draft edits not yet published) → used when non-nil + -- 2. Published .md file (last-published snapshot) → used when DB content is nil + -- 3. Empty string → fallback if neither exists + -- + -- This means: + -- - A purely draft post (never published) renders from DB content. + -- - A published-then-edited post renders from DB content (the draft edits). + -- - A published post with no pending edits renders from its .md file. + -- + -- Contrast with generation (see generation.allium GenerationPublishedOnly): + -- Generation uses *only* published .md file content, never DB draft content, + -- and excludes posts that have never been published. +} + invariant ThemeSwitching { -- Preview supports live theme/mode switching via query params -- ?theme=amber&mode=dark etc. diff --git a/specs/project.allium b/specs/project.allium index 864027a..29d3a09 100644 --- a/specs/project.allium +++ b/specs/project.allium @@ -8,6 +8,7 @@ surface ProjectControlSurface { provides: CreateProjectRequested(name, data_path) + OpenProjectRequested(folder_path) SetActiveProjectRequested(project) DeleteProjectRequested(project) } @@ -27,11 +28,37 @@ entity Project { tags: Tag with project = this -- Derived - internal_base_dir: String - -- {user_data}/projects/{id}/ - -- Contains: meta/, thumbnails/, tags.json - effective_data_dir: data_path ?? internal_base_dir - -- Custom data path overrides default + -- + -- data_path is the project folder: the directory that CONTAINS + -- meta/project.json. It is DISCOVERED from the project.json location at + -- load time and is never written into project.json itself — so the folder + -- can be moved/renamed freely. The current location is remembered only as a + -- machine-local pointer (a project registry under private_dir), never + -- embedded in the portable project. See DataPathNotPersistedInProjectJson. + public_dir: data_path + -- All user-owned, portable, webserver-bound content lives here, under + -- the project folder: + -- posts (.md), media + thumbnails, templates/, scripts/, + -- meta/ (project.json, categories.json, category-meta.json, + -- publishing.json), tags.json, menu.opml, and generated html/ output. + -- See PublicContentLivesInProjectFolder. + private_dir: String + -- The OS per-user app-data directory. Holds app-internal, + -- machine-specific, regenerable artifacts ONLY — never inside the repo + -- or the project folder: + -- the SQLite database, the per-project embeddings index + -- (projects/{id}/embeddings.usearch + .meta.json sidecar), the + -- downloaded embedding-model cache, the project registry, and + -- window/UI state. + -- OS-specific base (resolved via :filename.basedir, app name "bds"): + -- macOS: ~/Library/Application Support/bds (:user_config) + -- Linux: $XDG_CONFIG_HOME/bds (default ~/.config/bds) + -- Windows: %APPDATA%\\bds + -- See PrivateArtifactsLiveInOsAppDir. + cache_dir: private_dir + "/projects/" + id + -- Per-project subtree of private_dir for that project's regenerable + -- artifacts (embeddings index + sidecar). Overridable in tests via the + -- :project_cache_root setting; never the repo or the project folder. } surface ProjectSurface { @@ -48,8 +75,8 @@ surface ProjectSurface { project.posts.count project.media.count project.tags.count - project.internal_base_dir - project.effective_data_dir + project.public_dir + project.private_dir } invariant SingleActiveProject { @@ -73,8 +100,52 @@ rule CreateProject { data_path: data_path, is_active: false ) - ensures: StarterTemplatesCopied(project) - -- Bundled starter templates are copied into the new project + @guidance + -- data_path is the chosen project folder. CreateProject writes + -- meta/project.json into {data_path}/meta/ but never records data_path + -- inside it (DataPathNotPersistedInProjectJson). The default project's + -- folder is created at a per-user default content location on first + -- launch — never inside the application repo or private_dir. +} + +rule DiscoverProjectDataPath { + -- Opening or loading a project folder deduces its data_path from the + -- on-disk location of meta/project.json rather than from any stored value, + -- keeping projects movable (DataPathNotPersistedInProjectJson). + when: OpenProjectRequested(folder_path) + -- folder_path contains meta/project.json + let project = Projects where data_path = folder_path + ensures: project.data_path = folder_path +} + +invariant ProjectTemplatesDirectoryReservedForUserTemplates { + -- The project templates directory stores only user-managed templates. + -- Creating a project does not populate public_dir/templates with bundled defaults. +} + +invariant PublicContentLivesInProjectFolder { + -- Every file the user owns or that must be served by the webserver lives + -- under public_dir (= the project folder containing meta/project.json): + -- posts, media + thumbnails, templates, scripts, the meta/ JSON files, + -- tags.json, menu.opml, and the generated html/ output. None of this + -- content is ever written to private_dir or into the application repo. +} + +invariant PrivateArtifactsLiveInOsAppDir { + -- App-internal, machine-specific, regenerable artifacts — the SQLite + -- database, the per-project embeddings index and its sidecar, the + -- model cache, the project registry, and window/UI state — live ONLY + -- under private_dir (the OS per-user app-data directory). They are never + -- written into a project folder (public_dir) or into the application repo. + -- A regenerable artifact (e.g. the embeddings index) may be rebuilt from + -- the database when absent. +} + +invariant DataPathNotPersistedInProjectJson { + -- meta/project.json never stores its own path or data_path. A project's + -- location is determined solely by where its meta/project.json file sits, + -- so the folder can be moved or renamed without editing any file. The app + -- remembers the current location as a machine-local pointer in private_dir. } rule SetActiveProject { diff --git a/specs/publishing.allium b/specs/publishing.allium index 8335c8c..a618b06 100644 --- a/specs/publishing.allium +++ b/specs/publishing.allium @@ -5,12 +5,19 @@ use "./metadata.allium" as meta +enum PublishJobStatus { + pending + running + completed + failed +} + entity PublishJob { ssh_host: String ssh_user: String ssh_remote_path: String ssh_mode: scp | rsync - status: pending | running | completed | failed + status: PublishJobStatus transitions status { pending -> running @@ -76,6 +83,7 @@ rule UploadSite { rule StartPublishJob { when: PublishJobStarted(project, job, credentials) + requires: job.status = pending ensures: job.status = running ensures: UploadTargetStarted(job, html, "html/", credentials.ssh_remote_path, credentials) ensures: UploadTargetStarted(job, thumbnails, "thumbnails/", credentials.ssh_remote_path + "/thumbnails", credentials) diff --git a/specs/rendering.allium b/specs/rendering.allium new file mode 100644 index 0000000..8754a22 --- /dev/null +++ b/specs/rendering.allium @@ -0,0 +1,219 @@ +-- allium: 1 +-- bDS Rendering Subsystem +-- Scope: core — template rendering shared by preview and generation +-- Distilled from: lib/bds/rendering/{filters,labels,links_and_languages, +-- metadata,post_rendering,list_archive}.ex + +-- The rendering subsystem turns a post/list/not-found record plus project +-- metadata into the assigns consumed by a Liquid template. It is shared by the +-- preview server (on-demand) and static site generation (published .md files). +-- Rendering language is the CONTENT language (post/project), never the UI locale. + +use "./template.allium" as template +use "./template_context.allium" as template_context +use "./i18n.allium" as i18n +use "./post.allium" as post + +-- ─── Custom Liquid filters ─────────────────────────────────── + +-- The three custom filters available in the Liquid subset (see +-- template.allium LiquidFilterSubset). Applied during template rendering. + +surface RenderFilterSurface { + facing _: TemplateRenderer + + provides: + I18nFilterApplied(value, language) + MarkdownFilterApplied(value, post_id, language) + SlugifyFilterApplied(value) +} + +rule I18nFilter { + when: I18nFilterApplied(value, language) + -- {{ "Key" | i18n }} -> localized "render" domain string for `language`. + -- Empty/blank input returns empty string. + ensures: result = lgettext(language, "render", trim(value)) +} + +rule SlugifyFilter { + when: SlugifyFilterApplied(value) + -- {{ title | slugify }} -> URL-friendly slug (same Slug.slugify as posts). + ensures: result = slugify(value) +} + +rule MarkdownFilter { + when: MarkdownFilterApplied(value, post_id, language) + -- {{ body | markdown }} pipeline, in order: + -- 1. Expand built-in [[...]] macros (see ExpandBuiltinMacros) + -- 2. Convert markdown to HTML (Earmark; errors degrade to partial HTML) + -- 3. Rewrite href/src URLs to canonical post/media paths (see RewriteUrls) + ensures: result = rewritten_html +} + +-- ─── Built-in macros ───────────────────────────────────────── + +-- Macros use double-bracket syntax [[name param="value" ...]], expanded in +-- markdown before HTML conversion. NOT Liquid tags. Each renders a bundled +-- macro template (macros/{name}) in an isolated Liquid subscope. + +value BuiltinMacro { + name: String -- youtube | vimeo | gallery | photo_archive | tag_cloud + params: Map +} + +surface BuiltinMacroSurface { + context macro: BuiltinMacro + + exposes: + macro.name + macro.params +} + +rule ExpandBuiltinMacros { + when: MacroEncountered(macro, language, post_id) + -- Project-scoped macros (photo_archive, tag_cloud) resolve the owning + -- project id from the render context (carried in the Liquid context's + -- private data), so they populate correctly even when content is + -- pre-rendered outside a full page context. + -- Unknown macro names are left verbatim in the source. + if macro.name = "youtube" or macro.name = "vimeo": + -- Embeds video by `id`; localized default title when title param absent. + ensures: MacroTemplateRendered(format("macros/{name}", name: macro.name)) + if macro.name = "gallery": + -- Image gallery from the post's linked image media (PostMedia links, + -- restored from media sidecars on rebuild). Linking on import is + -- independent of AI enrichment. Ordered newest-first (created_at desc); + -- lightbox group is "gallery-{post_id}"; title is caption|title|name. + -- `columns` clamped to 1..6 (default 3). Empty when no post_id/images. + ensures: MacroTemplateRendered("macros/gallery") + if macro.name = "photo_archive": + -- Month-grouped image archive for the project (optional year/month filter). + ensures: MacroTemplateRendered("macros/photo-archive") + if macro.name = "tag_cloud": + -- Weighted tag cloud from PUBLISHED post tag counts; per-tag colours + -- from Tag rows. Words carry {text, size, count, url}; the JSON is + -- HTML-escaped into the data-tag-cloud-words attribute so the client + -- runtime can JSON.parse it. `orientation` aliases normalize to + -- horizontal | mixed-hv | mixed-diagonal; width (320..1600, def 900) + -- and height (180..900, def 420) clamp to defaults when out of range. + ensures: MacroTemplateRendered("macros/tag-cloud") +} + +invariant MacroIsolation { + -- Macro templates render in an isolated Liquid subscope so macro-local + -- assigns never leak into the surrounding template context. +} + +-- ─── URL rewriting ─────────────────────────────────────────── + +rule RewriteUrls { + when: RenderedHtmlProduced(html, canonical_post_paths, canonical_media_paths) + -- Rewrites href= and src= attribute values in rendered HTML. + -- External/special URLs (scheme:, //, #) are left untouched. + -- Internal post references (/post/{slug}, /posts/{slug}, dated paths) map to + -- the post's canonical dated path; query/fragment suffixes are preserved. + -- Internal /media/YYYY/MM/{file} references map to canonical media paths. + ensures: AttributesRewritten(html) +} + +-- ─── Links and languages ───────────────────────────────────── + +value LinkContext { + href: String + title: String + display_slug: String + language: String +} + +rule ResolveLanguagePrefix { + when: LanguagePrefixRequested(language, main_language) + -- "" for the main language (and nil/blank); "/{language}" otherwise. + if language = main_language: + ensures: prefix = "" + else: + ensures: prefix = format("/{lang}", lang: language) +} + +rule CollectLinkContexts { + when: LinkContextsRequested(project, post_id, direction) + -- direction = incoming (backlinks) | outgoing. + -- One LinkContext per linked post that still exists; missing targets dropped. + -- href = canonical post path, language normalized to main when unset. + ensures: List +} + +-- ─── Render labels ─────────────────────────────────────────── + +value RenderLabels { + -- Localized strings for rendered/preview output, resolved in the "render" + -- gettext domain for the CONTENT language (not the UI locale). Includes + -- taxonomy, backlinks, archive, pagination, calendar, search, not-found, + -- and macro fallback labels. Month names resolved 1..12 per language. +} + +invariant LabelsUseContentLanguage { + -- RenderLabels and the i18n filter resolve against the content/render + -- language, consistent with i18n.allium's split-localization rule. +} + +-- ─── Post render assigns ───────────────────────────────────── + +-- The full assigns map for a single post template, assembled from the post +-- record (Post or PostTranslation) and project metadata. + +value PostRenderAssigns { + language: String + language_prefix: String + page_title: String? + pico_stylesheet_href: String + blog_languages: List + alternate_links: List -- hreflang alternates for translations + menu_items: List + post_categories: List + post_tags: List + tag_color_by_name: Map + backlinks: List + canonical_post_path_by_slug: Map + canonical_media_path_by_source_path: Map + post_data_json_by_id: String -- PostData JSON for client widgets + post: template_context/PostContext -- includes incoming/outgoing links + labels: RenderLabels + calendar_initial_year: Integer? + calendar_initial_month: Integer? +} + +value AlternateLink { + language: String + href: String +} + +rule BuildPostAssigns { + when: PostAssignsRequested(project, assigns) + -- Loads the post/translation record, renders its markdown body (macros + + -- HTML + URL rewrite), collects incoming/outgoing links, and resolves all + -- metadata-derived assigns (menu, languages, alternates, tag colours, + -- calendar bounds, labels) for the post's language. + ensures: PostRenderAssigns +} + +rule BuildNotFoundAssigns { + when: NotFoundAssignsRequested(project, assigns) + -- Assigns for the 404 page: page_title defaults to "404", no alternates, + -- but shares language/menu/blog_languages/stylesheet/labels with posts. + -- Consumed by the not-found template (see generation.allium 404.html). + ensures: NotFoundRenderAssigns +} + +rule BuildListAssigns { + when: ListAssignsRequested(project, assigns) + -- Assigns for list/archive pages (home, category, tag, date archives): + -- paginated post list plus shared metadata-derived assigns. + ensures: ListRenderAssigns +} + +invariant SharedRenderPathForPreviewAndGeneration { + -- Preview and generation produce identical HTML for the same input because + -- both build assigns through this subsystem and render via the same Liquid + -- subset. They differ only in content SOURCE (see preview.allium + -- PreviewDraftOverlay and generation.allium GenerationPublishedOnly). +} diff --git a/specs/schema.allium b/specs/schema.allium index 45f0761..7382ef3 100644 --- a/specs/schema.allium +++ b/specs/schema.allium @@ -1,11 +1,31 @@ -- allium: 1 --- bDS SQLite Database Schema +-- bDS Persistence Data Contract -- Scope: core (Wave 1 — exact compatibility contract) --- Distilled from: ../bDS/src/main/database/schema.ts +-- Distilled from: ../bDS2/src/main/database/schema.ts -- --- This document specifies the exact SQLite schema that the Rust --- implementation must be able to read and write. It is the ground truth --- for database compatibility. +-- This document specifies the persisted data model the rewrite must be able +-- to read and write. It is the ground truth for storage compatibility. + +enum PostStatus { + draft + published + archived +} + +enum PostTranslationStatus { + draft + published +} + +enum TemplateStatus { + draft + published +} + +enum ScriptStatus { + draft + published +} -- ============================================================================ -- CORE ENTITIES @@ -29,7 +49,7 @@ entity Post { slug: String -- URL-friendly identifier excerpt: String? -- Optional summary content: String? -- Draft body (null when published) - status: draft | published | archived + status: PostStatus author: String? -- Author name created_at: Timestamp updated_at: Timestamp @@ -58,7 +78,7 @@ entity PostTranslation { title: String excerpt: String? content: String? -- Draft body (null when published) - status: draft | published + status: PostTranslationStatus created_at: Timestamp updated_at: Timestamp published_at: Timestamp? @@ -119,7 +139,7 @@ entity Template { enabled: Boolean version: Integer -- Incremented on each update file_path: String -- templates/{slug}.liquid - status: draft | published + status: TemplateStatus content: String? -- Draft body (null when published) created_at: Timestamp updated_at: Timestamp @@ -131,11 +151,11 @@ entity Script { slug: String -- URL-safe identifier title: String kind: macro | utility | transform - entrypoint: String -- Default: "render" for macros + entrypoint: String -- Default: "render" for macros, "main" otherwise enabled: Boolean version: Integer -- Incremented on each update - file_path: String -- scripts/{slug}.lua - status: draft | published + file_path: String -- scripts/{slug}.{extension} + status: ScriptStatus content: String? -- Draft body (null when published) created_at: Timestamp updated_at: Timestamp @@ -184,8 +204,7 @@ entity GeneratedFileHash { -- ============================================================================ entity PostSearchIndex { - -- SQLite FTS5 virtual table, not a real entity - -- Created via: CREATE VIRTUAL TABLE posts_fts USING fts5(...) + -- Full-text search index projection, not a user-authored entity -- Indexed fields: title, excerpt, content, tags, categories -- Plus all translation titles, excerpts, and content post: Post @@ -193,8 +212,7 @@ entity PostSearchIndex { } entity MediaSearchIndex { - -- SQLite FTS5 virtual table - -- Created via: CREATE VIRTUAL TABLE media_fts USING fts5(...) + -- Full-text search index projection -- Indexed fields: title, alt, caption, original_name, tags -- Plus all translation titles, alts, and captions media: Media @@ -221,16 +239,18 @@ entity ChatMessage { content: String? tool_call_id: String? -- For tool responses tool_calls: String? -- JSON array of tool calls + cache_read_tokens: Integer? + cache_write_tokens: Integer? created_at: Timestamp } entity AiProvider { -- Provider catalog, populated from upstream model registry. - -- Read-only in Rust core; TypeScript app manages inserts. + -- Managed by the application and treated as read-only during normal use. id: String -- PRIMARY KEY name: String env: String? -- Environment variable for API key - npm: String? -- NPM package reference (legacy) + package_ref: String? -- Legacy package reference api: String? -- Base API URL doc: String? -- Documentation URL updated_at: Timestamp @@ -261,7 +281,6 @@ entity AiModel { max_output_tokens: Integer interleaved: String? -- interleaved capability descriptor status: String? -- active | deprecated | preview - provider_npm: String? -- provider-specific NPM reference (legacy) updated_at: Timestamp } @@ -270,7 +289,7 @@ entity AiModelModality { provider: AiProvider model_id: String direction: String -- "input" | "output" - modality: String -- "text" | "image" | "audio" | "video" + modality: String -- "text" | "image" | "audio" | "file" | "tool" } entity AiCatalogMeta { @@ -287,7 +306,7 @@ entity EmbeddingKey { post_id: String project_id: String content_hash: String -- SHA-256 of title+content - vector: String -- Base64-encoded Float32Array bytes (1536 bytes for 384-dim) + vector: String -- Encoded vector payload (1536 bytes for 384-dim) } entity DismissedDuplicatePair { @@ -327,6 +346,305 @@ entity DbNotification { created_at: Timestamp } +surface ProjectRecordSurface { + context project: Project + + exposes: + project.id + project.name + project.slug + project.description when project.description != null + project.data_path when project.data_path != null + project.created_at + project.updated_at + project.is_active +} + +surface PostTranslationRecordSurface { + context translation: PostTranslation + + exposes: + translation.id + translation.project_id + translation.translation_for + translation.language + translation.title + translation.excerpt when translation.excerpt != null + translation.content when translation.content != null + translation.status + translation.created_at + translation.updated_at + translation.published_at when translation.published_at != null + translation.file_path + translation.checksum when translation.checksum != null +} + +surface MediaTranslationRecordSurface { + context translation: MediaTranslation + + exposes: + translation.id + translation.project_id + translation.translation_for + translation.language + translation.title when translation.title != null + translation.alt when translation.alt != null + translation.caption when translation.caption != null + translation.created_at + translation.updated_at +} + +surface TagRecordSurface { + context tag: Tag + + exposes: + tag.id + tag.project_id + tag.name + tag.color when tag.color != null + tag.post_template_slug when tag.post_template_slug != null + tag.created_at + tag.updated_at +} + +surface TemplateRecordSurface { + context template: Template + + exposes: + template.id + template.project_id + template.slug + template.title + template.kind + template.enabled + template.version + template.file_path + template.status + template.content when template.content != null + template.created_at + template.updated_at +} + +surface ScriptRecordSurface { + context script: Script + + exposes: + script.id + script.project_id + script.slug + script.title + script.kind + script.entrypoint + script.enabled + script.version + script.file_path + script.status + script.content when script.content != null + script.created_at + script.updated_at +} + +surface PostLinkRecordSurface { + context link: PostLink + + exposes: + link.id + link.source_post_id + link.target_post_id + link.link_text when link.link_text != null + link.created_at +} + +surface PostMediaLinkRecordSurface { + context link: PostMediaLink + + exposes: + link.id + link.project_id + link.post_id + link.media_id + link.sort_order + link.created_at +} + +surface SettingRecordSurface { + context setting: Setting + + exposes: + setting.key + setting.value + setting.updated_at +} + +surface GeneratedFileHashRecordSurface { + context record: GeneratedFileHash + + exposes: + record.project_id + record.relative_path + record.content_hash + record.updated_at +} + +surface PostSearchIndexRecordSurface { + context record: PostSearchIndex + + exposes: + record.post + record.stemmed_content +} + +surface MediaSearchIndexRecordSurface { + context record: MediaSearchIndex + + exposes: + record.media + record.stemmed_content +} + +surface ChatConversationRecordSurface { + context conversation: ChatConversation + + exposes: + conversation.id + conversation.title + conversation.model when conversation.model != null + conversation.copilot_session_id when conversation.copilot_session_id != null + conversation.created_at + conversation.updated_at +} + +surface ChatMessageRecordSurface { + context message: ChatMessage + + exposes: + message.id + message.conversation_id + message.role + message.content when message.content != null + message.tool_call_id when message.tool_call_id != null + message.tool_calls when message.tool_calls != null + message.cache_read_tokens when message.cache_read_tokens != null + message.cache_write_tokens when message.cache_write_tokens != null + message.created_at +} + +surface AiModelRecordSurface { + context model: AiModel + + exposes: + model.provider + model.model_id + model.name + model.family when model.family != null + model.attachment + model.reasoning + model.tool_call + model.structured_output + model.temperature + model.knowledge when model.knowledge != null + model.release_date when model.release_date != null + model.last_updated_date when model.last_updated_date != null + model.open_weights + model.input_price when model.input_price != null + model.output_price when model.output_price != null + model.cache_read_price when model.cache_read_price != null + model.cache_write_price when model.cache_write_price != null + model.context_window + model.max_input_tokens + model.max_output_tokens + model.interleaved when model.interleaved != null + model.status when model.status != null + model.updated_at +} + +surface AiModelModalityRecordSurface { + context modality: AiModelModality + + exposes: + modality.provider + modality.model_id + modality.direction + modality.modality +} + +surface AiCatalogMetaRecordSurface { + context meta: AiCatalogMeta + + exposes: + meta.key + meta.value +} + +surface EmbeddingKeyRecordSurface { + context key: EmbeddingKey + + exposes: + key.label + key.post_id + key.project_id + key.content_hash + key.vector +} + +surface DismissedDuplicatePairRecordSurface { + context pair: DismissedDuplicatePair + + exposes: + pair.id + pair.project_id + pair.post_id_a + pair.post_id_b + pair.dismissed_at +} + +surface ImportDefinitionRecordSurface { + context definition: ImportDefinition + + exposes: + definition.id + definition.project_id + definition.name + definition.wxr_file_path when definition.wxr_file_path != null + definition.uploads_folder_path when definition.uploads_folder_path != null + definition.last_analysis_result when definition.last_analysis_result != null + definition.created_at + definition.updated_at +} + +surface DbNotificationRecordSurface { + context notification: DbNotification + + exposes: + notification.id + notification.entity_type + notification.entity_id + notification.action + notification.from_cli + notification.seen_at when notification.seen_at != null + notification.created_at +} + +surface Fts5PostSchemaSurface { + context schema: Fts5PostSchema + + exposes: + schema.fields + schema.stemmer_languages +} + +surface Fts5MediaSchemaSurface { + context schema: Fts5MediaSchema + + exposes: + schema.fields + schema.stemmer_languages +} + +surface MigrationVersionSurface { + context _: MigrationVersion +} + -- ============================================================================ -- SCHEMA CONSTRAINTS AND INDEXES -- ============================================================================ diff --git a/specs/script.allium b/specs/script.allium index 078d379..4ce208f 100644 --- a/specs/script.allium +++ b/specs/script.allium @@ -1,16 +1,34 @@ -- allium: 1 -- bDS Scripting System --- Scope: core (Wave 6 — Lua runtime and scripting docs) +-- Scope: core (Wave 6 — scripting behaviour and file contracts) -- Distilled from: src/main/engine/ScriptEngine.ts, schema.ts --- Scripting runtime is Lua (mlua crate). Behavioural contract preserved. +-- Lua is the normative scripting language for user-authored scripts in the +-- rewrite. The concrete embedding strategy remains an implementation choice; +-- only the behavioural contract is normative here. + +config { + script_extension: String = "lua" + macro_timeout: Duration = 10.seconds + transform_max_toasts_per_script: Integer = 5 + transform_max_toasts_total: Integer = 20 + transform_max_toast_length: Integer = 300 + blogmark_max_title_length: Integer = 200 + blogmark_max_url_length: Integer = 2048 +} + +enum ScriptStatus { + draft + published +} entity Script { + project_id: String slug: String title: String kind: macro | utility | transform - entrypoint: String -- default: "render" for macros + entrypoint: String -- named Lua function used as the script entrypoint enabled: Boolean - status: draft | published + status: ScriptStatus content: String? version: Integer file_path: String @@ -48,8 +66,9 @@ surface ScriptManagementSurface { facing _: ScriptOperator provides: - CreateScriptRequested(title, kind, content, entrypoint) - CreateAndPublishScriptRequested(title, kind, content, entrypoint) + CreateScriptRequested(project, title, kind, content, entrypoint) + CreateAndPublishScriptRequested(project, title, kind, content, entrypoint) + UpdateScriptRequested(script, changes) PublishScriptRequested(script) DeleteScriptRequested(script) RunUtilityRequested(script) @@ -58,60 +77,124 @@ surface ScriptManagementSurface { RebuildScriptsFromFilesRequested(project) } +surface ScriptRuntimeSurface { + facing _: ScriptRuntime + + provides: + ValidateScript(source) + ExecuteScriptRequested(script, entrypoint, args, progress_sink) + + @guarantee SandboxedExecution + -- User-authored Lua executes from a sandboxed runtime state. + -- Filesystem mutation, process control, package loading, and other + -- unrestricted host capabilities are unavailable unless explicitly + -- re-exposed by the host application. + + @guarantee ExplicitHostCapabilities + -- Host-provided functions are exposed only through an explicit bds.* + -- capability table, never through ambient global access. + + @guarantee MacroTimeout + -- Macro execution has a short timeout budget of config.macro_timeout. + + @guarantee ManagedBatchExecution + -- Utility and transform scripts execute as managed jobs. + -- The contract does not define a fixed wall-clock limit for those + -- jobs because batch work can legitimately scale with project size. + -- Progress reporting, operator cancellation, and host orchestration + -- govern their lifecycle instead of a fixed timeout. + + @guarantee ProgressFeedback + -- Long-running utility and transform scripts may emit progress updates + -- through explicit host APIs during execution. + -- Progress reporting is cooperative and flows through the supplied + -- progress sink rather than ambient global side effects. + + @guarantee ScriptOutputRouting + -- print(…) calls and the explicit bds.app.log(…) API append lines to a + -- host-routed script output stream: the desktop app's Output panel, + -- stdout in the CLI, and the host logger when no sink is attached — + -- never the ambient console. + + @guarantee BatchCancellation + -- Managed utility and transform jobs can be cancelled by the host + -- operator boundary. +} + invariant UniqueScriptSlug { + -- Slug uniqueness is scoped per project, not globally. for a in Scripts: for b in Scripts: - a != b implies a.slug != b.slug + a != b and a.project_id = b.project_id implies a.slug != b.slug } invariant ScriptFileLayout { for s in Scripts where file_path != "": - s.file_path = format("scripts/{slug}.lua", slug: s.slug) + s.file_path = format("scripts/{slug}.{extension}", slug: s.slug, extension: config.script_extension) } --- Script files use .lua with standard --- YAML frontmatter +-- Script files use standard --- YAML frontmatter rule CreateScript { - when: CreateScriptRequested(title, kind, content, entrypoint) + when: CreateScriptRequested(project, title, kind, content, entrypoint) let slug = slugify(title) -- Creates a draft script: content stored in DB, no file written yet - ensures: Script.created( - slug: slug, - title: title, - kind: kind, - content: content, - entrypoint: entrypoint ?? "render", - status: draft, - enabled: true, - version: 1, - file_path: "" - ) + ensures: + let new_script = Script.created( + project_id: project.id, + slug: slug, + title: title, + kind: kind, + content: content, + entrypoint: entrypoint ?? if kind = macro: "render" else: "main", + status: draft, + enabled: true, + version: 1, + file_path: "" + ) + new_script.status = draft +} + +rule UpdateScript { + when: UpdateScriptRequested(script, changes) + ensures: ScriptFieldsUpdated(script, changes) + ensures: script.updated_at = now + ensures: script.version = script.version + 1 +} + +rule ReopenPublishedScript { + when: UpdateScriptRequested(script, changes) + requires: script.status = published + requires: script_changes_affect_published_output(changes) + ensures: script.status = draft } rule CreateAndPublishScript { -- Alternative creation path: create + immediately publish (file written) - -- TypeScript: createScript() does this in one step - when: CreateAndPublishScriptRequested(title, kind, content, entrypoint) + -- Some implementations may expose this as a single user action + when: CreateAndPublishScriptRequested(project, title, kind, content, entrypoint) let slug = slugify(title) requires: ValidateScript(content) = valid ensures: let new_script = Script.created( + project_id: project.id, slug: slug, title: title, kind: kind, content: null, - entrypoint: entrypoint ?? "render", + entrypoint: entrypoint ?? if kind = macro: "render" else: "main", status: published, enabled: true, version: 1, - file_path: format("scripts/{slug}.lua", slug: slug) + file_path: format("scripts/{slug}.{extension}", slug: slug, extension: config.script_extension) ) ScriptFileWritten(new_script) } rule PublishScript { when: PublishScriptRequested(script) + requires: script.status = draft requires: ValidateScript(script.content) = valid - -- AST parsing must succeed + -- Lua parsing must succeed before a script can be published ensures: script.status = published ensures: ScriptFileWritten(script) ensures: script.content = null @@ -129,9 +212,17 @@ rule ExecuteMacro { when: MacroExpansionRequested(script, template_context) requires: script.kind = macro requires: script.enabled = true + requires: script.entrypoint != "" -- Macro scripts are invoked during template rendering -- via [[slug param1=value1 param2=value2]] syntax in post content - -- They receive named parameters and the template context, return HTML + -- Unknown macro names are resolved against enabled macro scripts by slug. + -- They receive named parameters plus template_context.env fields that + -- include isPreview, mainLanguage, languagePrefix, hook, source.kind, + -- and translations. + -- They return HTML and run sequentially with config.macro_timeout per + -- invocation. + -- Macro failures degrade to empty output for that invocation and do not + -- abort rendering of the surrounding page. ensures: MacroOutputProduced(script, html_output) } @@ -139,7 +230,11 @@ rule ExecuteUtility { when: RunUtilityRequested(script) requires: script.kind = utility requires: script.enabled = true - -- Runs on-demand from the UI, produces stdout output + requires: script.entrypoint != "" + -- Utility scripts commonly perform long-running data manipulation work. + -- They are manually started by an operator action, run as managed jobs, + -- may issue host-backed API calls, may emit progress during execution, + -- and may be cancelled by the operator. ensures: UtilityOutputProduced(script, stdout) } @@ -147,19 +242,76 @@ rule ExecuteTransform { when: BlogmarkReceived(data) -- Transform scripts run sequentially on blogmark deep link data -- Input: title, content, tags, categories, source url - -- Each transform can modify the data before post creation - let transforms = Scripts where kind = transform and enabled = true - for t in ordered_by(transforms, s => s.slug): + -- Each transform can modify the data before post creation. + -- Execution uses the same managed job host API contract as other batch + -- scripts and may report progress while mass-processing remote or local + -- content. + let transforms = Scripts where project_id = data.project_id and kind = transform and enabled = true + for t in ordered_by(transforms, s => s.updated_at, s => s.slug, s => s.id): + requires: t.entrypoint != "" ensures: TransformApplied(t, data) + @guarantee BlogmarkInputHardening + -- The deep link is sanitized before transforms run. Title and url have + -- control characters stripped and are trimmed. The url is kept only + -- when it is http(s) with a host, after removing any credentials and + -- fragment, and at most config.blogmark_max_url_length characters; + -- otherwise it is dropped so it can never reach the post body. The + -- title is capped at config.blogmark_max_title_length and falls back to + -- the url host when blank. + + @guarantee BlogmarkDefaultBody + -- When the deep link supplies no content, the post body defaults to a + -- markdown link [title](url) to the bookmarked page, with the title + -- escaped, so transforms operate on the same candidate whether or not + -- an explicit body was provided. Explicit content always wins. + + @guarantee BlogmarkProjectTargeting + -- A link's own project_id determines the target project. When it names a + -- project that exists and differs from the active one, the shell + -- switches to it before importing so the new post is shown in context. + -- When it names a project that does not exist in this install (e.g. a + -- bookmarklet copied from another instance) the import is refused with + -- an error — the post is never redirected into a different project. + -- With no link project_id, the active project is used; with no link + -- project_id and no active project the import is refused with a warning. + + @guarantee BlogmarkColdStartDelivery + -- A deep link that arrives before the shell is ready (e.g. the + -- bookmarklet launching the app) is queued and replayed only after the + -- client has finished initialising and rehydrating its stored + -- workbench session, so the post is created and the editor it opens is + -- not overwritten by the session restore. A session restore arriving + -- after a deep link (e.g. when the link switched projects) preserves + -- the newly opened editor tab as the active tab. + + @guarantee TransformTrigger + -- Transform scripts are triggered automatically by blogmark import. + -- Each script receives the current post candidate plus a context with + -- source='blogmark' and the originating URL. + + @guarantee TransformPipelineContinuation + -- Transform errors are captured per script and do not roll back the + -- last valid post state produced by earlier transforms. + -- The pipeline continues with subsequent enabled transforms. + + @guarantee TransformToastBudget + -- Transform scripts may emit toast feedback. + -- At most config.transform_max_toasts_per_script toasts are accepted + -- from any one transform, with a total budget of + -- config.transform_max_toasts_total across the pipeline. + -- Individual toast messages are truncated to + -- config.transform_max_toast_length characters. + @guidance - -- bds://new-post deep links from browser bookmarks - -- Max 5 toast notifications per script, 20 total + -- bds2://new-post deep links from browser bookmarks + -- (bds2:// scheme avoids clashing with the legacy app's bds:// scheme) + -- Ordering is deterministic: updated_at, then slug, then id } rule RebuildScriptsFromFiles { when: RebuildScriptsFromFilesRequested(project) - for file in scan_directory(project.effective_data_dir + "/scripts", "*.lua"): + for file in scan_directory(project.public_dir + "/scripts", "*." + config.script_extension): let parsed = parse_script_file(file) ensures: Script.created(parsed) } diff --git a/specs/search.allium b/specs/search.allium index 88ba3b9..81da2dd 100644 --- a/specs/search.allium +++ b/specs/search.allium @@ -1,6 +1,6 @@ -- allium: 1 -- bDS Full-Text Search --- Scope: core (Wave 1 — FTS5 in-app search with Snowball stemmers) +-- Scope: core (Wave 1 — in-app full-text search with Snowball stemmers) -- Distilled from: src/main/engine/PostEngine.ts (FTS methods), -- MediaEngine.ts (FTS methods), stemmer.ts @@ -12,7 +12,7 @@ surface SearchControlSurface { provides: SearchPostsRequested(query, filters) - SearchMediaRequested(query) + SearchMediaRequested(query, filters) } surface SearchIndexRuntimeSurface { @@ -24,8 +24,9 @@ surface SearchIndexRuntimeSurface { } value StemmerLanguage { - -- Snowball stemmers for 24 languages - -- ISO 639-1 to Snowball mapping + -- Snowball stemmers via library (Stemex) + -- Languages with a Snowball algorithm get real stemming; + -- others pass through unstemmed -- Applied to both indexing and query processing code: String } @@ -38,19 +39,29 @@ surface StemmerLanguageSurface { } entity PostSearchIndex { - -- SQLite FTS5 virtual table - -- Indexed fields: title, excerpt, content, tags, categories - -- Plus all translation titles, excerpts, and content + -- FTS5 virtual table with per-field stemmed columns + -- Each field is stemmed independently; translations are + -- stemmed with their own language stemmer and appended + -- to the corresponding field post: post/Post - stemmed_content: String + title: String + excerpt: String + content: String + tags: String + categories: String } entity MediaSearchIndex { - -- SQLite FTS5 virtual table - -- Indexed fields: title, alt, caption, original_name, tags - -- Plus all translation titles, alts, and captions + -- FTS5 virtual table with per-field stemmed columns + -- Each field is stemmed independently; translations are + -- stemmed with their own language stemmer and appended + -- to the corresponding field media: media/Media - stemmed_content: String + title: String + alt: String + caption: String + original_name: String + tags: String } invariant CrossLanguageStemming { @@ -77,42 +88,80 @@ rule SearchPosts { } rule SearchMedia { - when: SearchMediaRequested(query) + when: SearchMediaRequested(query, filters) + -- Full-text search with optional filters: + -- language, tags, year, month, date range (from/to) + -- Returns paginated results with total count let stemmed_query = stem(query, detect_language(query)) - let matched = search_fts(MediaSearchIndex, stemmed_query) + let matched = search_fts(MediaSearchIndex, stemmed_query, filters) ensures: SearchResults( - media: matched + media: matched, + total: matched.count, + offset: filters.offset, + limit: filters.limit ) } rule IndexPost { when: SearchIndexUpdated(post) - -- Stems: title + excerpt + content + tags + categories - -- Plus all translations' title + excerpt + content - let all_text = concat_post_text(post) - -- Concatenates: post.title, post.excerpt, post.content, - -- join(post.tags, " "), join(post.categories, " "), - -- and all translations' title, excerpt, content - let index_entry = PostSearchIndex{post: post} - ensures: - if exists index_entry: - index_entry.stemmed_content = stem(all_text) - else: - PostSearchIndex.created(post: post, stemmed_content: stem(all_text)) + -- Delete-and-reinsert: no in-place update for FTS5 rows + -- Each field is stemmed per-language; translations are stemmed + -- with their own language stemmer and joined into the same field + let lang = post.language + let translations = post.translations + let title = join_stemmed( + stem(post.title, lang), + for t in translations: stem(t.title, t.language) + ) + let excerpt = join_stemmed( + stem(post.excerpt, lang), + for t in translations: stem(t.excerpt, t.language) + ) + let content = join_stemmed( + stem(post.content, lang), + for t in translations: stem(t.content, t.language) + ) + let tags = stem(join(post.tags, " "), lang) + let categories = stem(join(post.categories, " "), lang) + ensures: not exists PostSearchIndex{post: post} + ensures: PostSearchIndex.created( + post: post, + title: title, + excerpt: excerpt, + content: content, + tags: tags, + categories: categories + ) } rule IndexMedia { when: SearchIndexUpdated(media) - -- Stems: title + alt + caption + original_name + tags - -- Plus all translations' title, alt, caption - let all_text = concat_media_text(media) - -- Concatenates: media.title, media.alt, media.caption, - -- media.original_name, join(media.tags, " "), - -- and all translations' title, alt, caption - let index_entry = MediaSearchIndex{media: media} - ensures: - if exists index_entry: - index_entry.stemmed_content = stem(all_text) - else: - MediaSearchIndex.created(media: media, stemmed_content: stem(all_text)) + -- Delete-and-reinsert: no in-place update for FTS5 rows + -- Each field is stemmed per-language; translations are stemmed + -- with their own language stemmer and joined into the same field + let lang = media.language + let translations = media.translations + let title = join_stemmed( + stem(media.title, lang), + for t in translations: stem(t.title, t.language) + ) + let alt = join_stemmed( + stem(media.alt, lang), + for t in translations: stem(t.alt, t.language) + ) + let caption = join_stemmed( + stem(media.caption, lang), + for t in translations: stem(t.caption, t.language) + ) + let original_name = stem(media.original_name, lang) + let tags = stem(join(media.tags, " "), lang) + ensures: not exists MediaSearchIndex{media: media} + ensures: MediaSearchIndex.created( + media: media, + title: title, + alt: alt, + caption: caption, + original_name: original_name, + tags: tags + ) } diff --git a/specs/server.allium b/specs/server.allium new file mode 100644 index 0000000..01f5ebb --- /dev/null +++ b/specs/server.allium @@ -0,0 +1,90 @@ +-- allium: 1 +-- Headless Server Mode (issue #26, phase 1) +-- Scope: extension (client/server split — SSH transport for TUI and GUI) +-- Distilled from: lib/bds/server.ex, lib/bds/application.ex, lib/bds/tui.ex + +entity BootMode { + kind: desktop | server | tui + -- Resolved from the BDS_MODE environment variable at application start. + -- "server" is headless; "tui" is the headless server plus a TUI on the + -- launching terminal; anything else is desktop. +} + +entity SshKeyMaterial { + dir: String -- /ssh, next to the database + host_key: String -- ssh_host_rsa_key, generated on first boot + authorized_keys: String -- created empty (mode 600) on first boot + + -- Invariant: key material lives in the private app data dir, + -- never in the repo or an application priv dir. +} + +surface ServerRuntimeSurface { + facing _: ServerRuntime + + provides: + ApplicationStarted(mode) + SshClientConnected() + GuiConnectRequested() +} + +rule ModeResolution { + when: ApplicationStarted(mode) + -- BDS_MODE environment variable decides the mode once per boot. + -- Fallback (issue #33): resolved desktop mode boots as tui when no + -- graphical session is available, so the app always opens a UI + -- rather than crashing wx; a log line records the switch. Headless + -- means: non-Darwin unix without DISPLAY/WAYLAND_DISPLAY (plain ssh + -- included; ssh -X with a forwarded display keeps the GUI), or macOS + -- inside an ssh session (SSH_CONNECTION/SSH_CLIENT/SSH_TTY — macOS + -- never sets DISPLAY). Explicit server/tui modes and Windows are + -- never rewritten. Because the :desktop dependency boots wx in its + -- own application start, config/runtime.exs sets NO_WX=1 for every + -- non-desktop effective mode (BDS.Server.prepare_boot_env) so the + -- dependency starts inert before any BDS supervisor runs. In tui + -- mode prepare_boot_env also silences every other terminal writer — + -- the default logger handler moves to a rotating file in the private + -- app dir and Bumblebee's model-download progress bar is disabled — + -- because the TUI owns the terminal and each stray console line + -- scrolls the alternate screen up one row. Desktop/server modes and + -- SSH-served TUI sessions keep normal console logging. + ensures: BootMode.created(kind: mode) +} + +rule DesktopBoot { + when: ApplicationStarted(mode: desktop) + -- Unchanged behavior: wx window shell, loopback HTTP endpoint. + ensures: DesktopWindowStarted() +} + +rule ServerBoot { + when: ApplicationStarted(mode: server) + -- Headless: no wx children at all. Works on macOS, Windows, Linux. + ensures: LoopbackEndpointStarted() + -- HTTP endpoint stays bound to 127.0.0.1; never exposed directly + ensures: CliSyncWatcherStarted() + ensures: SshDaemonStarted() + -- ExRatatui.SSH.Daemon: public-key auth only (no passwords), + -- tcpip_tunnel_out enabled so GUI clients tunnel to the loopback + -- endpoint through the same SSH connection and the same keys + ensures: SshKeyMaterial.created() + -- host key generated and authorized_keys touched on first boot only +} + +rule GuiRemoteConnection { + when: GuiConnectRequested() + -- Desktop "Connect to Server…": ssh connect with the same public keys + -- (client identity + known_hosts in the private ssh dir), then an OTP + -- TCP/IP tunnel to the server's loopback endpoint; the webview loads + -- the local tunnel end. Disconnect returns to the local shell URL. + ensures: WebviewTunneledToServer() +} + +rule TuiSession { + when: SshClientConnected() + -- Each SSH client gets its own TUI app instance on the server; + -- only terminal cells cross the wire. + ensures: TuiAppMounted() + -- UI locale comes from the server-side UI language setting, + -- identical for every client +} diff --git a/specs/sidebar_views.allium b/specs/sidebar_views.allium index 5805063..00df891 100644 --- a/specs/sidebar_views.allium +++ b/specs/sidebar_views.allium @@ -338,7 +338,7 @@ surface ScriptListItemEntry { @guarantee CreateDefaults -- New scripts default to: kind=utility, content='print("new script")', - -- entrypoint='render', enabled=true. + -- entrypoint='main', enabled=true. @guarantee LiveRefresh -- Item list refreshes on scripts-changed event. @@ -652,6 +652,9 @@ rule ImportListClick { -- Full git interface, not the SidebarEntityList pattern. -- Three possible states: loading, not_a_repo, active_repo. +-- Backend: BDS.Git provides status, diff, commit_all, history, +-- fetch, pull, push, prune_lfs_cache, remote_state, initialize_repo. +-- The sidebar must surface these capabilities directly. -- State: not_a_repo -- Remote URL text input + "Initialize Git" button. diff --git a/specs/task.allium b/specs/task.allium index 657b902..8e75a18 100644 --- a/specs/task.allium +++ b/specs/task.allium @@ -17,6 +17,7 @@ entity Task { running -> completed running -> failed running -> cancelled + pending -> cancelled } } @@ -36,6 +37,7 @@ surface TaskRuntimeSurface { TaskWorkCompleted(task) TaskWorkFailed(task, error_message) ProgressReported(task, value, message) + FinishedTaskEvictionDue() } surface TaskSurface { @@ -54,6 +56,8 @@ surface TaskSurface { config { max_concurrent: Integer = 3 progress_throttle: Duration = 250.milliseconds + finished_task_ttl: Duration = 1.hour + recent_finished_limit: Integer = 10 } invariant MaxConcurrency { @@ -95,7 +99,7 @@ rule FailTask { rule CancelTask { when: CancelTaskRequested(task) requires: task.status = running or task.status = pending - -- AbortController-based cancellation + -- Cancellation uses a runtime-specific cancellation mechanism ensures: task.status = cancelled ensures: NextQueuedTaskStarted() } @@ -112,6 +116,23 @@ invariant ProgressThrottled { -- At most one progress event per 250ms per task } +invariant FinishedTaskRetention { + -- The status snapshot surfaces only the most recent finished tasks: + -- completed/failed/cancelled tasks beyond config.recent_finished_limit + -- (newest first) are not shown. + let finished = Tasks where status in {completed, failed, cancelled} + -- At most config.recent_finished_limit finished tasks are reported. +} + +rule EvictFinishedTasks { + when: FinishedTaskEvictionDue() + -- Periodic sweep (every config.finished_task_ttl). A finished task whose + -- finished_at is older than config.finished_task_ttl is dropped from state. + for task in Tasks where status in {completed, failed, cancelled}: + if now - task.finished_at >= config.finished_task_ttl: + ensures: not exists task +} + -- External tasks: lifecycle controlled by caller (e.g., renderer-side scripts) rule RegisterExternalTask { when: RegisterExternalTaskRequested(name) diff --git a/specs/template.allium b/specs/template.allium index c213534..fad142c 100644 --- a/specs/template.allium +++ b/specs/template.allium @@ -4,12 +4,18 @@ -- Distilled from: src/main/engine/TemplateEngine.ts, PageRenderer.ts, schema.ts, -- bundled starter templates in src/main/engine/templates/ +enum TemplateStatus { + draft + published +} + entity Template { + project_id: String slug: String title: String kind: post | list | not_found | partial enabled: Boolean - status: draft | published + status: TemplateStatus content: String? version: Integer file_path: String @@ -18,8 +24,8 @@ entity Template { -- Derived content_location: if status = published: file_path else: content - referencing_posts: Posts where template_slug = this.slug - referencing_tags: Tags where post_template_slug = this.slug + referencing_posts: Posts where template_slug = this.slug and project_id = this.project_id + referencing_tags: Tags where post_template_slug = this.slug and project_id = this.project_id transitions status { draft -> published @@ -28,11 +34,11 @@ entity Template { } surface TemplateManagementSurface { - facing operator: TemplateOperator + facing _: TemplateOperator provides: - CreateTemplateRequested(title, kind, content) - CreateAndPublishTemplateRequested(title, kind, content) + CreateTemplateRequested(project, title, kind, content) + CreateAndPublishTemplateRequested(project, title, kind, content) UpdateTemplateRequested(template, changes) PublishTemplateRequested(template) DeleteTemplateRequested(template) @@ -40,9 +46,10 @@ surface TemplateManagementSurface { } invariant UniqueTemplateSlug { + -- Slug uniqueness is scoped per project, not globally. for a in Templates: for b in Templates: - a != b implies a.slug != b.slug + a != b and a.project_id = b.project_id implies a.slug != b.slug } invariant TemplateFrontmatter { @@ -57,30 +64,56 @@ invariant TemplateFileLayout { t.file_path = format("templates/{slug}.liquid", slug: t.slug) } +invariant BundledDefaultTemplatesExistOutsideProjectData { + -- The application ships bundled default templates for: + -- single-post + -- post-list + -- not-found + -- plus supporting partials and macros. + -- These bundled templates are available for rendering even when the project + -- has no template files and no Template rows for those defaults. +} + +invariant UserTemplateDirectoryOverridesBundledDefaults { + -- When a project template file or published Template row resolves the same slug + -- as a bundled template or partial, the project-owned template wins. + -- Bundled templates are fallback roots, not copied seed data. +} + +invariant RebuildTemplatesIndexesOnlyProjectTemplates { + -- Rebuild-from-files scans only project.public_dir/templates. + -- Bundled defaults are render-time fallbacks and are not indexed into Templates + -- unless the user has created matching project files. +} + rule CreateTemplate { - when: CreateTemplateRequested(title, kind, content) + when: CreateTemplateRequested(project, title, kind, content) let slug = slugify(title) -- Creates a draft template: content stored in DB, no file written yet - ensures: Template.created( - slug: slug, - title: title, - kind: kind, - content: content, - status: draft, - enabled: true, - version: 1, - file_path: "" - ) + ensures: + let new_template = Template.created( + project_id: project.id, + slug: slug, + title: title, + kind: kind, + content: content, + status: draft, + enabled: true, + version: 1, + file_path: "" + ) + new_template.status = draft } rule CreateAndPublishTemplate { -- Alternative creation path: create + immediately publish (file written) - -- TypeScript: createTemplate() does this in one step - when: CreateAndPublishTemplateRequested(title, kind, content) + -- Some implementations may expose this as a single user action + when: CreateAndPublishTemplateRequested(project, title, kind, content) let slug = slugify(title) requires: ValidateLiquid(content) = valid ensures: let new_template = Template.created( + project_id: project.id, slug: slug, title: title, kind: kind, @@ -100,10 +133,18 @@ rule UpdateTemplate { ensures: template.version = template.version + 1 } +rule ReopenPublishedTemplate { + when: UpdateTemplateRequested(template, changes) + requires: template.status = published + requires: template_changes_affect_rendered_output(changes) + ensures: template.status = draft +} + rule PublishTemplate { when: PublishTemplateRequested(template) + requires: template.status = draft requires: ValidateLiquid(template.content) = valid - -- LiquidJS parser must accept the template + -- The template parser must accept the template ensures: template.status = published ensures: TemplateFileWritten(template) -- Writes frontmatter + liquid to templates/{slug}.liquid @@ -130,10 +171,16 @@ rule CascadeSlugUpdate { rule RebuildTemplatesFromFiles { when: RebuildTemplatesFromFilesRequested(project) - for file in scan_directory(project.effective_data_dir + "/templates", "*.liquid"): + for file in scan_directory(project.public_dir + "/templates", "*.liquid"): let parsed = parse_template_file(file) ensures: Template.created(parsed) -- or updated if slug already exists + -- Prune stale published templates: a published Template whose file_path is + -- neither among the scanned files nor present on disk is deleted, clearing + -- its references first (posts' template_slug, tags' post_template_slug). + for template in project.templates where status = published and file_path != "": + if template.file_path not in scanned_files and not file_exists(template.file_path): + ensures: not exists template } -- Exact Liquid subset required (distilled from bundled starter templates) @@ -158,11 +205,16 @@ invariant LiquidFilterSubset { -- | default: fallback_value -- | append: suffix_string -- - -- Custom filters (2): + -- Custom filters (3): -- | i18n: language — translates a key string for given language -- | markdown: post_id, post_data_json_by_id, canonical_post_path_by_slug, -- canonical_media_path_by_source_path, language, language_prefix -- — renders Markdown to HTML with link rewriting (6 arguments) + -- | slugify — slugifies a string (used for tag/category URLs) + -- + -- Enforced at publish/validation time (LiquidParser.validate); any other + -- filter is rejected even though Liquex would otherwise apply it as a + -- built-in standard filter. -- -- NOT used: date, strip_html, truncate, split, join, size (as filter), -- upcase, downcase, replace, remove, sort, map, where, first, last, @@ -177,6 +229,10 @@ invariant LiquidOperatorSubset { -- Special values: blank (nil/empty comparison) -- Property access: dot notation (object.property), .size on arrays, -- bracket notation for map lookups (map[key]) + -- + -- Enforced at publish/validation time (LiquidParser.validate); any other + -- comparison operator (!=, <, >=, <=, contains) is rejected even though + -- Liquex would otherwise evaluate it. } invariant LiquidRenderContext { diff --git a/specs/template_context.allium b/specs/template_context.allium index 6afa81a..5d0fef1 100644 --- a/specs/template_context.allium +++ b/specs/template_context.allium @@ -1,7 +1,7 @@ -- allium: 1 -- bDS Liquid Template Context Specification -- Scope: core (Wave 4 — rendering parity) --- Distilled from: ../bDS/src/main/engine/GenerationRouteRendererFactory.ts, +-- Distilled from: ../bDS2/src/main/engine/GenerationRouteRendererFactory.ts, -- PageRenderer.ts, BlogGenerationEngine.ts -- -- This document specifies the exact data structure passed to Liquid templates @@ -62,7 +62,7 @@ value RenderContext { language: String -- Current language code language_prefix: String? -- "/de" or "" depending on language html_theme_attribute: String -- Theme class for element - page_title: String -- Page title for tag + page_title: String -- Blog title (description, else name) for the <title> tag; archive-specific labels come from archive_context pico_stylesheet_href: String -- Path to Pico CSS theme blog_languages: List<BlogLanguage> alternate_links: List<AlternateLink> @@ -127,7 +127,7 @@ value PostContext { excerpt: String? author: String? language: String? - show_title: Boolean -- Always true for post pages + show_title: Boolean -- True for post pages; in list/archive pages it is false when any of the post's categories has "show title" disabled (e.g. asides) published_at: Timestamp created_at: Timestamp updated_at: Timestamp @@ -181,7 +181,7 @@ value LinkContext { value ArchiveContext { kind: category | tag | date - name: String? + name: String? -- Heading label: for categories the configured descriptive title (falls back to the category key), for tags the raw tag name month: Integer? year: Integer? day: Integer? @@ -234,26 +234,26 @@ value PaginationContext { -- ============================================================================ -- LIQUID FILTER SPECIFICATION -- Note: Allium does not have a 'filter' keyword. Filters are documented here --- for reference but are implemented in the Liquid engine, not in Allium specs. +-- for reference but are implemented in the template engine, not in Allium specs. -- ============================================================================ --- Built-in filters (from liquid-rust crate): +-- Built-in filters: -- default: {{ value | default: "fallback" }} -- escape: {{ text | escape }} -- url_encode: {{ text | url_encode }} -- append: {{ text | append: suffix }} -- --- Custom filters (must be implemented in Rust Liquid engine): +-- Custom filters: -- i18n: {{ "key" | i18n: language }} - translation lookup -- markdown: {{ content | markdown: ... }} - markdown rendering with macro expansion -- ============================================================================ -- BUILT-IN MACROS -- Note: Allium does not have a 'macro' keyword. Macros are documented here --- for reference but are implemented as Rust functions in bds-core/render. +-- for reference but are implemented by the rendering subsystem. -- ============================================================================ --- Built-in macros (implemented as native Rust functions): +-- Built-in macros: -- gallery: [[gallery images=post.linked_media columns=3]] -- youtube: [[youtube id=dQw4w9WgXcQ]] -- vimeo: [[vimeo id=123456789]] diff --git a/specs/translation.allium b/specs/translation.allium index c61273b..6f96ba3 100644 --- a/specs/translation.allium +++ b/specs/translation.allium @@ -7,6 +7,11 @@ use "./post.allium" as post use "./media.allium" as media +enum PostTranslationStatus { + draft + published +} + surface TranslationControlSurface { facing _: TranslationOperator @@ -22,6 +27,7 @@ surface TranslationRuntimeSurface { provides: PostPublished(post) PublishTranslationRequested(translation) + TranslationEdited(translation, changes) } value SupportedLanguage { @@ -42,7 +48,7 @@ entity PostTranslation { title: String excerpt: String? content: String? - status: draft | published + status: PostTranslationStatus file_path: String checksum: String? created_at: Timestamp @@ -58,6 +64,18 @@ entity PostTranslation { } } +invariant TranslationFilesCarryFullMetadata { + -- Translation markdown files include status and timestamps alongside + -- language-specific fields. This allows each translation to be rebuilt + -- independently. On rebuild, missing fields fall back to canonical post + -- values for compatibility with legacy files. + for t in PostTranslations where file_path != "": + translation_file(t).has_fields( + id, translation_for, language, title, excerpt?, + status, created_at, updated_at, published_at + ) +} + surface PostTranslationSurface { context translation: PostTranslation @@ -119,6 +137,7 @@ rule PublishPostTranslation { rule PublishTranslation { when: PublishTranslationRequested(translation) + requires: translation.status = draft ensures: translation.status = published ensures: translation.published_at = translation.published_at ?? now ensures: TranslationFileWritten(translation) @@ -126,6 +145,14 @@ rule PublishTranslation { -- Content moves to filesystem } +rule ReopenPublishedTranslation { + when: TranslationEdited(translation, changes) + requires: translation.status = published + requires: translation_edit_affects_published_content(changes) + ensures: translation.status = draft + ensures: translation.updated_at = now +} + rule DeletePostTranslation { when: DeletePostTranslationRequested(translation) ensures: not exists translation @@ -159,3 +186,131 @@ invariant FtsIncludesTranslations { for t in post.translations: includes_text(search_index(post), t.title) } + +-- =========================================================================== +-- Auto-Translation System +-- Distilled from: lib/bds/posts/auto_translation.ex +-- +-- Two entry points share one translation primitive: +-- 1. ScheduleAutoTranslation - reactive, fired only by an explicit manual +-- save (or publish) in the post editor — never by post creation or +-- auto-save. One background task per missing language produces a DRAFT +-- translation, then cascades to the post's linked media. +-- 2. FillMissingTranslations - batch maintenance action. Scans every +-- published post, AUTO-PUBLISHES the generated translations, fills linked +-- media, reports progress and returns a summary. +-- All AI work is gated by a resolvable endpoint and runs on background tasks. +-- =========================================================================== + +config { + -- Background translations use the "AI" task group named per project. + auto_translation_task_group_name: String = "AI" +} + +surface AutoTranslationControlSurface { + facing _: TranslationOperator + + provides: + -- Reactive trigger: emitted only when a post is updated with + -- auto_translate enabled (the editor's manual save/publish path); + -- post creation and auto-save never emit it. + PostSavedForAutoTranslation(post) + -- Batch trigger: "fill missing translations" maintenance action. + FillMissingTranslationsRequested(project) +} + +invariant AutoTranslationGatedByEndpoint { + -- No automatic translation runs unless an endpoint is resolvable for the + -- current mode. Airplane mode needs url+model; online additionally needs an + -- api_key. When unconfigured, scheduling is a silent no-op. + -- See ai.allium AirplaneModeGating for endpoint selection. + for post in Posts: + auto_translation_runs(post) implies endpoint_configured(post.project) +} + +invariant AutoTranslationSkipsDoNotTranslate { + -- Posts flagged do_not_translate never schedule background translation, + -- and the batch scan rejects them before computing missing languages. + for post in Posts where post.do_not_translate: + not auto_translation_runs(post) +} + +invariant AutoTranslationOnlyMissingLanguages { + -- The target set is the configured languages (main_language plus + -- blog_languages, normalized + de-duplicated) minus the post's source + -- language and any language that already has a translation. + for post in Posts: + auto_translation_targets(post) = + configured_languages(post.project) + - source_language(post) + - post.available_languages +} + +rule ScheduleAutoTranslation { + when: PostSavedForAutoTranslation(post) + requires: not post.do_not_translate + requires: endpoint_configured(post.project) + -- One background task per missing language, each producing a DRAFT + -- translation (not auto-published) followed by a media cascade. + for language in auto_translation_targets(post): + ensures: BackgroundTaskSubmitted( + group: post.project, + group_name: config.auto_translation_task_group_name) + ensures: AutoTranslatePost(post, language, auto_publish: false) + ensures: AutoTranslateMediaCascade(post, language) + + @guidance + -- Best-effort: missing metadata, unconfigured endpoint, or + -- do_not_translate all collapse to a silent success with no task. +} + +rule AutoTranslatePost { + when: AutoTranslatePost(post, language, auto_publish) + requires: trim(editor_body(post)) != "" + -- Calls the AI endpoint with the post's source language, then upserts a + -- translation marked auto_generated. Publishes only in the batch path. + ensures: + let translation = UpsertPostTranslation(post, language) + if auto_publish: + translation.status = published + else: + translation.status = draft + + @guidance + -- An empty body yields a no_content_to_translate error and no + -- translation is created. +} + +rule AutoTranslateMediaCascade { + when: AutoTranslateMediaCascade(post, language) + -- After a post translation, each linked media (ordered by sort_order) gets + -- its own background task when its source language differs from the target + -- and it lacks a translation in that language. + for m in post.linked_media: + if m.language != "" and m.language != language and not (language in m.available_languages): + ensures: BackgroundTaskSubmitted( + group: post.project, + group_name: config.auto_translation_task_group_name) + ensures: media/UpsertMediaTranslation(m, language) +} + +rule FillMissingTranslations { + when: FillMissingTranslationsRequested(project) + -- Batch maintenance. No-op (nothing_to_do) when there is at most one + -- configured language or nothing is missing. Otherwise scans published, + -- non-do_not_translate posts and their linked media. + requires: configured_languages(project).count > 1 + for post in project.posts where status = published and not do_not_translate: + for language in auto_translation_targets(post): + ensures: AutoTranslatePost(post, language, auto_publish: true) + ensures: AutoTranslateMediaCascade(post, language) + ensures: ProgressReported(project) + ensures: FillMissingTranslationsCompleted(project) + + @guidance + -- Returns a summary of counts: translated_posts, translated_media, + -- failed_count, warned_count, and nothing_to_do. nothing_to_do is true + -- when there is at most one configured language or nothing is missing. + -- Per-item failures increment failed_count and never abort the batch. + -- Progress runs through scanning (0.0-0.15) then per-item (0.15-1.0). +} diff --git a/specs/tui.allium b/specs/tui.allium new file mode 100644 index 0000000..a6549aa --- /dev/null +++ b/specs/tui.allium @@ -0,0 +1,211 @@ +-- allium: 1 +-- Terminal UI (issue #26, phase 4) +-- Scope: extension (second renderer over the shared UI core) +-- Distilled from: lib/bds/tui.ex, lib/bds/ui/sidebar.ex, lib/bds/ui/post_editor/*.ex + +entity TuiState { + view: posts | media | templates | scripts | tags | settings | git + focus: sidebar | editor + selected_index: Integer + editing_post: Boolean + -- The editor drives the same BDS.UI.PostEditor workflow as the GUI: + -- canonical-language edits update the post, other languages update + -- translations; publish routes through the same publishing pipeline. +} + +surface TuiSurface { + facing _: TuiRuntime + + provides: + TuiKeyPressed(code, modifiers) + TuiEventReceived(entity, entity_id, action) + TuiSettingsChanged(key) + ShellTaskCompleted(route) +} + +rule SidebarNavigation { + when: TuiKeyPressed(code: "up" | "down" | "j" | "k") + -- Selection moves over the flattened sidebar items and skips + -- section headers; data comes from BDS.UI.Sidebar.view, identical + -- to the GUI sidebar content. + ensures: TuiState.selected_index.updated() +} + +rule OpenEntry { + when: TuiKeyPressed(code: "enter") + -- Posts open in the editor by default (title + textarea seeded from + -- the persisted form; syntax highlighting and word wrap on; ctrl+e + -- switches to the rendered preview). New empty posts created with + -- "n" open in the editor as well. Images open in the terminal image + -- preview. Other entities report that terminal editing is not + -- available yet. + ensures: TuiState.editing_post.updated() +} + +rule SaveAndPublish { + when: TuiKeyPressed(code: "s" | "p", modifiers: "ctrl") + -- ctrl+s saves the draft, ctrl+p saves and publishes — both through + -- BDS.UI.PostEditor.Persistence, so files, embeddings, search, and + -- the event bus behave exactly as a GUI save. + ensures: PostPersisted() +} + +rule EditorMode { + when: TuiKeyPressed(code: "e", modifiers: "ctrl") + -- ctrl+e toggles between the syntax-highlighted editor (the default + -- mode when opening a post) and the read-only rendered-Markdown + -- preview (rendered through tui-markdown so headings/emphasis/lists + -- are styled); keys in preview never edit content, and esc returns + -- to the sidebar from either mode. + ensures: PreviewToggled() +} + +rule CodeEditorWrapping { + when: TuiState.editing_post.updated() + -- The post editor renders the buffer with syntax highlighting + -- (markdown with [[macro]] accents for posts, HTML+Liquid for + -- templates, Lua for scripts), soft-wraps long lines to the + -- viewport width, keeps the cursor's visual row visible while + -- scrolling, and highlights the cursor's source line. There is no + -- line-number gutter. + ensures: TuiState.updated() +} + +rule AiQuickAction { + when: TuiKeyPressed(code: "g", modifiers: "ctrl") + -- One-shot AI post analysis (title/excerpt suggestions). Gated by + -- airplane mode: without a local endpoint the TUI shows a status + -- message instead of calling out. + ensures: AiSuggestionsRequestedOrRefused() +} + +rule ProjectSwitcher { + when: TuiKeyPressed(code: "p") + -- "p" opens the projects overlay: all projects from the caching + -- database with the active one marked; enter activates the selected + -- project (BDS.Projects.set_active_project) and reloads the sidebar. + -- "o" switches to a folder-path prompt with bash-style tab + -- completion: tab completes a unique directory (trailing slash + -- appended), an ambiguous prefix completes to the longest common + -- prefix and lists the candidate folders; only directories are + -- offered, dotfolders only for a dot-prefixed input, and a leading + -- "~" expands to the home directory. A typed path is validated + -- to be an existing directory, then opened as a project + -- (BDS.Projects.create_project with data_path, named after the + -- folder) and activated; a full database rebuild is queued + -- automatically through the rebuild_database shell command so the + -- folder's content is loaded into the caching database. Invalid + -- paths keep the prompt open and report the error. + ensures: ProjectSwitchedOrOpened() +} + +rule SidebarSearch { + when: TuiKeyPressed(code: "/") + -- "/" in sidebar focus opens a vi-style search prompt in the status + -- line that live-filters the current view through the same + -- BDS.UI.Sidebar filter params the GUI sidebar uses (posts, pages, + -- media). Plain words are a text search, "tag:x" and "category:x" + -- filter like the GUI chips, and an ISO date (yyyy-mm-dd) narrows + -- to that day — the day filter extends the shared sidebar queries. + -- Tokens combine with AND. Enter keeps the filter (shown appended + -- to the sidebar title), esc clears it; filters are per view. + ensures: SidebarFiltered() +} + +rule CommandPrompt { + when: TuiKeyPressed(code: ":") + -- Vi-style prompt: ":" opens a filterable list of the parameterless + -- Blog-menu shell commands (metadata diff, validate site, force + -- render, rebuilds, reindex, translations, duplicates, upload, + -- browser preview URL) and runs the selection through the same + -- BDS.Desktop.ShellCommands backend as the GUI menu; typing ":?" + -- shows the full list as help. Overlays clear the cells beneath + -- them. Commands whose follow-up UI is GUI-only (validate + -- translations, find duplicates) are marked. Queued tasks report + -- progress in the status line until all tasks finish. + ensures: CommandListShownOrExecuted() +} + +rule SettingsPanel { + when: TuiKeyPressed(code: "6") + -- "6" opens the settings panel (issue #29), matching the GUI panel + -- numbering: the sidebar lists the same preference sections as the + -- GUI settings editor (Project, Editor, Content, AI, Technology, + -- Publishing, Data, MCP, Style — from BDS.UI.Sidebar's settings + -- view). Enter on a section opens a section-specific editor in the + -- main area: a generic typed-field form from BDS.UI.SettingsForm. + -- Enter edits a text field in a status-line prompt, toggles a + -- boolean, or cycles an enum through its options; read-only info + -- rows are skipped by the selection. ctrl+s saves the section + -- through the same backends as the GUI editor (BDS.Metadata, + -- BDS.Settings, BDS.AI, BDS.MCP.AgentConfig) and reloads the form; + -- esc cancels the prompt, then closes the form. Category removal + -- and AI model discovery remain GUI-only. + ensures: SettingsFormShownEditedOrSaved() +} + +rule TagsPanel { + when: TuiKeyPressed(code: "5") + -- "5" opens the tags view (issue #34), matching the GUI panel + -- numbering. The sidebar lists the same three sections as the GUI + -- tags editor (Tag Cloud, Create / Edit, Merge Tags — from + -- BDS.UI.Sidebar's tags nav view); enter opens the section in the + -- main area, all backed by BDS.UI.TagsPanel over the same BDS.Tags + -- operations as the GUI editor. Cloud lists tags with their post + -- usage counts ordered by usage; manage is alphabetical with "n" + -- (create prompt), enter (rename prompt), "c" (cycle colour through + -- the shared presets), "t" (cycle post template), "d" then "y" + -- (delete with post-count confirmation; any other key cancels) and + -- "s" (sync tags from post tags); merge marks tags with space and + -- "m" merges the marked tags into the selected one (at least two, + -- target included). Text prompts live in the status line; esc + -- cancels the prompt, then closes the panel. Domain tag events + -- reload an open panel, keeping it in sync with GUI and CLI writes. + ensures: TagsPanelShownOrEdited() +} + +rule GitPanel { + when: TuiKeyPressed(code: "7") + -- "7" opens the git panel (issue #30) for content sync: the sidebar + -- lists the changed files from git status (code + path, branch with + -- ahead/behind counts in the title) with the commit history in its + -- lower half, like the GUI (short hash + subject; ↑ marks commits + -- that still need a push, ↓ remote-only ones), the main area shows a + -- scrollable whole-folder diff (staged + unstaged, capped) paged + -- with pgup/pgdn; enter on a file jumps the diff to it. "c" opens a + -- status-line commit prompt (git add -A + commit, empty messages + -- rejected), "u" pulls (ff-only) and "s" pushes — both run off the + -- UI process and report back as a status toast. Every action is + -- BDS.Git, the same backend as the GUI git panel. A project folder + -- that is not a git repository shows a message and refuses the + -- actions. + ensures: GitPanelShownOrSynced() +} + +rule ReportPanels { + when: ShellTaskCompleted(route: "metadata_diff" | "site_validation") + -- Completed metadata diff and site validation tasks open a + -- scrollable report panel showing the differences. Enter applies + -- the whole report — metadata diff repairs all items from the + -- filesystem (file → db) and imports orphan files; site validation + -- applies the full report incrementally (render missing, remove + -- extra, re-render updated), matching the GUI defaults. Esc closes + -- the report without applying. Reports completed before this + -- session started never pop up. + ensures: ReportShownThenAppliedOrCancelled() +} + +rule MultiClientSync { + when: TuiEventReceived(entity, entity_id, action) + -- Domain events refresh the sidebar; a post deleted elsewhere closes + -- the open editor. Keeps TUI sessions synchronized with GUI clients + -- and pipeline ingestion. + ensures: TuiState.updated() +} + +rule ServerSideLocale { + when: TuiSettingsChanged(key: "ui.language") + -- The TUI re-reads the server-side UI language and re-renders; all + -- clients always share one language. + ensures: TuiRelocalized() +} diff --git a/specs/ui_data_flow.allium b/specs/ui_data_flow.allium index 326a4c5..e10dfe9 100644 --- a/specs/ui_data_flow.allium +++ b/specs/ui_data_flow.allium @@ -35,9 +35,9 @@ surface UserNavigation { -- ─── 1. Sidebar -> Editor flows ────────────────────────────── --- All UI coordination flows through a shared reactive store (Zustand in TS, --- Iced subscriptions in Rust). There are no direct calls between sidebar --- and editor. Both read from and write to the store; changes propagate +-- All UI coordination flows through shared application state. +-- There are no direct calls between sidebar and editor. +-- Both read from and write to the same state model; changes propagate -- reactively. rule SidebarEntityClick {