Initial commit: allium behavioural specs distilled from bDS TypeScript app

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 18:36:20 +02:00
commit aec533b54f
32 changed files with 4569 additions and 0 deletions
--- a/ideas/recursive-sparking-haven.md
+++ b/ideas/recursive-sparking-haven.md
@@ -0,0 +1,303 @@
+# Feasibility Analysis: bDS Rewrite in Rust (Green-Field, No JavaScript)
+
+## Context
+
+Green-field rewrite of bDS blogging CMS in pure Rust. No migration — only the on-disc content format (Markdown + YAML frontmatter) must be preserved. Everything else is up for debate.
+
+**Constraints:**
+- No JavaScript at all (supply chain security concern — no npm/webview/JS runtime)
+- Python scripting replaced with Rust-native scripting language (Lua)
+- WYSIWYG Markdown editor not needed — syntax-highlighting source editor is sufficient
+- Mac is first-class, Windows slightly behind, Linux as compatibility option
+- Built by team of AI agents, no timeline pressure
+- Green-field: existing codebase is reference only, not migrated
+
+---
+
+## Current Scale
+
+| Layer | LOC | Key Tech |
+|-------|-----|----------|
+| Main process (engines, IPC, DB) | ~45,800 | Node.js, 43+ engine classes, 257 IPC methods |
+| Renderer (UI) | ~30,200 | React 19, Zustand, 80 components, Milkdown, Monaco |
+| Tests | ~69,500 | Vitest, 210 test files |
+| Dependencies | 54 runtime + 27 dev | AI SDK, Pyodide, sharp, git, MCP, etc. |
+
+---
+
+## Rust GUI Framework Comparison (Mac-First)
+
+The "Mac must be first-class" constraint is critical. It reshuffles the ranking.
+
+### Framework Assessment
+
+| Framework | Maturity | Mac Quality | Editor Story | Widget Set |
+|-----------|----------|-------------|-------------|------------|
+| **GTK4-rs + libadwaita** | 9/10 | 6/10 — functional but non-native feel (no system menu bar by default, non-native scrolling, GTK file dialogs) | GtkSourceView (excellent) | Rich |
+| **Iced** | 6/10 | 8/10 — GPU-rendered via Metal, consistent look, respects platform DPI. Not "native macOS" but consistent everywhere (like Electron was). | Custom needed (`syntect` + `ropey` + `cosmic-text`) | Growing, needs custom work |
+| **Floem** | 5/10 | 7/10 — used in Lapce which runs well on Mac | Lapce has one but not extracted | Sparse |
+| **Dioxus** | 4/10 | Variable | None | Tiny |
+| **Slint** | 7/10 | 7/10 — native rendering support | None | Limited |
+| **egui** | 8/10 | 7/10 — works well on Mac via wgpu | Wrong paradigm for editing | Immediate mode |
+
+### Mac-specific concerns
+
+**GTK4 on macOS:**
+- Requires GTK4 installed (Homebrew/MacPorts) or bundled (~50MB overhead)
+- Doesn't use native macOS menu bar by default (can be configured but finicky)
+- Non-native scrolling physics, selection behavior, keyboard shortcuts
+- File/print dialogs look GTK, not macOS
+- Apps like GIMP/Inkscape run on Mac via GTK — functional but feel "foreign"
+
+**Iced on macOS:**
+- Renders via wgpu/Metal — native GPU acceleration
+- Consistent cross-platform look (no "foreign toolkit" feel)
+- You control all UX conventions (can implement macOS cmd-shortcuts, native-feeling scrolling)
+- No external toolkit dependency — statically compiled
+- Binary stays small (~10-20MB)
+- The trade-off: you build more widgets yourself, but AI agents can handle this
+
+### The editor situation
+
+With WYSIWYG dropped, a syntax-highlighting text editor is achievable in pure Rust:
+
+| Approach | Effort | Quality |
+|----------|--------|---------|
+| **GtkSourceView** (GTK4 only) | 0 weeks (built-in) | Production-grade, 50+ languages |
+| **Custom iced widget** (`ropey` + `syntect` + `cosmic-text`) | 4-6 weeks | Good — proven building blocks, AI agents can build this |
+| **Extract cosmic-edit** from COSMIC | 3-4 weeks | Untested outside COSMIC |
+
+For an iced-based app, a custom editor widget built from `ropey` (rope buffer) + `syntect` (highlighting) + `cosmic-text` (text layout/shaping) is realistic. This is how Lapce and COSMIC's editors work internally.
+
+### Recommendation: **Iced**
+
+Given Mac-first + no JS + green-field + AI agents building it:
+
+1. **Iced** gives the best Mac experience without GTK's foreign feel
+2. Custom editor is feasible (4-6 weeks for AI agents using proven crates)
+3. GPU rendering (wgpu/Metal) = smooth, modern feel on all platforms
+4. Elm architecture = predictable state management, good for AI-generated code
+5. No external toolkit dependency = simpler distribution
+6. System76 backing ensures continued development
+
+---
+
+## Scripting Language Replacement (Python → ?)
+
+| Language | Crate | Maturity | Speed | Ecosystem | Best For |
+|----------|-------|----------|-------|-----------|----------|
+| **Lua** | `mlua` | 10/10 | Very fast (LuaJIT) | Huge (30+ years, games, tools) | Battle-tested embedding, rich stdlib |
+| **Rhai** | `rhai` | 8/10 | Moderate | Small but growing | Rust-native, safe sandbox, no FFI |
+| **Starlark** | `starlark-rust` | 7/10 | Fast | Niche (Bazel/Buck) | Python-like syntax, deterministic |
+
+**Recommendation: Lua via `mlua`.**
+- Most mature embedding story. Used by Neovim, Redis, nginx, game engines.
+- `mlua` supports Lua 5.4 and LuaJIT. Sandboxing built in.
+- Exposing engine methods to Lua is trivial (`mlua::UserData` trait).
+- Distribution: Lua runtime is ~300KB, compiled into the binary. No external install needed.
+- Existing Python macros would need rewriting, but Lua syntax is simple for end users.
+
+---
+
+## Two Viable Paths
+
+### Path A: Iced (Recommended — Mac-first, fully native, zero JS)
+
+| Aspect | Assessment |
+|--------|-----------|
+| **JS purity** | 100%. No webview, no JS runtime, no npm. |
+| **Rendering** | wgpu (Metal on Mac, Vulkan/DX12 elsewhere). Smooth, consistent everywhere. |
+| **Markdown editor** | Custom widget: `ropey` + `syntect` + `cosmic-text`. Live preview pane via `pulldown-cmark` rendering. |
+| **Code editor** | Same custom widget with Lua/Liquid/CSS language definitions via `syntect`. |
+| **Widgets needed** | Tree view, resizable panes, tabs, modals, toasts, forms — all buildable in iced, some exist in `iced_aw`. |
+| **Mac experience** | Native GPU, respects DPI, no foreign toolkit feel. cmd-key shortcuts, native-feeling scroll. |
+| **Distribution** | Single static binary. No external deps. ~15-25MB. |
+| **Risk** | Medium. Custom editor is the biggest piece. Iced ecosystem is growing but you build more yourself. |
+
+### Path B: GTK4-rs + GtkSourceView (Fastest, Linux-best)
+
+| Aspect | Assessment |
+|--------|-----------|
+| **JS purity** | 100%. |
+| **Rendering** | Native GTK (Cairo/Vulkan). |
+| **Editors** | GtkSourceView — zero work, production-grade. |
+| **Widgets** | All built-in (TreeView, Paned, Notebook, etc.). |
+| **Mac experience** | 6/10. Functional but feels non-native. Requires GTK4 installed via Homebrew. |
+| **Distribution** | Requires GTK4 runtime on user machine (Mac/Windows). |
+| **Risk** | Low for functionality, medium-high for Mac UX polish. |
+
+---
+
+## Backend Subsystem Assessment (Green-Field)
+
+Since this is a rewrite, effort is for building clean, not porting messy. AI agents can generate verbose Rust boilerplate efficiently.
+
+### 1. Database — Easy
+- **`rusqlite`** with `refinery` for migrations. Design schema fresh.
+- On-disc format preserved: Markdown + YAML frontmatter files. DB is for indexing/metadata only.
+- **Key crates:** `rusqlite`, `serde`, `serde_yaml`, `gray_matter` or manual YAML parsing
+- **Cost:** 2-3 weeks
+
+### 2. Core Engine Layer — Medium
+- Fresh design with Rust idioms (traits, enums, Result). No need to mirror 43 TS classes.
+- **No WXR import** — dropped entirely.
+- **Key crates:** `pulldown-cmark` (Markdown), `liquid` (templates), `image` (thumbnails/WEBP), `git2` (Git, no LFS — shell out for LFS ops), `notify` (file watching), `uuid`, `serde_json`
+- **Cost:** 8-12 weeks
+
+### 3. AI/LLM Integration — Easy ✅ (simplified)
+- **OpenAI-compatible endpoint only.** Single wire format (Chat Completions API).
+- Single-shot calls only (no streaming, no chat, no tool call loops). Used for: translation, image description, title generation.
+- **`reqwest`** + `serde_json` + OpenAI request/response structs. ~500 LOC.
+- Works with OpenCode Zen, local Ollama, any OpenAI-compatible proxy.
+- **Cost:** 1-2 weeks
+
+### 4. Lua Scripting — Easy
+- **`mlua`** crate: embed Lua 5.4. Sandboxed, ~300KB. Expose engine APIs via `UserData` trait.
+- **Cost:** 2-3 weeks
+
+### 5. Static Site Generation — Medium
+- `pulldown-cmark` → HTML, `liquid` templates, `rayon` for parallel generation
+- Pagefind CLI for search index (unchanged, it's a binary)
+- RSS/Atom/sitemap generation via `quick-xml`
+- **Cost:** 4-6 weeks
+
+### 6. Publishing — Easy
+- SSH: `ssh2` crate. rsync: shell out to `rsync`. Git: `git2`.
+- **Cost:** 1-2 weeks
+
+### 7. Embeddings & Similarity — Medium
+- `ort` (ONNX Runtime) for local embeddings. `usearch` Rust bindings for HNSW index.
+- **Cost:** 2-3 weeks
+
+### 8. MCP Server — Optional (v2)
+- Deferred. Not needed for v1.
+- **Cost:** 0 weeks (v1)
+
+### 9. Preview Server — Easy
+- `axum` HTTP server serving generated HTML + media.
+- **Cost:** 1-2 weeks
+
+### 10. UI (Iced) — Hard (largest piece)
+- Custom editor widget: `ropey` + `syntect` + `cosmic-text` (4-6 weeks)
+- Application chrome: tree sidebar, tab bar, resizable panes, settings, modals (6-8 weeks)
+- Chat panel, AI integration UI (2-3 weeks)
+- Git diff view, validation views, import wizard (3-4 weeks)
+- **Cost:** 15-21 weeks
+
+### 11. Tests — Medium
+- Rust `#[test]` + `mockall` for traits. AI agents generate test boilerplate well.
+- Test as you build (TDD per CLAUDE.md rules)
+- **Cost:** Folded into each subsystem (add ~40% to each estimate)
+
+---
+
+## What Gets Better (All Paths)
+
+1. **Binary size:** 5-15MB vs 150-200MB
+2. **Memory:** ~50-80% less RAM (no Chromium main process)
+3. **Startup:** Significantly faster
+4. **Generation perf:** `rayon` + `pulldown-cmark` + `liquid` = faster site generation
+5. **Type safety:** Compile-time guarantees on everything
+6. **Security:** Memory-safe, smaller supply-chain surface
+
+## What Gets Worse (All Paths)
+
+1. **Dev velocity:** Rust is slower to iterate than TypeScript
+2. **AI SDK:** Maintain your own streaming multi-provider SDK
+3. **Macro migration:** Existing user Python macros must be rewritten in Lua
+4. **Talent pool:** Smaller contributor base
+
+---
+
+## Effort Summary (Green-Field with AI Agents)
+
+| Subsystem | Weeks (with tests) |
+|-----------|-------------------|
+| Database + on-disc format | 3-4 |
+| Core engines (posts, media, tags, projects) | 11-17 |
+| AI/LLM (single-shot, OpenAI-compat) | 1-2 |
+| Lua scripting | 3-4 |
+| Static site generation | 6-8 |
+| Publishing + Git | 2-3 |
+| Embeddings + search | 3-4 |
+| Preview server | 1-2 |
+| **UI (Iced)** | **21-29** |
+| Integration + polish | 4-6 |
+| **Total** | **55-79 person-weeks** |
+
+Compared to original estimate (76-103 weeks): **dropping WXR import, streaming chat, multi-provider AI, and MCP server saves ~20 weeks.**
+
+With AI agents running 2-3 parallel workstreams: **4-8 months** of active sessions.
+
+---
+
+## Recommended Architecture: Iced + Rust
+
+```
+bds-rust/
+├── crates/
+│   ├── bds-core/          # Domain types, traits, on-disc format (Markdown+YAML)
+│   ├── bds-db/            # SQLite layer (rusqlite + refinery migrations)
+│   ├── bds-engine/        # Business logic (posts, media, tags, projects, generation)
+│   ├── bds-ai/            # Multi-provider LLM client (streaming, tool calls)
+│   ├── bds-lua/           # Lua scripting runtime (mlua)
+│   ├── bds-git/           # Git operations (git2 + LFS shell-out)
+│   ├── bds-publish/       # SSH/rsync deployment
+│   ├── bds-search/        # Embeddings (ort) + similarity (usearch) + pagefind
+│   ├── bds-mcp/           # MCP server
+│   ├── bds-preview/       # HTTP preview server (axum)
+│   └── bds-editor/        # Syntax-highlighting editor widget (ropey+syntect+cosmic-text)
+├── src/                   # Iced application (UI, state, views)
+├── assets/                # Icons, themes
+└── Cargo.toml             # Workspace
+```
+
+**Key Rust crates:**
+| Purpose | Crate |
+|---------|-------|
+| GUI framework | `iced` |
+| Text buffer | `ropey` |
+| Syntax highlighting | `syntect` |
+| Text layout | `cosmic-text` |
+| Database | `rusqlite` |
+| Migrations | `refinery` |
+| Markdown → HTML | `pulldown-cmark` |
+| Templates | `liquid` |
+| YAML frontmatter | `serde_yaml` |
+| Image processing | `image` |
+| HTTP server (preview) | `axum` |
+| HTTP client (AI) | `reqwest` |
+| Git | `git2` |
+| SSH | `ssh2` |
+| File watching | `notify` |
+| Lua scripting | `mlua` |
+| ONNX inference | `ort` |
+| Vector search | `usearch` |
+| Async runtime | `tokio` |
+| Parallelism | `rayon` |
+| Serialization | `serde` + `serde_json` |
+| UUID | `uuid` |
+| Error handling | `thiserror` + `anyhow` |
+
+---
+
+## Verdict
+
+**Feasible: Yes.** With the constraints resolved (no WYSIWYG, Lua instead of Python, green-field, AI agents, no timeline):
+
+- Every subsystem has proven Rust crate equivalents
+- The hardest piece is the custom editor widget (~4-6 weeks) and AI streaming client (~5-7 weeks)
+- Iced gives Mac-first cross-platform GUI without GTK's foreign feel or any JS runtime
+- Single static binary, ~15-25MB, zero external dependencies
+- AI agents are well-suited to Rust's verbose but structured patterns
+
+**The main risk** is Iced's maturity (6/10). It's real software backing a real desktop environment (COSMIC), but the widget ecosystem is young. You will build things that React gave you for free. With AI agents and no deadline, this is acceptable.
+
+---
+
+## Remaining Questions
+
+1. Confirm Iced vs GTK4-rs preference? (Iced = better Mac, more custom work. GTK = more widgets, worse Mac feel.)
+2. Which AI providers are must-have? (Anthropic + OpenAI + Ollama? Or all 5?)
+3. Should the MCP server remain, or is it optional for v1?
+4. Is WordPress WXR import a v1 requirement?