bDS2/TUI_EDITOR.md

TUI Editor — Implementation Plan (Issue #32)

Tracking issue: #32 — "markdown, liquid and lua code editor with syntax highlighting in the TUI". Spec under change: specs/tui.allium.

Goal: a syntax-highlighted, word-wrapping TUI editor for posts (markdown with [[macro]] syntax), templates (HTML + Liquid), and scripts (Lua), replacing the current plain ExRatatui.Widgets.Textarea body.

Decision: pure Elixir — no new Rust

ExRatatui already ships everything we need:

• ExRatatui.CodeBlock.highlight/3 (deps/ex_ratatui/lib/ex_ratatui/code_block.ex:103) returns [%ExRatatui.Text.Line{}] of styled spans via the bundled syntect NIF. Supports Lua, Markdown, HTML, … out of the box.
• ExRatatui.Widgets.CodeBlock is read-only; we do not use it directly for editing.
• ExRatatui.Widget protocol (deps/ex_ratatui/lib/ex_ratatui/widget.ex:1) lets us implement a custom widget in pure Elixir that composes primitives every frame. This is the seam we use.
• The editing engine stays Rust-owned (ExRatatui.textarea_new/0, textarea_handle_key/3, textarea_get_value/1, textarea_cursor/1, textarea_line_count/1). The custom widget only re-renders the buffer through a styling + wrap pipeline.
• ExRatatui.Widgets.Paragraph accepts [%Text.Line{}] of styled spans with wrap: true and a scroll: {vertical, horizontal} offset — exactly the output shape we need.

The existing comment at lib/bds/tui.ex:1451-1454 already pre-announces this approach as "the planned MarkdownEditor custom widget".

Languages supported (matches GUI Monaco registrations in assets/js/monaco/languages.js):

Entity	Base lexer (NIF)	Elixir overlay
post body	markdown	[[macro …]] recoloured
script body	lua	none
template body	html	{{ … }} and {% … %} recoloured

Liquid is always HTML+Liquid in this codebase (no markdown+Liquid templates exist). Plain text falls back to syntect's plain renderer.

GUI styling parity (from assets/js/monaco/theme.js + languages.js)

• macro [[ … ]] → keyword.macro = #C586C0, bold.
• inside macro: attribute.name (e.g. names) → #9CDCFE, attribute.value (e.g. "x") → #CE9178.
• Liquid uses base vs-dark tokens; terminal approximation uses the same base theme accent colours. No custom Liquid token colours overridden.

Use {:rgb,197,134,192} etc. via ExRatatui.Style (already supported by ExRatatui.CodeBlock.from_native/1).

Architecture

state.editor.textarea            (unchanged, Rust-owned buffer/cursor/undo)
state.editor.language            (:markdown_macros | :liquid | :lua | :text)
state.editor.preview?            (unchanged ctrl+e toggle — default false for posts)

body_widget/3  edit branch:      %BDS.UI.CodeEditor{textarea:, language:, theme:, wrap:, cursor_style:, line_highlight_style:, block:}
                                 ↳ impl ExRatatui.Widget.render:
                                      1. content = ExRatatui.textarea_get_value(textarea)
                                      2. {row, col} = ExRatatui.textarea_cursor(textarea)
                                      3. lines = BDS.UI.CodeEditor.Highlight.highlight(content, language, theme)
                                      4. apply current-line bg style into every span on lines[row]
                                      5. cut the span at byte col on lines[row], splice cursor-style cell
                                      6. top = vertical-scroll offset so cursor visual row stays in window (Latin-width count)
                                      7. return [{%Paragraph{text: lines, wrap: true, scroll: {top,0}, block: block}, rect}]

editor_key/1 in lib/bds/tui.ex keeps calling ExRatatui.textarea_handle_key/3 — no edit-model rewrite, the textarea's own undo/redo/clipboard keeps working.

Phases (test-first, per AGENTS.md)

Phase 0 — Spec

Edit specs/tui.allium:

• OpenEntry: posts open in the editor (wrap on, highlighting on) by default — not the rendered Markdown preview. New empty posts still open in the editor (unchanged).
• Rename WrappedPreview → EditorMode: ctrl+e toggles editor ↔ read-only rendered-Markdown preview (via ExRatatui.Widgets.Markdown). Drop the "textarea cannot wrap" caveat.
• Add CodeEditorWrapping rule: syntax highlighting + auto wrap + highlighted current line. Explicitly no line-number gutter.
• Validate with allium check specs/tui.allium.

Phase 1 — Red tests

New file test/bds/ui/code_editor_test.exs:

• BDS.UI.CodeEditor.Highlight.highlight/3 for :lua colours a function … end keyword with the theme's keyword colour.
• :markdown_macros recolours [[gallery names="x"]]:
  • [[, ]] and macro name → {:rgb,197,134,192} bold
  • names → {:rgb,156,220,254}
  • "x" → {:rgb,206,145,120}
  • surrounding markdown still base-styled.
• :liquid recolours {{ x }} and {% if x %} over an HTML base; if keyword distinct from identifiers.
• Widget emits a %Paragraph{wrap: true, scroll:}, never a bare %Textarea, when language is one of the highlighted set.

Extended test/bds/tui_test.exs:

• Opening an existing post lands in the editor (preview? == false), not the rendered preview. Replace the assertion in test "opening a post lands in the markdown preview, not the editor". Test name/logic flips.
• Long line wraps across N visual rows in a 40-wide viewport (assert via CellSession.take_cells/1).
• Cursor at the last visual row stays visible when scrolling a small viewport (assert cursor-style cell appears in snapshot cells).
• Current line's cells have a distinct bg from non-current rows on the same buffer.
• ctrl+e flips to preview and back; preview still rendered Markdown.
• ctrl+s persists textarea bytes unchanged regardless of highlight/wrap (existing assertion still covers this — keep it green).

Phase 2 — lib/bds/ui/code_editor/highlight.ex + overlay.ex

• BDS.UI.CodeEditor.Highlight.highlight(content, language, theme) → [%ExRatatui.Text.Line{}]:
  • :lua, :text → pass straight through to ExRatatui.CodeBlock.highlight/3.
  • :markdown_macros → base highlight as "markdown", then apply Overlay.markdown_macros/1 per source line.
  • :liquid → base highlight as "html", then apply Overlay.liquid/1 per source line.
• BDS.UI.CodeEditor.Overlay:
  • apply_ranges(line, ranges_with_styles) — re-splits a Line's spans into a new span list, preserving the base styling outside each range.
  • markdown_macro_ranges/1 — ~r/\[\[\s*[a-zA-Z][\w-]*[^]]*\]\]/s.
  • macro_inner_ranges/1 — break attr= (attr-name) and quoted value into separate styled sub-ranges inside a macro.
  • liquid_ranges/1 — ~r/(\{\{.*?\}\}|\{%-?\s*(\w+).*?-?%\})/s, with a sub-range for the tag keyword (if, for, assign, …).
• Cache results by {content_hash, language, theme} if profiling shows re-tokenisation overhead on keystroke (Phase 2 stretch goal — skip until a test says otherwise).

Phase 3 — lib/bds/ui/code_editor/widget.ex

• %BDS.UI.CodeEditor{} struct fields: textarea, language, theme (default :base16_ocean_dark, readable in dark terminals), wrap: true, cursor_style, line_highlight_style, block, scroll.
• defimpl ExRatatui.Widget per the pipeline above.
• Cursor cell: re-style the single codepoint at col on source line row into cursor_style. Paragraph wraps the same line; the styled cell rides the wrap.
• Scroll math: count display rows by wrapping each source line into width columns (Latin-width assumption — 1 char = 1 cell). Note the limitation explicitly; CJK width is a follow-up if/when needed.
• Line highlight: apply line_highlight_style (a bg: colour) to every span on the cursor's source line before cursor cut-in.

Phase 4 — Wire into lib/bds/tui.ex

• build_editor/4 (around lib/bds/tui.ex:1968): default preview? = false for existing posts. New posts already open in editor — unchanged.
• body_widget/3 edit branch (lib/bds/tui.ex:1466): emit %BDS.UI.CodeEditor{} instead of %ExRatatui.Widgets.Textarea{} for entities that have a syntax highlighter (:lua, :markdown_macros, :liquid). Plain text can keep %Textarea{} or feed :text through the new widget — pick whichever keeps the test surface smaller.
• Remove the "wrap limitation" comment block at lib/bds/tui.ex:1451-1454.
• cycle_language/1 continues per post; cycle through the active set (:markdown_macros, :liquid, :lua, :text) matching the existing GUI editor language options.
• Preview branch (body_widget/3 preview arm at lib/bds/tui.ex:1455) stays on ExRatatui.Widgets.Markdown — unchanged.
• No new user-facing strings: the existing body title already flows through dgettext("ui", …). If a new toolbar/hint is added later, it MUST go through gettext and provide de/fr/it/es translations.

Phase 5 — Verify

• mix test (write to /tmp/test_run.log first, grep for failures — per AGENTS.md).
• mix credo --strict
• mix deps.audit --ignore-file .mix_audit.ignore
• mix dialyzer — treat warnings as errors.

No bundle rebuild, no asset change, no NIF rebuild. Monaco still drives the GUI editor; this change is TUI-only.

Risks / limits

• Per-frame NIF cost: re-tokenises the whole buffer on every keystroke. For typical post sizes (<50 KB) syntect is sub-ms. Add the {content_hash, language, theme} cache only if a test shows it.
• CJK cell width: wrap math assumes 1 char = 1 cell. Fine for de/fr/it/es. Note as TODO; not in scope for this phase.
• Liquid mis-lexing inside HTML is cosmetically possible (an HTML attribute containing {{). Overlay wins after the base lexer; the result is visually tolerable. A custom .sublime-syntax in the dep would be the real fix — out of scope here per "stay on Elixir".
• Cursor in a wrapped paragraph rides the styled cell. Equivalent to a cell-caret; GUI uses a real caret — acceptable TUI parity.

Files touched (expected)

• specs/tui.allium — edited.
• lib/bds/ui/code_editor/highlight.ex — new.
• lib/bds/ui/code_editor/overlay.ex — new.
• lib/bds/ui/code_editor/widget.ex — new.
• lib/bds/tui.ex — edited (build_editor, body_widget, comment removal, default preview?).
• test/bds/ui/code_editor_test.exs — new.
• test/bds/tui_test.exs — edited (flip the "opens in preview" assertion, add wrap/cursor/current-line tests).

No changes to mix.exs, no new deps, no Rust, no assets.