Align background task execution and progress with bDS2 #111

New Issue

hugo · 2026-07-24T07:36:09Z

hugo commented

2026-07-24 07:36:09 +00:00

Summary

Align RuDS background-task coverage, scheduling, cancellation, progress reporting, task UI, scripting semantics, and remote integration with bDS2. Preserve the stronger RuDS atomic database-rebuild behavior while matching the observable bDS2 phases and progress.

This issue is based on a static code-path audit; runtime performance measurements remain part of acceptance.

Current standing

Already broadly aligned:

FIFO states: pending, running, completed, failed, cancelled
Default concurrency equals online CPU count
250 ms per-task progress throttling
Task groups, recent 10 finished tasks, one-hour retention
Full/forced site rendering uses five section tasks followed by search indexing
RuDS uses one shared Rayon CPU pool for page rendering
Embedding rebuild, fill-missing-translations, thumbnail regeneration, publishing, auto-translation, and utility scripts use the manager

Partial or missing parity:

Queued jobs enter spawn_blocking before admission and occupy blocking-pool workers while waiting.
Cancelling a running task releases its manager slot before the worker necessarily stops.
Task lookup/progress scans retained Vec entries instead of keyed storage plus FIFO queues.
A final 1.0 progress event may be throttled, leaving stale phase text.
The status bar compares localized status text with lowercase running, so its task indicator never appears.
The Tasks panel applies a second 10-entry cap and can hide active tasks or truncate groups.
Headless/server retention is not lazily pruned.
Lua task lookup, cancellation, and clear semantics differ from bDS2.
Remote task events are logged but not integrated into the Tasks panel/status bar.
Render-route estimation runs synchronously on the UI thread; validation totals can describe the full section instead of selected URLs.

bDS2 task coverage to align

Managed in both, but RuDS needs progress or workflow improvements:

Search text reindex
Embedding rebuild
Posts/media/scripts/templates rebuilds
Post links and missing thumbnails
Full database rebuild
Render/force render and search index
Validation apply
Calendar regeneration
Fill missing translations and automatic translations
Publishing

Currently bypassing the RuDS manager or missing UI integration:

Site validation
Translation validation
Metadata diff, repair, and orphan import
Duplicate search
Manual post/media translation
Media-link rebuild

Keep editor-scoped async work outside the global manager where bDS2 does the same: WXR import, previews, chat, and one-shot analysis. Git, media validation, image import/enrichment, and managed scripts remain valid RuDS additions.

Tag delete/merge/rename are synchronous in both implementations but violate the synchronized Allium background-task requirement; correct RuDS as part of this work.

Implementation

Update task/layout/script/generation Allium where snapshot ordering, cancellation-slot release, and task coverage are ambiguous; validate specs.
Replace retained task scans with standard-library keyed storage and FIFO ordering.
Admit Tokio work asynchronously before spawn_blocking; hold capacity until the actual worker exits, including after cancellation.
Centralize terminal lifecycle so panic, failure, and cancellation cannot drift from worker state.
Add cooperative cancellation at phase/item boundaries and terminate owned subprocesses where possible.
Always accept terminal progress; preserve localized final messages.
Lazily prune retained tasks and align get/all/running/cancel/clear scripting behavior with bDS2.
Keep typed task status through UI state; localize only its display label.
Show first running task, otherwise first pending task, with overflow across every active task.
Remove the panel secondary limit; show all active tasks plus ten newest finished tasks with complete groups.
Move render preparation off the UI thread and derive exact totals from prepared/selected work.
Route site/translation validation, metadata workflows, duplicate search, manual translation, media-link rebuild, and tag bulk operations through the shared task launcher.
Keep database rebuild under its SQLite savepoint; report bDS2-equivalent weighted phases and include thumbnail plus embedding phases.
Structure validation apply as prepare -> affected sections -> optional calendar -> search index -> final validation.
Add per-item or per-phase progress to entity/link rebuilds, calendar, auto-translation, and publishing.
Extend remote task snapshots with group and cancellability data; merge them with local tasks using a typed source and route cancellation through the existing call protocol.
Update all localization catalogs and README. No new dependency or database migration.

Acceptance criteria

Actual concurrent workers never exceed the configured maximum, including during cancellation.
Pending tasks consume no Tokio blocking-pool worker. FIFO remains deterministic.
Every bDS2 manager-backed operation listed above appears in RuDS Tasks with localized phase/progress and a correct terminal result.
Failure cancels dependent workflow phases; cancellation stops cooperatively without releasing capacity early.
Status bar works in every locale and reports pending/running overflow correctly.
More than ten active tasks and complete task groups remain visible; only finished history is capped at ten.
Local, server/TUI, scripting, and remote snapshots share the same retention and status semantics.
Full rebuild preserves RuDS database rollback behavior and includes missing thumbnails.
Generated large-site measurements record UI enqueue latency, maximum active workers, render wall time, and task-update frequency before/after.
Unit/integration tests cover FIFO, concurrency, cancellation, panic cleanup, progress throttling, retention, scripting APIs, workflow sequencing, remote tasks, and localized UI state.
cargo test --workspace, cargo build --workspace, Allium validation, macOS bundling, and Computer Use inspection pass.

Defaults

Retain the 500 ms desktop refresh unless measurement shows material cost.
Retain TaskManager::new as the concurrency override; do not add a user preference without evidence.
Use standard library and existing Tokio/Rayon facilities only.

## Summary Align RuDS background-task coverage, scheduling, cancellation, progress reporting, task UI, scripting semantics, and remote integration with bDS2. Preserve the stronger RuDS atomic database-rebuild behavior while matching the observable bDS2 phases and progress. This issue is based on a static code-path audit; runtime performance measurements remain part of acceptance. ## Current standing Already broadly aligned: - FIFO states: pending, running, completed, failed, cancelled - Default concurrency equals online CPU count - 250 ms per-task progress throttling - Task groups, recent 10 finished tasks, one-hour retention - Full/forced site rendering uses five section tasks followed by search indexing - RuDS uses one shared Rayon CPU pool for page rendering - Embedding rebuild, fill-missing-translations, thumbnail regeneration, publishing, auto-translation, and utility scripts use the manager Partial or missing parity: - Queued jobs enter `spawn_blocking` before admission and occupy blocking-pool workers while waiting. - Cancelling a running task releases its manager slot before the worker necessarily stops. - Task lookup/progress scans retained `Vec` entries instead of keyed storage plus FIFO queues. - A final `1.0` progress event may be throttled, leaving stale phase text. - The status bar compares localized status text with lowercase `running`, so its task indicator never appears. - The Tasks panel applies a second 10-entry cap and can hide active tasks or truncate groups. - Headless/server retention is not lazily pruned. - Lua task lookup, cancellation, and clear semantics differ from bDS2. - Remote task events are logged but not integrated into the Tasks panel/status bar. - Render-route estimation runs synchronously on the UI thread; validation totals can describe the full section instead of selected URLs. ## bDS2 task coverage to align Managed in both, but RuDS needs progress or workflow improvements: - Search text reindex - Embedding rebuild - Posts/media/scripts/templates rebuilds - Post links and missing thumbnails - Full database rebuild - Render/force render and search index - Validation apply - Calendar regeneration - Fill missing translations and automatic translations - Publishing Currently bypassing the RuDS manager or missing UI integration: - Site validation - Translation validation - Metadata diff, repair, and orphan import - Duplicate search - Manual post/media translation - Media-link rebuild Keep editor-scoped async work outside the global manager where bDS2 does the same: WXR import, previews, chat, and one-shot analysis. Git, media validation, image import/enrichment, and managed scripts remain valid RuDS additions. Tag delete/merge/rename are synchronous in both implementations but violate the synchronized Allium background-task requirement; correct RuDS as part of this work. ## Implementation - [ ] Update task/layout/script/generation Allium where snapshot ordering, cancellation-slot release, and task coverage are ambiguous; validate specs. - [ ] Replace retained task scans with standard-library keyed storage and FIFO ordering. - [ ] Admit Tokio work asynchronously before `spawn_blocking`; hold capacity until the actual worker exits, including after cancellation. - [ ] Centralize terminal lifecycle so panic, failure, and cancellation cannot drift from worker state. - [ ] Add cooperative cancellation at phase/item boundaries and terminate owned subprocesses where possible. - [ ] Always accept terminal progress; preserve localized final messages. - [ ] Lazily prune retained tasks and align get/all/running/cancel/clear scripting behavior with bDS2. - [ ] Keep typed task status through UI state; localize only its display label. - [ ] Show first running task, otherwise first pending task, with overflow across every active task. - [ ] Remove the panel secondary limit; show all active tasks plus ten newest finished tasks with complete groups. - [ ] Move render preparation off the UI thread and derive exact totals from prepared/selected work. - [ ] Route site/translation validation, metadata workflows, duplicate search, manual translation, media-link rebuild, and tag bulk operations through the shared task launcher. - [ ] Keep database rebuild under its SQLite savepoint; report bDS2-equivalent weighted phases and include thumbnail plus embedding phases. - [ ] Structure validation apply as prepare -> affected sections -> optional calendar -> search index -> final validation. - [ ] Add per-item or per-phase progress to entity/link rebuilds, calendar, auto-translation, and publishing. - [ ] Extend remote task snapshots with group and cancellability data; merge them with local tasks using a typed source and route cancellation through the existing call protocol. - [ ] Update all localization catalogs and README. No new dependency or database migration. ## Acceptance criteria - Actual concurrent workers never exceed the configured maximum, including during cancellation. - Pending tasks consume no Tokio blocking-pool worker. FIFO remains deterministic. - Every bDS2 manager-backed operation listed above appears in RuDS Tasks with localized phase/progress and a correct terminal result. - Failure cancels dependent workflow phases; cancellation stops cooperatively without releasing capacity early. - Status bar works in every locale and reports pending/running overflow correctly. - More than ten active tasks and complete task groups remain visible; only finished history is capped at ten. - Local, server/TUI, scripting, and remote snapshots share the same retention and status semantics. - Full rebuild preserves RuDS database rollback behavior and includes missing thumbnails. - Generated large-site measurements record UI enqueue latency, maximum active workers, render wall time, and task-update frequency before/after. - Unit/integration tests cover FIFO, concurrency, cancellation, panic cleanup, progress throttling, retention, scripting APIs, workflow sequencing, remote tasks, and localized UI state. - `cargo test --workspace`, `cargo build --workspace`, Allium validation, macOS bundling, and Computer Use inspection pass. ## Defaults - Retain the 500 ms desktop refresh unless measurement shows material cost. - Retain `TaskManager::new` as the concurrency override; do not add a user preference without evidence. - Use standard library and existing Tokio/Rayon facilities only.

hugo added the enhancement label 2026-07-24 07:36:09 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/RuDS#111