Group site generation into section tasks followed by search indexing #14

New Issue

hugo · 2026-07-18T20:22:03Z

hugo commented

2026-07-18 20:22:03 +00:00

Goal

Make full generation and validation-apply use the bDS2 task structure: one task group with section render tasks, followed by a final search-index task.

Current state

Core generation already renders pages, language variants, archives, feeds, sitemap, assets, changed-file hashes, and Pagefind output.
Page rendering is already parallel internally.
The UI task manager already supports grouped tasks.
Message::GenerateSite in crates/bds-ui/src/app/engine_handlers.rs currently wraps the entire operation in one coarse task.
specs/generation.allium, especially GenerationRuntimeSurface.ProgressReporting, is normative.

Required behavior

For full generation, submit a single visible group containing these render tasks:

Render Site Core
Render Single Posts
Render Category Archives
Render Tag Archives
Render Date Archives

After all requested render tasks complete successfully, submit Build Search Index in the same group. The index task must not start after a failed or cancelled render phase.

Validation apply uses the same structure but creates only tasks for affected sections, then refreshes ancillary output/search data as required by specs/generation.allium.

Each render task reports one progress message per written URL using the specified Generated /url (n/total) or Rewrote /url (n/total) wording. Preserve published-snapshot-only generation, incremental content-hash skipping, language routing, and existing page-level parallelism.

Implementation notes

Reuse TaskManager::submit_grouped and the existing generation engine. Do not add another scheduler.
Split orchestration at existing section boundaries; do not duplicate rendering logic.
Build Pagefind once, only after section rendering.
Cancellation and failure must settle the group cleanly and leave a useful Output/task error.
Follow red/green TDD.
Compare with ../bDS2/lib/bds/desktop/shell_commands.ex section task queuing and ../bDS2/lib/bds/generation.ex when the spec is silent.

Acceptance criteria

Full generation shows the five named section tasks and one final Build Search Index task in one group.
The index begins only after every render task succeeds.
Failure/cancellation prevents indexing and is reflected in group/task state.
Validation apply creates only affected section tasks and follows them with the required final phase.
Output is equivalent to the current generator and passes site validation, including multilingual routes and Pagefind artifacts.
Tests verify task names/order/dependency, progress messages, cancellation/failure behavior, and complete generated output.
allium check specs/*.allium, cargo fmt --all -- --check, cargo build --workspace, and cargo test --workspace pass.

## Goal Make full generation and validation-apply use the bDS2 task structure: one task group with section render tasks, followed by a final search-index task. ## Current state - Core generation already renders pages, language variants, archives, feeds, sitemap, assets, changed-file hashes, and Pagefind output. - Page rendering is already parallel internally. - The UI task manager already supports grouped tasks. - `Message::GenerateSite` in `crates/bds-ui/src/app/engine_handlers.rs` currently wraps the entire operation in one coarse task. - `specs/generation.allium`, especially `GenerationRuntimeSurface.ProgressReporting`, is normative. ## Required behavior For full generation, submit a single visible group containing these render tasks: 1. `Render Site Core` 2. `Render Single Posts` 3. `Render Category Archives` 4. `Render Tag Archives` 5. `Render Date Archives` After all requested render tasks complete successfully, submit `Build Search Index` in the same group. The index task must not start after a failed or cancelled render phase. Validation apply uses the same structure but creates only tasks for affected sections, then refreshes ancillary output/search data as required by `specs/generation.allium`. Each render task reports one progress message per written URL using the specified `Generated /url (n/total)` or `Rewrote /url (n/total)` wording. Preserve published-snapshot-only generation, incremental content-hash skipping, language routing, and existing page-level parallelism. ## Implementation notes - Reuse `TaskManager::submit_grouped` and the existing generation engine. Do not add another scheduler. - Split orchestration at existing section boundaries; do not duplicate rendering logic. - Build Pagefind once, only after section rendering. - Cancellation and failure must settle the group cleanly and leave a useful Output/task error. - Follow red/green TDD. - Compare with `../bDS2/lib/bds/desktop/shell_commands.ex` section task queuing and `../bDS2/lib/bds/generation.ex` when the spec is silent. ## Acceptance criteria - Full generation shows the five named section tasks and one final `Build Search Index` task in one group. - The index begins only after every render task succeeds. - Failure/cancellation prevents indexing and is reflected in group/task state. - Validation apply creates only affected section tasks and follows them with the required final phase. - Output is equivalent to the current generator and passes site validation, including multilingual routes and Pagefind artifacts. - Tests verify task names/order/dependency, progress messages, cancellation/failure behavior, and complete generated output. - `allium check specs/*.allium`, `cargo fmt --all -- --check`, `cargo build --workspace`, and `cargo test --workspace` pass.

hugo added the enhancement label 2026-07-18 20:22:03 +00:00

hugo commented

2026-07-19 08:28:06 +00:00

Implemented in commit 90a9002. Full generation now queues Render Site Core, Render Single Posts, Render Category Archives, Render Tag Archives, and Render Date Archives in one group, followed by Build Search Index only after every render task succeeds. Validation apply queues only affected sections, rewrites only reported URLs, and rebuilds search output. Failure and cancellation settle the group without indexing; progress, published snapshots, multilingual routes, day archives, 404 pages, localized Pagefind assets, and stale index cleanup are covered. Verified with allium check specs/*.allium, cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings, cargo build --workspace, and cargo test --workspace.

Implemented in commit 90a9002. Full generation now queues Render Site Core, Render Single Posts, Render Category Archives, Render Tag Archives, and Render Date Archives in one group, followed by Build Search Index only after every render task succeeds. Validation apply queues only affected sections, rewrites only reported URLs, and rebuilds search output. Failure and cancellation settle the group without indexing; progress, published snapshots, multilingual routes, day archives, 404 pages, localized Pagefind assets, and stale index cleanup are covered. Verified with allium check specs/*.allium, cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings, cargo build --workspace, and cargo test --workspace.

hugo closed this issue

2026-07-19 08:28:16 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/RuDS#14