Implement project-isolated local semantic indexing and expose similar posts, tag suggestions, semantic search, and duplicate review through working UI.
Current state
embedding_keys and dismissed_duplicate_pairs tables exist.
No embedding model, vector index, lifecycle engine, or semantic query path exists.
Find Duplicates is a workspace placeholder.
specs/embedding.allium and DuplicatesSurface in specs/editor_misc.allium are normative.
Required behavior
Load/cache the specified real multilingual local embedding model and use native acceleration where supported.
Generate embeddings from the normative post text representation and store content hashes/vector cache in SQLite.
Maintain a project-isolated vector index with debounced persistence and stable label-to-post mapping.
Embed or update posts after normal post mutations, remove deleted posts, index missing content, and support full rebuild.
Skip unchanged content by hash and recover a missing/corrupt index from the database cache.
Implement semantic search, similar-post lookup, similarity computation, and tag suggestions.
Find duplicate pairs above the configured threshold, page results, exclude self/reversed duplicates, and persist single/batch dismissals.
Replace Find Duplicates with a localized review UI linked to the relevant posts.
Respect the project semantic-similarity setting and avoid network access at runtime after the model is available.
Keep metadata diff and rebuild behavior synchronized with embedding lifecycle requirements.
Implementation notes
Read docs/UI_STYLE_GUIDE.md before UI work.
Keep SQLite as the authoritative vector cache and one index per project.
Reuse post mutation/rebuild hooks and the shared task/event paths.
Follow red/green TDD; do not substitute a lexical heuristic for the specified neural model.
Acceptance criteria
Model load/cache, hash skip, insert/update/delete, index persistence/recovery, and project isolation are tested.
Semantic search and similar-post results use deterministic fixtures with threshold/order assertions.
Duplicate symmetry, pagination, dismiss, batch dismiss, and rebuild are tested.
Post CRUD and filesystem rebuild leave database keys and index labels consistent.
Find Duplicates is a working localized UI with no placeholder.
cargo fmt --all -- --check, cargo build --workspace, and cargo test --workspace pass.
## Goal
Implement project-isolated local semantic indexing and expose similar posts, tag suggestions, semantic search, and duplicate review through working UI.
## Current state
- embedding_keys and dismissed_duplicate_pairs tables exist.
- No embedding model, vector index, lifecycle engine, or semantic query path exists.
- Find Duplicates is a workspace placeholder.
- specs/embedding.allium and DuplicatesSurface in specs/editor_misc.allium are normative.
## Required behavior
- Load/cache the specified real multilingual local embedding model and use native acceleration where supported.
- Generate embeddings from the normative post text representation and store content hashes/vector cache in SQLite.
- Maintain a project-isolated vector index with debounced persistence and stable label-to-post mapping.
- Embed or update posts after normal post mutations, remove deleted posts, index missing content, and support full rebuild.
- Skip unchanged content by hash and recover a missing/corrupt index from the database cache.
- Implement semantic search, similar-post lookup, similarity computation, and tag suggestions.
- Find duplicate pairs above the configured threshold, page results, exclude self/reversed duplicates, and persist single/batch dismissals.
- Replace Find Duplicates with a localized review UI linked to the relevant posts.
- Respect the project semantic-similarity setting and avoid network access at runtime after the model is available.
- Keep metadata diff and rebuild behavior synchronized with embedding lifecycle requirements.
## Implementation notes
- Read docs/UI_STYLE_GUIDE.md before UI work.
- Keep SQLite as the authoritative vector cache and one index per project.
- Reuse post mutation/rebuild hooks and the shared task/event paths.
- Follow red/green TDD; do not substitute a lexical heuristic for the specified neural model.
## Acceptance criteria
- Model load/cache, hash skip, insert/update/delete, index persistence/recovery, and project isolation are tested.
- Semantic search and similar-post results use deterministic fixtures with threshold/order assertions.
- Duplicate symmetry, pagination, dismiss, batch dismiss, and rebuild are tested.
- Post CRUD and filesystem rebuild leave database keys and index labels consistent.
- Find Duplicates is a working localized UI with no placeholder.
- cargo fmt --all -- --check, cargo build --workspace, and cargo test --workspace pass.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Goal
Implement project-isolated local semantic indexing and expose similar posts, tag suggestions, semantic search, and duplicate review through working UI.
Current state
Required behavior
Implementation notes
Acceptance criteria
Implemented and pushed in
e876426. Added the real multilingual-e5-small ONNX backend with lazy local caching, query-prefix preprocessing, 384-dimensional normalized vectors, Core ML/DirectML acceleration with CPU fallback, project-isolated USearch HNSW indexes and durable SQLite BLOB cache. Added hash-skipping lifecycle hooks, batched cancellable backfill/reindex, debounced persistence, corrupt-index recovery, deletion/project cleanup, semantic sidebar search, similar-post/link ranking, tag suggestions, duplicate detection/pagination/canonical single and 100-row batch dismissals, localized native duplicate-review UI, menu actions, scripting API, metadata diff/repair, filesystem rebuild integration, migrations, docs, and plan updates. Neutral review compared the issue, bDS2 implementation/tests, specs/embedding.allium, and DuplicatesSurface in specs/editor_misc.allium; review fixes included stable-label upserts, enable-time backfill, managed progress/cancellation, cache cleanup, rebuild ordering, true batch inserts, and native-provider feature activation. Verification: real multilingual/cache model release test passed; cargo fmt --all -- --check passed; cargo build --workspace passed; cargo test --workspace passed (core 441 passed/3 intentionally ignored plus all UI/CLI/MCP/editor/integration/doc tests); cargo clippy --workspace --all-targets -- -D warnings passed; allium check specs returned zero findings.