From a1004d72bfccf60a438b5f160581eed8314ed242 Mon Sep 17 00:00:00 2001 From: Chili Palmer Date: Fri, 29 May 2026 14:04:51 +0200 Subject: [PATCH] fix: A1-14 real neural embeddings via Bumblebee multilingual-e5-small with Float32 BLOB vector cache --- SPECGAPS.md | 5 +- config/config.exs | 7 +- config/runtime.exs | 5 + config/test.exs | 8 ++ lib/bds/application.ex | 13 ++- lib/bds/embeddings.ex | 24 +++- lib/bds/embeddings/backends/in_app.ex | 10 +- lib/bds/embeddings/backends/neural.ex | 104 ++++++++++++++++++ lib/bds/embeddings/index.ex | 8 +- lib/bds/embeddings/key.ex | 4 +- mix.exs | 7 +- mix.lock | 31 ++++-- ...15329_convert_embedding_vector_to_blob.exs | 33 ++++++ specs/embedding.allium | 3 + test/bds/embeddings/backends/neural_test.exs | 39 +++++++ test/bds/embeddings_test.exs | 30 +++++ 16 files changed, 310 insertions(+), 21 deletions(-) create mode 100644 lib/bds/embeddings/backends/neural.ex create mode 100644 priv/repo/migrations/20260529115329_convert_embedding_vector_to_blob.exs create mode 100644 test/bds/embeddings/backends/neural_test.exs diff --git a/SPECGAPS.md b/SPECGAPS.md index dab261b..6765c75 100644 --- a/SPECGAPS.md +++ b/SPECGAPS.md @@ -23,7 +23,8 @@ Gap categories: **SC** = spec correct, fix code | **CS** = code correct, update | A1-11 | ~~Graceful shutdown with inflight request tracking~~ | preview.allium:47-48 | `stop_preview` now closes the listener, parks the reply, and drains monitored inflight request tasks before reporting stopped | **Resolved:** acceptor transfers socket ownership to each request task; GenServer monitors inflight tasks, `begin_graceful_stop` stops accepting and finalizes via `:DOWN`/`:drain_timeout` (5s force-kill cap), 1 test added | | A1-12 | ~~Real Pagefind integration for search~~ | generation.allium:208 | Functional client-side search: `PagefindUI` defined in bundled `pagefind-ui.js`, fragment index records url/title/body-scoped text per page, search-runtime wires it up | **Resolved:** bundled real `PagefindUI` (fetch index, ranked full-text match, highlighted excerpts) + `pagefind-ui.css` as local assets read into `Pagefind`; index scoped to `data-pagefind-body` (unmarked pages excluded per PagefindHtmlMarking), title from ``/`<h1>`; localized "No results found" label via `data-search-no-results` (de/fr/it/es); 3 unit tests added | | A1-13 | ~~Git sidebar shows only "Working tree" placeholder~~ | sidebar_views.allium:651-770 | `git_view/1` now builds a full `layout: "git"` view from `BDS.Git` (repository/remote_state/status/history); `SidebarComponents` renders active + not_a_repo states | **Resolved:** `git_view/1` in sidebar.ex assembles branch/upstream/ahead/behind, status files, paginated history (20/page); `render_git_sidebar` renders branch header, sync legend, fetch/pull/push/prune-lfs buttons, commit form, clickable status files (open git_diff), history entries; shell_live wires `git_commit` (closes git_diff tabs), `git_fetch`/`git_pull`/`git_push`/`git_prune_lfs`, `git_initialize`; `BDS.Git.history` enriched with author/date, `BDS.Git.set_remote/2` added; i18n for de/fr/it/es; 3 shell tests + git author/date assertions added | -| A1-14 | Embedding uses TF-IDF hash projection instead of real neural model | embedding.allium:44-53, invariants ModelCaching/VectorCacheInDb | `backends/in_app.ex` hashes terms into sparse vectors via `:erlang.phash2`; no ONNX model, no `"query: "` prefix, no mean pooling, vectors stored as JSON text not Float32Array BLOB, snapshot-based neighbor lookup instead of USearch HNSW index | Fix code: (1) add Bumblebee + ONNX runtime deps to run `Xenova/multilingual-e5-small`, (2) implement lazy model download + cache in app data dir, (3) `"query: "` prefix + mean pooling + L2 norm in backend, (4) store vectors as binary BLOB (1536 bytes), (5) replace JSON snapshot with USearch HNSW index (cosine, M=16, ef=128/64, 5s debounce), (6) cross-language semantic similarity must work | +| A1-14 | ~~Embedding uses TF-IDF hash projection instead of real neural model~~ | embedding.allium:44-53, invariants RealNeuralModel/ModelCaching/VectorCacheInDb | `Backends.Neural` runs `intfloat/multilingual-e5-small` (e5 weights behind the Xenova id) via Bumblebee+EXLA | **Resolved (core):** added bumblebee/nx/exla deps; `Backends.Neural` is a lazily-loaded GenServer that builds the Bumblebee text-embedding serving on first request (`"query: "` prefix + mean pooling + L2 norm), downloads+caches the model under the app data dir (ModelCaching), and is wired into the supervision tree when configured; vectors now persisted as packed little-endian Float32 BLOB (384×4=1536 bytes) instead of JSON text (VectorCacheInDb) with migration recreating `embedding_keys.vector` as BLOB; `InApp` demoted to documented offline/test stub; test config uses the stub so the suite stays offline; spec EmbeddingModel clarified (Xenova id ↔ intfloat weights via Bumblebee); 3 tests added (BLOB round-trip + Neural model_info/behaviour). **Deferred to A1-14b:** USearch HNSW index. | +| A1-14b | USearch HNSW ANN index + debounced persistence not implemented | embedding.allium:75-87 (config), FindSimilar, invariant DebouncedPersistence | Neighbor lookup still uses the JSON cosine snapshot (`Embeddings.Index`), not a USearch HNSW index; no 5s debounced index persistence (snapshot rebuilt synchronously) | Fix code: replace JSON snapshot with USearch HNSW index file (`embeddings.usearch`, cosine, M=16, efConstruction=128, efSearch=64), label→post_id mapping, 5s debounced save + force-save on project switch/shutdown | | A1-15 | ~~Preview vs generation content source strategy undocumented~~ | preview.allium (no invariant), generation.allium (no invariant) | Generation uses only published .md file content (`Generation.Data` snapshots set `content: nil`); preview includes published+draft posts and prefers DB content over file (`Preview.Router` queries `:published`/`:draft`, uses `editor_body`) | **Resolved:** added `PreviewDraftOverlay` invariant to preview.allium and `GenerationPublishedOnly` invariant to generation.allium; both cross-reference each other; code already correct, 3 tests added for draft-in-preview behavior | ### A2. Spec Should Update (code is normative) @@ -184,7 +185,7 @@ All reconciled to follow code. Specs must be self-consistent and match code. ## Priority Order for Resolution -1. **A1-1 through A1-14** — code must follow spec (includes auto-save, on-demand preview, template lookup, validation gates, real Pagefind, graceful shutdown, real embedding model) +1. **A1-1 through A1-14b** — code must follow spec (includes auto-save, on-demand preview, template lookup, validation gates, real Pagefind, graceful shutdown, real embedding model; A1-14b = USearch HNSW index still open) 2. **D1-1 through D1-18** — untested invariants/guarantees 3. **C-1 through C-3** — internal spec inconsistencies (reconcile to code) 4. **B1-1 through B1-6** — major code behaviors missing from spec diff --git a/config/config.exs b/config/config.exs index e0f4897..c594b3f 100644 --- a/config/config.exs +++ b/config/config.exs @@ -61,10 +61,15 @@ config :bds, :scripting, job_max_reductions: :none config :bds, :embeddings, - backend: BDS.Embeddings.Backends.InApp, + backend: BDS.Embeddings.Backends.Neural, model_id: "Xenova/multilingual-e5-small", + model_repo: "intfloat/multilingual-e5-small", dimensions: 384 +# Cache downloaded model files under the app data directory so they persist +# across sessions (ModelCaching invariant). Overridden at runtime in prod. +config :bumblebee, :cache_dir, Path.expand("../priv/data/models", __DIR__) + config :logger, :console, format: "$time $metadata[$level] $message\n", metadata: [:request_id] diff --git a/config/runtime.exs b/config/runtime.exs index 4e61208..36ca560 100644 --- a/config/runtime.exs +++ b/config/runtime.exs @@ -8,4 +8,9 @@ if config_env() == :prod do config :bds, BDS.Repo, database: database_path, pool_size: String.to_integer(System.get_env("POOL_SIZE") || "1") + + # Persist downloaded embedding model files alongside the database data dir. + config :bumblebee, :cache_dir, + System.get_env("BDS_MODEL_CACHE_DIR") || + Path.join(Path.dirname(Path.expand(database_path)), "models") end diff --git a/config/test.exs b/config/test.exs index c32ea51..ebafef0 100644 --- a/config/test.exs +++ b/config/test.exs @@ -8,3 +8,11 @@ config :bds, BDS.Repo, busy_timeout: 15_000 config :logger, level: :warning + +# Tests use the deterministic lexical stub backend so the suite stays offline +# and never downloads the ~100 MB neural model. +config :bds, :embeddings, + backend: BDS.Embeddings.Backends.InApp, + model_id: "Xenova/multilingual-e5-small", + model_repo: "intfloat/multilingual-e5-small", + dimensions: 384 diff --git a/lib/bds/application.ex b/lib/bds/application.ex index 05a7d31..129c19f 100644 --- a/lib/bds/application.ex +++ b/lib/bds/application.ex @@ -38,13 +38,22 @@ defmodule BDS.Application do BDS.Scripting.JobStore, {Task.Supervisor, name: BDS.Scripting.TaskSupervisor}, BDS.Scripting.JobSupervisor - | desktop_children(current_env()) - ] + ] ++ embedding_children() ++ desktop_children(current_env()) opts = [strategy: :one_for_one, name: BDS.Supervisor] Supervisor.start_link(children, opts) end + # The neural embedding backend runs as a supervised, lazily-initialised + # GenServer (it loads the model only on the first embedding request). Only + # start it when it is the configured backend. + defp embedding_children do + case Application.get_env(:bds, :embeddings, [])[:backend] do + BDS.Embeddings.Backends.Neural -> [BDS.Embeddings.Backends.Neural] + _other -> [] + end + end + defp current_env do Application.get_env(:bds, :current_env_override) || @compiled_env end diff --git a/lib/bds/embeddings.ex b/lib/bds/embeddings.ex index a4ecd95..30af2c1 100644 --- a/lib/bds/embeddings.ex +++ b/lib/bds/embeddings.ex @@ -217,7 +217,7 @@ defmodule BDS.Embeddings do post_id: post.id, project_id: post.project_id, content_hash: content_hash, - vector: Jason.encode!(vector) + vector: encode_vector(vector) }) |> Repo.insert_or_update() @@ -256,7 +256,7 @@ defmodule BDS.Embeddings do else {:ok, vector} = embed_text(raw_text, post.language) label = if existing_key, do: existing_key.label, else: next_label - {:upsert, [label, post.id, post.project_id, content_hash, Jason.encode!(vector)]} + {:upsert, [label, post.id, post.project_id, content_hash, encode_vector(vector)]} end end @@ -655,7 +655,9 @@ defmodule BDS.Embeddings do end defp embed_text(raw_text, language) do - configured_backend().embed("query: " <> raw_text, language: language) + # Per-backend preprocessing (e5 "query: " prefix, pooling, normalisation) + # is the backend's responsibility — see BDS.Embeddings.Backends.Neural. + configured_backend().embed(raw_text, language: language) end defp rebuild_snapshot(project_id) do @@ -726,8 +728,22 @@ defmodule BDS.Embeddings do defp hash_text(text), do: :crypto.hash(:sha256, text) |> Base.encode16(case: :lower) + # Vectors are persisted as a packed little-endian Float32 BLOB + # (`dimensions` * 4 bytes; 1536 bytes for multilingual-e5-small) per the + # VectorCacheInDb invariant in specs/embedding.allium. + defp encode_vector(values) when is_list(values) do + for value <- values, into: <<>>, do: <<float32(value)::float-32-little>> + end + + defp float32(value) when is_float(value), do: value + defp float32(value) when is_integer(value), do: value * 1.0 + defp decode_vector(nil), do: [] - defp decode_vector(vector), do: Jason.decode!(vector) + defp decode_vector(<<>>), do: [] + + defp decode_vector(binary) when is_binary(binary) do + for <<value::float-32-little <- binary>>, do: value + end defp cosine_similarity([], _other), do: 0.0 defp cosine_similarity(_vector, []), do: 0.0 diff --git a/lib/bds/embeddings/backends/in_app.ex b/lib/bds/embeddings/backends/in_app.ex index d1b20c7..0b5eb5f 100644 --- a/lib/bds/embeddings/backends/in_app.ex +++ b/lib/bds/embeddings/backends/in_app.ex @@ -1,5 +1,13 @@ defmodule BDS.Embeddings.Backends.InApp do - @moduledoc false + @moduledoc """ + Deterministic lexical embedding stub. + + This backend does NOT satisfy the `RealNeuralModel` invariant — it projects + stemmed tokens and bigrams into a sparse hashed vector. It exists only as an + offline, dependency-free fallback for tests and environments where the neural + model (see `BDS.Embeddings.Backends.Neural`) cannot be loaded. Production and + development use the neural backend. + """ @behaviour BDS.Embeddings.Backend diff --git a/lib/bds/embeddings/backends/neural.ex b/lib/bds/embeddings/backends/neural.ex new file mode 100644 index 0000000..767267a --- /dev/null +++ b/lib/bds/embeddings/backends/neural.ex @@ -0,0 +1,104 @@ +defmodule BDS.Embeddings.Backends.Neural do + @moduledoc """ + Real on-device neural embedding backend. + + Implements the `RealNeuralModel` and `ModelCaching` invariants from + `specs/embedding.allium`: embeddings are produced by the actual + multilingual-e5-small transformer (the `intfloat/multilingual-e5-small` + weights behind the `Xenova/multilingual-e5-small` identifier) via + Bumblebee + EXLA, never by a lexical approximation. + + * Lazy-loaded — the model pipeline is built on the first embedding + request, not at application startup. + * Model files (~100 MB) are downloaded from the Hugging Face Hub on + first use and cached on disk (Bumblebee cache dir), persisting across + sessions and project switches. + * Text preprocessing follows the e5 convention: every input is prefixed + with `"query: "`, pooled with mean pooling over the attention mask, and + L2-normalised. This is what makes cross-language semantic similarity + work. + """ + + @behaviour BDS.Embeddings.Backend + + use GenServer + + @query_prefix "query: " + @embed_timeout :timer.minutes(2) + + @default_model_id "Xenova/multilingual-e5-small" + @default_model_repo "intfloat/multilingual-e5-small" + @default_dimensions 384 + + def child_spec(opts) do + %{id: __MODULE__, start: {__MODULE__, :start_link, [opts]}} + end + + def start_link(opts \\ []) do + GenServer.start_link(__MODULE__, opts, name: __MODULE__) + end + + @impl BDS.Embeddings.Backend + def model_info do + config = config() + + %{ + model_id: Keyword.get(config, :model_id, @default_model_id), + dimensions: Keyword.get(config, :dimensions, @default_dimensions) + } + end + + @impl BDS.Embeddings.Backend + def embed(text, _opts) when is_binary(text) do + GenServer.call(__MODULE__, {:embed, @query_prefix <> text}, @embed_timeout) + catch + :exit, reason -> {:error, {:embedding_backend_unavailable, reason}} + end + + @impl GenServer + def init(_opts), do: {:ok, %{serving: nil}} + + @impl GenServer + def handle_call({:embed, text}, _from, state) do + case ensure_serving(state) do + {:ok, %{serving: serving} = next_state} -> + %{embedding: tensor} = Nx.Serving.run(serving, text) + {:reply, {:ok, Nx.to_flat_list(tensor)}, next_state} + + {:error, _reason} = error -> + {:reply, error, state} + end + rescue + exception -> + {:reply, {:error, Exception.message(exception)}, state} + end + + defp ensure_serving(%{serving: nil} = state) do + case build_serving() do + {:ok, serving} -> {:ok, %{state | serving: serving}} + {:error, _reason} = error -> error + end + end + + defp ensure_serving(state), do: {:ok, state} + + defp build_serving do + repo = {:hf, Keyword.get(config(), :model_repo, @default_model_repo)} + + with {:ok, model_info} <- Bumblebee.load_model(repo), + {:ok, tokenizer} <- Bumblebee.load_tokenizer(repo) do + serving = + Bumblebee.Text.text_embedding(model_info, tokenizer, + output_pool: :mean_pooling, + output_attribute: :hidden_state, + embedding_processor: :l2_norm, + compile: [batch_size: 1, sequence_length: 512], + defn_options: [compiler: EXLA] + ) + + {:ok, serving} + end + end + + defp config, do: Application.get_env(:bds, :embeddings, []) +end diff --git a/lib/bds/embeddings/index.ex b/lib/bds/embeddings/index.ex index 00bb7d7..83447ee 100644 --- a/lib/bds/embeddings/index.ex +++ b/lib/bds/embeddings/index.ex @@ -192,8 +192,14 @@ defmodule BDS.Embeddings.Index do Path.join(Path.dirname(snapshot_path), "embeddings.index.json") end + # Vectors are stored as a packed little-endian Float32 BLOB; see + # BDS.Embeddings and the VectorCacheInDb invariant in embedding.allium. defp decode_vector(nil), do: [] - defp decode_vector(vector), do: Jason.decode!(vector) + defp decode_vector(<<>>), do: [] + + defp decode_vector(binary) when is_binary(binary) do + for <<value::float-32-little <- binary>>, do: value + end defp cosine_similarity([], _other), do: 0.0 defp cosine_similarity(_vector, []), do: 0.0 diff --git a/lib/bds/embeddings/key.ex b/lib/bds/embeddings/key.ex index 547d5c4..d4eecd3 100644 --- a/lib/bds/embeddings/key.ex +++ b/lib/bds/embeddings/key.ex @@ -12,7 +12,9 @@ defmodule BDS.Embeddings.Key do belongs_to :project, BDS.Projects.Project, type: :string field :content_hash, :string - field :vector, :string + # Packed little-endian Float32 BLOB (dimensions * 4 bytes), per the + # VectorCacheInDb invariant in specs/embedding.allium. + field :vector, :binary end def changeset(key, attrs) do diff --git a/mix.exs b/mix.exs index c8fd342..2900eb7 100644 --- a/mix.exs +++ b/mix.exs @@ -33,7 +33,10 @@ defmodule BDS.MixProject do {:plug, "~> 1.18"}, {:bandit, "~> 1.5"}, {:desktop, "~> 1.5"}, - {:image, "~> 0.65"}, + {:image, "~> 0.67"}, + {:nx, "~> 0.10"}, + {:exla, "~> 0.10"}, + {:bumblebee, "~> 0.6.3"}, {:stemex, "~> 0.2.1"}, {:gettext, "~> 0.24"}, {:tailwind, "~> 0.3", runtime: Mix.env() == :dev}, @@ -60,7 +63,7 @@ defmodule BDS.MixProject do env = Mix.env() [ - plt_add_apps: [:mix, :inets, :ssl], + plt_add_apps: [:mix, :inets, :ssl, :nx, :exla, :bumblebee], paths: ["_build/#{env}/lib/bds/ebin"] ] end diff --git a/mix.lock b/mix.lock index 5ea2aa7..70e3eb0 100644 --- a/mix.lock +++ b/mix.lock @@ -1,12 +1,16 @@ %{ - "bandit": {:hex, :bandit, "1.10.4", "02b9734c67c5916a008e7eb7e2ba68aaea6f8177094a5f8d95f1fb99069aac17", [:mix], [{:hpax, "~> 1.0", [hex: :hpax, repo: "hexpm", optional: false]}, {:plug, "~> 1.18", [hex: :plug, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:thousand_island, "~> 1.0", [hex: :thousand_island, repo: "hexpm", optional: false]}, {:websock, "~> 0.5", [hex: :websock, repo: "hexpm", optional: false]}], "hexpm", "a5faf501042ac1f31d736d9d4a813b3db4ef812e634583b6a457b0928798a51d"}, + "axon": {:hex, :axon, "0.7.0", "2e2c6d93b4afcfa812566b8922204fa022b60081e86ebd411df4db7ea30f5457", [:mix], [{:kino, "~> 0.7", [hex: :kino, repo: "hexpm", optional: true]}, {:kino_vega_lite, "~> 0.1.7", [hex: :kino_vega_lite, repo: "hexpm", optional: true]}, {:nx, "~> 0.9", [hex: :nx, repo: "hexpm", optional: false]}, {:polaris, "~> 0.1", [hex: :polaris, repo: "hexpm", optional: false]}, {:table_rex, "~> 3.1.1", [hex: :table_rex, repo: "hexpm", optional: true]}], "hexpm", "ee9857a143c9486597ceff434e6ca833dc1241be6158b01025b8217757ed1036"}, + "bandit": {:hex, :bandit, "1.11.1", "1eb33123cc3c17ae0c3447874eb83399ee530f960c39711ed240342fbd4865fa", [:mix], [{:hpax, "~> 1.0", [hex: :hpax, repo: "hexpm", optional: false]}, {:plug, "~> 1.18", [hex: :plug, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:thousand_island, "~> 1.0", [hex: :thousand_island, repo: "hexpm", optional: false]}, {:websock, "~> 0.5", [hex: :websock, repo: "hexpm", optional: false]}], "hexpm", "d4401016df9abbc6dcd325c0b78b2b193e7c7c96bb68f31e576112be025d84a5"}, + "bumblebee": {:hex, :bumblebee, "0.6.3", "c0028643c92de93258a9804da1d4d48797eaf7911b702464b3b3dd2cc7f938f1", [:mix], [{:axon, "~> 0.7.0", [hex: :axon, repo: "hexpm", optional: false]}, {:jason, "~> 1.4.0", [hex: :jason, repo: "hexpm", optional: false]}, {:nx, "~> 0.9.0 or ~> 0.10.0", [hex: :nx, repo: "hexpm", optional: false]}, {:nx_image, "~> 0.1.0", [hex: :nx_image, repo: "hexpm", optional: false]}, {:nx_signal, "~> 0.2.0", [hex: :nx_signal, repo: "hexpm", optional: false]}, {:progress_bar, "~> 3.0", [hex: :progress_bar, repo: "hexpm", optional: false]}, {:safetensors, "~> 0.1.3", [hex: :safetensors, repo: "hexpm", optional: false]}, {:tokenizers, "~> 0.4", [hex: :tokenizers, repo: "hexpm", optional: false]}, {:unpickler, "~> 0.1.0", [hex: :unpickler, repo: "hexpm", optional: false]}, {:unzip, "~> 0.12.0", [hex: :unzip, repo: "hexpm", optional: false]}], "hexpm", "c619197787561f8e5fb2ffba269c341654accaec9d591999b7fddd55761dd079"}, + "castore": {:hex, :castore, "1.0.19", "6903cabdfd9d1af46454126e7c8385186659dd33ecfb74a885cae52221ad6109", [:mix], [], "hexpm", "3669e6cab13f54c2df26b3e6833745d647f35b6e30d8ddd5975df0d5c842ca98"}, "cc_precompiler": {:hex, :cc_precompiler, "0.1.11", "8c844d0b9fb98a3edea067f94f616b3f6b29b959b6b3bf25fee94ffe34364768", [:mix], [{:elixir_make, "~> 0.7", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm", "3427232caf0835f94680e5bcf082408a70b48ad68a5f5c0b02a3bea9f3a075b9"}, - "color": {:hex, :color, "0.12.0", "f59f9bb6452a460760d44116ec0c1cf86f9d7707c8756c01f83c6d8fe042ae67", [:mix], [{:bandit, "~> 1.5", [hex: :bandit, repo: "hexpm", optional: true]}, {:plug, "~> 1.15", [hex: :plug, repo: "hexpm", optional: true]}], "hexpm", "1e17768919dad0bd44f48d0daf294d24bdd5a615bbfe0b4e01a51312203bd294"}, + "color": {:hex, :color, "0.13.0", "068110e5397ac5d3c9f97658282e0f4ab9a32468be6d7a2a91a8804e67b228d7", [:mix], [{:bandit, "~> 1.5", [hex: :bandit, repo: "hexpm", optional: true]}, {:plug, "~> 1.15", [hex: :plug, repo: "hexpm", optional: true]}], "hexpm", "de127946869931d418bac2d82dc29feae1a8f5f729f135922fbccf0059a58ab2"}, + "complex": {:hex, :complex, "0.7.0", "695632ef9487517aa5d57edd1697801079d622414cb2e1a7cf538b1f9a50f205", [:mix], [], "hexpm", "0ee39c0803129f546e7f3f640da8f021c9e659402bf59da6f7f2c4848f068f8d"}, "date_time_parser": {:hex, :date_time_parser, "1.3.0", "6ba16850b5ab83dd126576451023ab65349e29af2336ca5084aa1e37025b476e", [:mix], [{:kday, "~> 1.0", [hex: :kday, repo: "hexpm", optional: false]}], "hexpm", "93c8203a8ddc66b1f1531fc0e046329bf0b250c75ffa09567ef03d2c09218e8c"}, "db_connection": {:hex, :db_connection, "2.9.0", "a6a97c5c958a2d7091a58a9be40caf41ab496b0701d21e1d1abff3fa27a7f371", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "17d502eacaf61829db98facf6f20808ed33da6ccf495354a41e64fe42f9c509c"}, "dbus": {:hex, :dbus, "0.8.0", "7c800681f35d909c199265e55a8ee4aea9ebe4acccce77a0740f89f29cc57648", [:make], [], "hexpm", "a9784f2d9717ffa1f74169144a226c39633ac0d9c7fe8cb3594aeb89c827cca5"}, "debouncer": {:hex, :debouncer, "0.1.13", "af5906b231c196943ac8386b5b5f45a2f36d54a8bcd7e1b29eef2671de33d287", [:mix], [], "hexpm", "a14f57420c7d4a287f8f08e715fc8759b5d28dcd1032f9585d57c45d22123382"}, - "decimal": {:hex, :decimal, "2.3.0", "3ad6255aa77b4a3c4f818171b12d237500e63525c2fd056699967a3e7ea20f62", [:mix], [], "hexpm", "a4d66355cb29cb47c3cf30e71329e58361cfcb37c34235ef3bf1d7bf3773aeac"}, + "decimal": {:hex, :decimal, "2.4.1", "6c0fbede12fb122ba685e9ab41c6a40c129e322b3aa192f9e072e61f3a6ffaf2", [:mix], [], "hexpm", "7e618897933a8455f19a727d7c5e50a2c071a544b700e5e724298ecb4340187f"}, "desktop": {:hex, :desktop, "1.5.3", "dcf875dcff5b49a54646b4e6964acb079545c8c9c3790799aa5f1ccdcd314d15", [:mix], [{:debouncer, "~> 0.1", [hex: :debouncer, repo: "hexpm", optional: false]}, {:ex_sni, "~> 0.2", [hex: :ex_sni, repo: "hexpm", optional: false]}, {:gettext, "> 0.10.0", [hex: :gettext, repo: "hexpm", optional: false]}, {:oncrash, "~> 0.1", [hex: :oncrash, repo: "hexpm", optional: false]}, {:phoenix, "> 1.0.0", [hex: :phoenix, repo: "hexpm", optional: false]}, {:phoenix_live_view, "> 0.15.0", [hex: :phoenix_live_view, repo: "hexpm", optional: false]}, {:plug, "> 1.0.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm", "3750aabb8ed8aaf09b33f3cad5bda20f8ce4dfa65b026c019baed99c5264e2aa"}, "dialyxir": {:hex, :dialyxir, "1.4.7", "dda948fcee52962e4b6c5b4b16b2d8fa7d50d8645bbae8b8685c3f9ecb7f5f4d", [:mix], [{:erlex, ">= 0.2.8", [hex: :erlex, repo: "hexpm", optional: false]}], "hexpm", "b34527202e6eb8cee198efec110996c25c5898f43a4094df157f8d28f27d9efe"}, "earmark": {:hex, :earmark, "1.4.48", "5f41e579d85ef812351211842b6e005f6e0cef111216dea7d4b9d58af4608434", [:mix], [], "hexpm", "a461a0ddfdc5432381c876af1c86c411fd78a25790c75023c7a4c035fdc858f9"}, @@ -19,6 +23,7 @@ "ex_dbus": {:hex, :ex_dbus, "0.1.4", "053df83d45b27ba0b9b6ef55a47253922069a3ace12a2a7dd30d3aff58301e17", [:mix], [{:dbus, "~> 0.8.0", [hex: :dbus, repo: "hexpm", optional: false]}, {:saxy, "~> 1.4.0", [hex: :saxy, repo: "hexpm", optional: false]}], "hexpm", "d8baeaf465eab57b70a47b70e29fdfef6eb09ba110fc37176eebe6ac7874d6d5"}, "ex_sni": {:hex, :ex_sni, "0.2.9", "81f9421035dd3edb6d69f1a4dd5f53c7071b41628130d32ba5ab7bb4bfdc2da0", [:mix], [{:debouncer, "~> 0.1", [hex: :debouncer, repo: "hexpm", optional: false]}, {:ex_dbus, "~> 0.1", [hex: :ex_dbus, repo: "hexpm", optional: false]}, {:saxy, "~> 1.4.0", [hex: :saxy, repo: "hexpm", optional: false]}], "hexpm", "921d67d913765ed20ea8354fd1798dabc957bf66990a6842d6aaa7cd5ee5bc06"}, "ex_stemmers": {:hex, :ex_stemmers, "0.1.0", "63a84ae3a6f0c28a1d75768411f0ae15cfe8462fb70589b60977aa1b04c9372d", [:mix], [{:rustler, "~> 0.32.1", [hex: :rustler, repo: "hexpm", optional: false]}], "hexpm", "498826e2188e502f41d1a15f3d90e7738f0d94747e197367f03a2a44c09167c0"}, + "exla": {:hex, :exla, "0.10.0", "93e7d75a774fbc06ce05b96de20c4b01bda413b315238cb3c727c09a05d2bc3a", [:make, :mix], [{:elixir_make, "~> 0.6", [hex: :elixir_make, repo: "hexpm", optional: false]}, {:fine, "~> 0.1.0", [hex: :fine, repo: "hexpm", optional: false]}, {:nimble_pool, "~> 1.0", [hex: :nimble_pool, repo: "hexpm", optional: false]}, {:nx, "~> 0.10.0", [hex: :nx, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:xla, "~> 0.9.0", [hex: :xla, repo: "hexpm", optional: false]}], "hexpm", "16fffdb64667d7f0a3bc683fdcd2792b143a9b345e4b1f1d5cd50330c63d8119"}, "expo": {:hex, :expo, "1.1.1", "4202e1d2ca6e2b3b63e02f69cfe0a404f77702b041d02b58597c00992b601db5", [:mix], [], "hexpm", "5fb308b9cb359ae200b7e23d37c76978673aa1b06e2b3075d814ce12c5811640"}, "exqlite": {:hex, :exqlite, "0.36.0", "07b4f95d61cb82b8d52946d0639497fa7d32117e09b2c8d25e24a38723c295cb", [:make, :mix], [{:cc_precompiler, "~> 0.1", [hex: :cc_precompiler, repo: "hexpm", optional: false]}, {:db_connection, "~> 2.1", [hex: :db_connection, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.8", [hex: :elixir_make, repo: "hexpm", optional: false]}, {:table, "~> 0.1.0", [hex: :table, repo: "hexpm", optional: true]}], "hexpm", "cbeca3ce781f9ff07cfa9a87486f3ebd512a143ad6a14ed5c9fca21fe0bf3ae7"}, "fine": {:hex, :fine, "0.1.6", "4bf7151493443c454aac9f2fa2f34f5fefd0346a83fb5586a016c4a135c63247", [:mix], [], "hexpm", "5638eb4495488e885ebec167fa57973e5c35e1a50c344eb7666c90ec1c4e3b12"}, @@ -26,8 +31,8 @@ "hpax": {:hex, :hpax, "1.0.3", "ed67ef51ad4df91e75cc6a1494f851850c0bd98ebc0be6e81b026e765ee535aa", [:mix], [], "hexpm", "8eab6e1cfa8d5918c2ce4ba43588e894af35dbd8e91e6e55c817bca5847df34a"}, "html_entities": {:hex, :html_entities, "0.5.2", "9e47e70598da7de2a9ff6af8758399251db6dbb7eebe2b013f2bbd2515895c3c", [:mix], [], "hexpm", "c53ba390403485615623b9531e97696f076ed415e8d8058b1dbaa28181f4fdcc"}, "html_sanitize_ex": {:hex, :html_sanitize_ex, "1.4.4", "271455b4d300d5d53a5d92b5bd1c00ad14c5abf1c9ff87be069af5736496515c", [:mix], [{:mochiweb, "~> 2.15 or ~> 3.1", [hex: :mochiweb, repo: "hexpm", optional: false]}], "hexpm", "12e1754204e7db5df1750df0a5dba1bbdf89260800019ab081f2b046596be56b"}, - "image": {:hex, :image, "0.65.0", "44908233a1a0dcdbb6ae873ec09fd9ae533d1840d300d8b0b1b186d586b935e6", [:mix], [{:color, "~> 0.4", [hex: :color, repo: "hexpm", optional: false]}, {:evision, "~> 0.1.33 or ~> 0.2", [hex: :evision, repo: "hexpm", optional: true]}, {:exla, "0.11.0", [hex: :exla, repo: "hexpm", optional: true]}, {:jason, "~> 1.4", [hex: :jason, repo: "hexpm", optional: true]}, {:kino, "~> 0.13", [hex: :kino, repo: "hexpm", optional: true]}, {:nx, "~> 0.11.0", [hex: :nx, repo: "hexpm", optional: true]}, {:nx_image, "~> 0.1", [hex: :nx_image, repo: "hexpm", optional: true]}, {:phoenix_html, "~> 2.1 or ~> 3.2 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: false]}, {:plug, "~> 1.13", [hex: :plug, repo: "hexpm", optional: true]}, {:req, "~> 0.4", [hex: :req, repo: "hexpm", optional: true]}, {:rustler, "> 0.0.0", [hex: :rustler, repo: "hexpm", optional: true]}, {:scholar, "~> 0.3", [hex: :scholar, repo: "hexpm", optional: true]}, {:sweet_xml, "~> 0.7", [hex: :sweet_xml, repo: "hexpm", optional: false]}, {:vix, "~> 0.33", [hex: :vix, repo: "hexpm", optional: false]}, {:xav, "~> 0.10", [hex: :xav, repo: "hexpm", optional: true]}], "hexpm", "d2060e08d0f42564f49de1ea97a82a5d237f9ac91edb141dece51f1238dd8b4a"}, - "jason": {:hex, :jason, "1.4.4", "b9226785a9aa77b6857ca22832cffa5d5011a667207eb2a0ad56adb5db443b8a", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "c5eb0cab91f094599f94d55bc63409236a8ec69a21a67814529e8d5f6cc90b3b"}, + "image": {:hex, :image, "0.67.0", "886325f45bd39f3d705d32f223680163f3eaba142526d34f7f871c2232577e64", [:mix], [{:color, "~> 0.13", [hex: :color, repo: "hexpm", optional: false]}, {:evision, "~> 0.1.33 or ~> 0.2", [hex: :evision, repo: "hexpm", optional: true]}, {:exla, "~> 0.10", [hex: :exla, repo: "hexpm", optional: true]}, {:kino, "~> 0.13", [hex: :kino, repo: "hexpm", optional: true]}, {:nx, "~> 0.10", [hex: :nx, repo: "hexpm", optional: true]}, {:nx_image, "~> 0.1", [hex: :nx_image, repo: "hexpm", optional: true]}, {:phoenix_html, "~> 2.1 or ~> 3.2 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: false]}, {:plug, "~> 1.13", [hex: :plug, repo: "hexpm", optional: true]}, {:req, "~> 0.4", [hex: :req, repo: "hexpm", optional: true]}, {:rustler, "> 0.0.0", [hex: :rustler, repo: "hexpm", optional: true]}, {:scholar, "~> 0.3", [hex: :scholar, repo: "hexpm", optional: true]}, {:sweet_xml, "~> 0.7", [hex: :sweet_xml, repo: "hexpm", optional: false]}, {:vix, "~> 0.33", [hex: :vix, repo: "hexpm", optional: false]}, {:xav, "~> 0.10", [hex: :xav, repo: "hexpm", optional: true]}], "hexpm", "401c3e13137af8932eee377ad8bc8a8ae1a8343894266543e8bedd36b414c999"}, + "jason": {:hex, :jason, "1.4.5", "2e3a008590b0b8d7388c20293e9dcc9cf3e5d642fd2a114e4cbbb52e595d940a", [:mix], [{:decimal, "~> 1.0 or ~> 2.0 or ~> 3.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "b0c823996102bcd0239b3c2444eb00409b72f6a140c1950bc8b457d836b30684"}, "kday": {:hex, :kday, "1.1.0", "64efac85279a12283eaaf3ad6f13001ca2dff943eda8c53288179775a8c057a0", [:mix], [{:ex_doc, "~> 0.21", [hex: :ex_doc, repo: "hexpm", optional: true]}], "hexpm", "69703055d63b8d5b260479266c78b0b3e66f7aecdd2022906cd9bf09892a266d"}, "lazy_html": {:hex, :lazy_html, "0.1.11", "136c8e9cd616b4f4e9c1562daa683880891120b759606dc4c3b6b18058ba5d79", [:make, :mix], [{:cc_precompiler, "~> 0.1", [hex: :cc_precompiler, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.9.0", [hex: :elixir_make, repo: "hexpm", optional: false]}, {:fine, "~> 0.1.0", [hex: :fine, repo: "hexpm", optional: false]}], "hexpm", "3b1be592929c31eca1a21673d25696e5c14cddfe922d9d1a3e3b48be4163883b"}, "liquex": {:hex, :liquex, "0.13.1", "49f90d0b85fb2908f2558f35cd49d78497fe77a895eb55b360889940e1d7afb9", [:mix], [{:date_time_parser, "~> 1.2", [hex: :date_time_parser, repo: "hexpm", optional: false]}, {:html_entities, "~> 0.5.2", [hex: :html_entities, repo: "hexpm", optional: false]}, {:html_sanitize_ex, "~> 1.4.3", [hex: :html_sanitize_ex, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.4", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "fbea5b9db264c1758a69bfafdcc8aaebcd56e168365bb9575392cd55d800108f"}, @@ -35,23 +40,35 @@ "mime": {:hex, :mime, "2.0.7", "b8d739037be7cd402aee1ba0306edfdef982687ee7e9859bee6198c1e7e2f128", [:mix], [], "hexpm", "6171188e399ee16023ffc5b76ce445eb6d9672e2e241d2df6050f3c771e80ccd"}, "mochiweb": {:hex, :mochiweb, "3.3.0", "2898ad0bfeee234e4cbae623c7052abc3ff0d73d499ba6e6ffef445b13ffd07a", [:rebar3], [], "hexpm", "aa85b777fb23e9972ebc424e40b5d35106f19bc998873e026dedd876df8ee50c"}, "nimble_parsec": {:hex, :nimble_parsec, "1.4.2", "8efba0122db06df95bfaa78f791344a89352ba04baedd3849593bfce4d0dc1c6", [:mix], [], "hexpm", "4b21398942dda052b403bbe1da991ccd03a053668d147d53fb8c4e0efe09c973"}, + "nimble_pool": {:hex, :nimble_pool, "1.1.0", "bf9c29fbdcba3564a8b800d1eeb5a3c58f36e1e11d7b7fb2e084a643f645f06b", [:mix], [], "hexpm", "af2e4e6b34197db81f7aad230c1118eac993acc0dae6bc83bac0126d4ae0813a"}, + "nx": {:hex, :nx, "0.10.0", "128e4a094cb790f663e20e1334b127c1f2a4df54edfb8b13c22757ec33133b4f", [:mix], [{:complex, "~> 0.6", [hex: :complex, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "3db8892c124aeee091df0e6fbf8e5bf1b81f502eb0d4f5ba63e6378ebcae7da4"}, + "nx_image": {:hex, :nx_image, "0.1.2", "0c6e3453c1dc30fc80c723a54861204304cebc8a89ed3b806b972c73ee5d119d", [:mix], [{:nx, "~> 0.4", [hex: :nx, repo: "hexpm", optional: false]}], "hexpm", "9161863c42405ddccb6dbbbeae078ad23e30201509cc804b3b3a7c9e98764b81"}, + "nx_signal": {:hex, :nx_signal, "0.2.0", "e1ca0318877b17c81ce8906329f5125f1e2361e4c4235a5baac8a95ee88ea98e", [:mix], [{:nx, "~> 0.6", [hex: :nx, repo: "hexpm", optional: false]}], "hexpm", "7247e5e18a177a59c4cb5355952900c62fdeadeb2bad02a9a34237b68744e2bb"}, "oncrash": {:hex, :oncrash, "0.1.0", "9cf4ae8eba4ea250b579470172c5e9b8c75418b2264de7dbcf42e408d62e30fb", [:mix], [], "hexpm", "6968e775491cd857f9b6ff940bf2574fd1c2fab84fa7e14d5f56c39174c00018"}, "phoenix": {:hex, :phoenix, "1.8.5", "919db335247e6d4891764dc3063415b0d2457641c5f9b3751b5df03d8e20bbcf", [:mix], [{:bandit, "~> 1.0", [hex: :bandit, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 2.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:phoenix_template, "~> 1.0", [hex: :phoenix_template, repo: "hexpm", optional: false]}, {:phoenix_view, "~> 2.0", [hex: :phoenix_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.7", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:plug_crypto, "~> 1.2 or ~> 2.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:websock_adapter, "~> 0.5.3", [hex: :websock_adapter, repo: "hexpm", optional: false]}], "hexpm", "83b2bb125127e02e9f475c8e3e92736325b5b01b0b9b05407bcb4083b7a32485"}, "phoenix_html": {:hex, :phoenix_html, "4.3.0", "d3577a5df4b6954cd7890c84d955c470b5310bb49647f0a114a6eeecc850f7ad", [:mix], [], "hexpm", "3eaa290a78bab0f075f791a46a981bbe769d94bc776869f4f3063a14f30497ad"}, "phoenix_live_view": {:hex, :phoenix_live_view, "1.1.28", "8a8e123d018025f756605a2fb02a4854f0d3cd7b207f710fef1fd5d9d72d0254", [:mix], [{:igniter, ">= 0.6.16 and < 1.0.0-0", [hex: :igniter, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:lazy_html, "~> 0.1.0", [hex: :lazy_html, repo: "hexpm", optional: true]}, {:phoenix, "~> 1.6.15 or ~> 1.7.0 or ~> 1.8.0-rc", [hex: :phoenix, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 3.3 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: false]}, {:phoenix_template, "~> 1.0", [hex: :phoenix_template, repo: "hexpm", optional: false]}, {:phoenix_view, "~> 2.0", [hex: :phoenix_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.15", [hex: :plug, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.2 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "24faad535b65089642c3a7d84088109dc58f49c1f1c5a978659855d643466353"}, "phoenix_pubsub": {:hex, :phoenix_pubsub, "2.2.0", "ff3a5616e1bed6804de7773b92cbccfc0b0f473faf1f63d7daf1206c7aeaaa6f", [:mix], [], "hexpm", "adc313a5bf7136039f63cfd9668fde73bba0765e0614cba80c06ac9460ff3e96"}, "phoenix_template": {:hex, :phoenix_template, "1.0.4", "e2092c132f3b5e5b2d49c96695342eb36d0ed514c5b252a77048d5969330d639", [:mix], [{:phoenix_html, "~> 2.14.2 or ~> 3.0 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: true]}], "hexpm", "2c0c81f0e5c6753faf5cca2f229c9709919aba34fab866d3bc05060c9c444206"}, - "plug": {:hex, :plug, "1.19.1", "09bac17ae7a001a68ae393658aa23c7e38782be5c5c00c80be82901262c394c0", [:mix], [{:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.1.1 or ~> 1.2 or ~> 2.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.3 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "560a0017a8f6d5d30146916862aaf9300b7280063651dd7e532b8be168511e62"}, + "plug": {:hex, :plug, "1.19.2", "e4950525b22c6789dfb38a3f95d47171ba159da3fc5a33be9643b43d5e8adb98", [:mix], [{:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.1.1 or ~> 1.2 or ~> 2.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.3 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "b6fce20a56af5e60fa5dfecf3f907bb98ec981be43c79a3809a499bc3d133de0"}, "plug_crypto": {:hex, :plug_crypto, "2.1.1", "19bda8184399cb24afa10be734f84a16ea0a2bc65054e23a62bb10f06bc89491", [:mix], [], "hexpm", "6470bce6ffe41c8bd497612ffde1a7e4af67f36a15eea5f921af71cf3e11247c"}, + "polaris": {:hex, :polaris, "0.1.0", "dca61b18e3e801ecdae6ac9f0eca5f19792b44a5cb4b8d63db50fc40fc038d22", [:mix], [{:nx, "~> 0.5", [hex: :nx, repo: "hexpm", optional: false]}], "hexpm", "13ef2b166650e533cb24b10e2f3b8ab4f2f449ba4d63156e8c569527f206e2c2"}, + "progress_bar": {:hex, :progress_bar, "3.0.0", "f54ff038c2ac540cfbb4c2bfe97c75e7116ead044f3c2b10c9f212452194b5cd", [:mix], [{:decimal, "~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}], "hexpm", "6981c2b25ab24aecc91a2dc46623658e1399c21a2ae24db986b90d678530f2b7"}, "rustler": {:hex, :rustler, "0.32.1", "f4cf5a39f9e85d182c0a3f75fa15b5d0add6542ab0bf9ceac6b4023109ebd3fc", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:toml, "~> 0.6", [hex: :toml, repo: "hexpm", optional: false]}], "hexpm", "b96be75526784f86f6587f051bc8d6f4eaff23d6e0f88dbcfe4d5871f52946f7"}, + "rustler_precompiled": {:hex, :rustler_precompiled, "0.9.0", "3a052eda09f3d2436364645cc1f13279cf95db310eb0c17b0d8f25484b233aa0", [:mix], [{:rustler, "~> 0.23", [hex: :rustler, repo: "hexpm", optional: true]}], "hexpm", "471d97315bd3bf7b64623418b3693eedd8e47de3d1cb79a0ac8f9da7d770d94c"}, + "safetensors": {:hex, :safetensors, "0.1.3", "7ff3c22391e213289c713898481d492c9c28a49ab1d0705b72630fb8360426b2", [:mix], [{:jason, "~> 1.4", [hex: :jason, repo: "hexpm", optional: false]}, {:nx, "~> 0.5", [hex: :nx, repo: "hexpm", optional: false]}], "hexpm", "fe50b53ea59fde4e723dd1a2e31cfdc6013e69343afac84c6be86d6d7c562c14"}, "saxy": {:hex, :saxy, "1.4.0", "c7203ad20001f72eaaad07d08f82be063fa94a40924e6bb39d93d55f979abcba", [:mix], [], "hexpm", "3fe790354d3f2234ad0b5be2d99822a23fa2d4e8ccd6657c672901dac172e9a9"}, "stemex": {:hex, :stemex, "0.2.1", "47017c6b10cdd6926a0d523ccf1f801c5f3faf5a0a9c862f49304e07f9b5584f", [:mix], [], "hexpm", "dbfc76d27adfa31d831d183979c595942884e6530a4496714aa5b70d0964c2e4"}, "sweet_xml": {:hex, :sweet_xml, "0.7.5", "803a563113981aaac202a1dbd39771562d0ad31004ddbfc9b5090bdcd5605277", [:mix], [], "hexpm", "193b28a9b12891cae351d81a0cead165ffe67df1b73fe5866d10629f4faefb12"}, "tailwind": {:hex, :tailwind, "0.4.1", "e7bcc222fe96a1e55f948e76d13dd84a1a7653fb051d2a167135db3b4b08d3e9", [:mix], [], "hexpm", "6249d4f9819052911120dbdbe9e532e6bd64ea23476056adb7f730aa25c220d1"}, - "telemetry": {:hex, :telemetry, "1.4.1", "ab6de178e2b29b58e8256b92b382ea3f590a47152ca3651ea857a6cae05ac423", [:rebar3], [], "hexpm", "2172e05a27531d3d31dd9782841065c50dd5c3c7699d95266b2edd54c2dafa1c"}, + "telemetry": {:hex, :telemetry, "1.4.2", "a0cb522801dffb1c49fe6e30561badffc7b6d0e180db1300df759faa22062855", [:rebar3], [], "hexpm", "928f6495066506077862c0d1646609eed891a4326bee3126ba54b60af61febb1"}, "thousand_island": {:hex, :thousand_island, "1.4.3", "2158209580f633be38d43ec4e3ce0a01079592b9657afff9080d5d8ca149a3af", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "6e4ce09b0fd761a58594d02814d40f77daff460c48a7354a15ab353bb998ea0b"}, + "tokenizers": {:hex, :tokenizers, "0.5.1", "b0975d92b4ee5b18e8f47b5d65b9d5f1e583d9130189b1a2620401af4e7d4b35", [:mix], [{:castore, "~> 0.1 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: false]}, {:rustler, ">= 0.0.0", [hex: :rustler, repo: "hexpm", optional: true]}, {:rustler_precompiled, "~> 0.6", [hex: :rustler_precompiled, repo: "hexpm", optional: false]}], "hexpm", "5f08d97cc7f2ed3d71d370d68120da6d3de010948ccf676c9c0eb591ba4bacc9"}, "toml": {:hex, :toml, "0.7.0", "fbcd773caa937d0c7a02c301a1feea25612720ac3fa1ccb8bfd9d30d822911de", [:mix], [], "hexpm", "0690246a2478c1defd100b0c9b89b4ea280a22be9a7b313a8a058a2408a2fa70"}, + "unpickler": {:hex, :unpickler, "0.1.0", "c2262c0819e6985b761e7107546cef96a485f401816be5304a65fdd200d5bd6a", [:mix], [], "hexpm", "e2b3f61e62406187ac52afead8a63bfb4e49394028993f3c4c42712743cab79e"}, + "unzip": {:hex, :unzip, "0.12.0", "beed92238724732418b41eba77dcb7f51e235b707406c05b1732a3052d1c0f36", [:mix], [], "hexpm", "95655b72db368e5a84951f0bed586ac053b55ee3815fd96062fce10ce4fc998d"}, "vix": {:hex, :vix, "0.38.0", "77529ee4f6ced339c3d5f90a9eacf306f5b7109d3d1b5e3ef391a984ad404f75", [:make, :mix], [{:cc_precompiler, "~> 0.1.4 or ~> 0.2", [hex: :cc_precompiler, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.7.3 or ~> 0.8", [hex: :elixir_make, repo: "hexpm", optional: false]}, {:kino, "~> 0.7", [hex: :kino, repo: "hexpm", optional: true]}], "hexpm", "dca58f654922fa678d5df8e028317483d9c0f8acb2e2714076a8468695687aa7"}, "websock": {:hex, :websock, "0.5.3", "2f69a6ebe810328555b6fe5c831a851f485e303a7c8ce6c5f675abeb20ebdadc", [:mix], [], "hexpm", "6105453d7fac22c712ad66fab1d45abdf049868f253cf719b625151460b8b453"}, "websock_adapter": {:hex, :websock_adapter, "0.5.9", "43dc3ba6d89ef5dec5b1d0a39698436a1e856d000d84bf31a3149862b01a287f", [:mix], [{:bandit, ">= 0.6.0", [hex: :bandit, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.6", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:websock, "~> 0.5", [hex: :websock, repo: "hexpm", optional: false]}], "hexpm", "5534d5c9adad3c18a0f58a9371220d75a803bf0b9a3d87e6fe072faaeed76a08"}, + "xla": {:hex, :xla, "0.9.1", "cca0040ff94902764007a118871bfc667f1a0085d4a5074533a47d6b58bec61e", [:make, :mix], [{:elixir_make, "~> 0.4", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm", "eb5e443ae5391b1953f253e051f2307bea183b59acee138053a9300779930daf"}, } diff --git a/priv/repo/migrations/20260529115329_convert_embedding_vector_to_blob.exs b/priv/repo/migrations/20260529115329_convert_embedding_vector_to_blob.exs new file mode 100644 index 0000000..4f84f23 --- /dev/null +++ b/priv/repo/migrations/20260529115329_convert_embedding_vector_to_blob.exs @@ -0,0 +1,33 @@ +defmodule BDS.Repo.Migrations.ConvertEmbeddingVectorToBlob do + use Ecto.Migration + + # Embedding vectors are now persisted as a packed little-endian Float32 BLOB + # (VectorCacheInDb invariant) instead of JSON text. The table is a rebuildable + # cache and the previous lexical vectors are incompatible with the neural + # model, so we drop and recreate it; rows are re-embedded on next index. + + def up do + drop table(:embedding_keys) + create_embedding_keys(:binary) + end + + def down do + drop table(:embedding_keys) + create_embedding_keys(:text) + end + + defp create_embedding_keys(vector_type) do + create table(:embedding_keys, primary_key: false) do + add :label, :integer, primary_key: true + add :post_id, references(:posts, column: :id, type: :string, on_delete: :delete_all), + null: false + + add :project_id, references(:projects, type: :string, on_delete: :delete_all), null: false + add :content_hash, :string, null: false + add :vector, vector_type + end + + create index(:embedding_keys, [:post_id]) + create index(:embedding_keys, [:project_id]) + end +end diff --git a/specs/embedding.allium b/specs/embedding.allium index 4021528..5432a00 100644 --- a/specs/embedding.allium +++ b/specs/embedding.allium @@ -48,6 +48,9 @@ value EmbeddingModel { -- Lazy-loaded: pipeline created on first embedding request, not at startup -- Text preprocessing: prefix all input with "query: " (e5 convention) -- Pooling: mean pooling + L2 normalization + -- Loaded on-device via Bumblebee+EXLA; the canonical e5 weights come from + -- the "intfloat/multilingual-e5-small" repository, surfaced under the + -- "Xenova/multilingual-e5-small" model_id identifier. model_id: String -- "Xenova/multilingual-e5-small" dimensions: Integer -- 384 } diff --git a/test/bds/embeddings/backends/neural_test.exs b/test/bds/embeddings/backends/neural_test.exs new file mode 100644 index 0000000..3690d56 --- /dev/null +++ b/test/bds/embeddings/backends/neural_test.exs @@ -0,0 +1,39 @@ +defmodule BDS.Embeddings.Backends.NeuralTest do + use ExUnit.Case, async: false + + alias BDS.Embeddings.Backends.Neural + + setup do + previous = Application.get_env(:bds, :embeddings) + + on_exit(fn -> + if previous == nil do + Application.delete_env(:bds, :embeddings) + else + Application.put_env(:bds, :embeddings, previous) + end + end) + + :ok + end + + test "reports the configured spec model id and dimensions without loading the model" do + Application.put_env(:bds, :embeddings, + backend: Neural, + model_id: "Xenova/multilingual-e5-small", + model_repo: "intfloat/multilingual-e5-small", + dimensions: 384 + ) + + assert %{model_id: "Xenova/multilingual-e5-small", dimensions: 384} = Neural.model_info() + end + + test "implements the embeddings backend behaviour" do + behaviours = + Neural.module_info(:attributes) + |> Keyword.get_values(:behaviour) + |> List.flatten() + + assert BDS.Embeddings.Backend in behaviours + end +end diff --git a/test/bds/embeddings_test.exs b/test/bds/embeddings_test.exs index 8a09595..90b5e4d 100644 --- a/test/bds/embeddings_test.exs +++ b/test/bds/embeddings_test.exs @@ -321,6 +321,36 @@ defmodule BDS.EmbeddingsTest do assert BDS.Embeddings.index_path(project.id) =~ "/embeddings.usearch" end + test "stored embedding vectors are packed Float32 BLOBs, not JSON text", %{project: project} do + assert {:ok, _metadata} = + BDS.Metadata.update_project_metadata(project.id, %{semantic_similarity_enabled: true}) + + assert {:ok, post} = + BDS.Posts.create_post(%{ + project_id: project.id, + title: "Blob", + content: "space rocket orbit mission galaxy", + language: "en" + }) + + assert {:ok, post} = BDS.Posts.publish_post(post.id) + assert {:ok, _indexed} = BDS.Embeddings.index_unindexed(project.id) + + key = BDS.Repo.get_by!(BDS.Embeddings.Key, project_id: project.id, post_id: post.id) + + assert is_binary(key.vector) + # 384 dimensions * 4 bytes per little-endian Float32 (VectorCacheInDb). + assert byte_size(key.vector) == 384 * 4 + refute String.starts_with?(key.vector, "[") + + decoded = for <<value::float-32-little <- key.vector>>, do: value + assert length(decoded) == 384 + + # The packed vector still drives similarity queries. + assert {:ok, scores} = BDS.Embeddings.compute_similarities(post.id, [post.id]) + assert is_map(scores) + end + test "reindex_all rebuilds stored embeddings for the whole project", %{project: project} do assert {:ok, _metadata} = BDS.Metadata.update_project_metadata(project.id, %{semantic_similarity_enabled: true})