From 99dc1c22162895455c4f9d19a61b6f7eb799e368 Mon Sep 17 00:00:00 2001 From: Chili Palmer Date: Mon, 11 May 2026 10:35:24 +0200 Subject: [PATCH] chore: remove redundant export-only tests, add test audit procedure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Deleted chat_editor_test.exs and import_editor_test.exs which only checked function_exported?/Code.ensure_loaded? without exercising any behavior — both components are already tested via LiveView rendering in shell_live_test.exs and import_shell_live_test.exs respectively. Added TESTAUDIT.md documenting the procedure for periodic test suite audits to catch non-behavioral tests. Co-Authored-By: Claude Opus 4.6 --- TESTAUDIT.md | 87 +++++++++++++++++++ .../desktop/shell_live/chat_editor_test.exs | 9 -- .../desktop/shell_live/import_editor_test.exs | 11 --- 3 files changed, 87 insertions(+), 20 deletions(-) create mode 100644 TESTAUDIT.md delete mode 100644 test/bds/desktop/shell_live/chat_editor_test.exs delete mode 100644 test/bds/desktop/shell_live/import_editor_test.exs diff --git a/TESTAUDIT.md b/TESTAUDIT.md new file mode 100644 index 0000000..c27bf3d --- /dev/null +++ b/TESTAUDIT.md @@ -0,0 +1,87 @@ +# Test Audit Procedure + +Periodic review of the unit test suite to ensure every test exercises production +code against real assumptions and behavior. + +## Scope + +All `*_test.exs` files under `test/`. + +## What counts as a valid unit test + +A valid unit test **calls at least one production function** from `lib/bds/` and +**asserts on its return value, side effects, or observable behavior**. + +Acceptable patterns: + +- Calling a production function and asserting its return value. +- Calling a production function with injected test doubles (fake HTTP clients, + fake runtimes) and asserting the production code's orchestration logic. +- Mounting a LiveView or rendering a LiveComponent and asserting HTML output + or database state after interactions. +- Sending events to a GenServer and asserting state transitions. + +### Source-property tests (acceptable, not flagged) + +Tests that verify structural properties of source code are acceptable and should +not be flagged during this audit. Examples: + +- Checking that all public functions have `@spec` annotations (AST parsing). +- Asserting absence of `String.to_atom` or `cond do` in specific files. +- Verifying CSS/JS/template assets contain expected class names or imports. +- Checking that `API.md` matches the output of a documentation generator. +- Verifying database indexes exist via `EXPLAIN QUERY PLAN`. +- Asserting `.allium` spec files have consistent parameter signatures. +- Checking config files for expected values. +- Verifying function decomposition patterns in source. + +These are linting/contract/consistency checks. They serve a purpose but are +distinct from behavioral tests. + +## What gets flagged + +1. **Export-existence-only tests** — tests that call `function_exported?/3` or + `Code.ensure_loaded?/1` without ever invoking the function. These verify + compilation, not behavior. They are redundant when the same module is already + tested via rendering or direct calls in another test file. + +2. **Mock-only tests** — tests that define a fake/stub module and only assert + on that fake's behavior without routing through any production code path. + +3. **Trivially-passing tests** — tests whose assertions succeed regardless of + whether the production code is correct (e.g., asserting on a hardcoded value + that never touches production logic). + +## How to run the audit + +Ask Claude Code to: + +> Analyse the unit tests of the project and check if all of them actually call +> proper production code or if there are tests that essentially only test +> scaffolds, mocks and helper functions. Every unit test must test proper +> production code against assumptions and behaviour. Source-property tests +> (structure, @spec, asset presence, schema verification, doc staleness) are +> acceptable and should not be flagged. + +The audit should: + +1. Read every `*_test.exs` file under `test/` in full. +2. For each test block, identify which production function (if any) is called. +3. Flag any test that falls into the categories above. +4. Report flagged tests with file path, line number, and explanation. + +## Audit log + +### 2026-05-11 + +Reviewed all 71 test files (69 after cleanup). Found 2 redundant files: + +- `test/bds/desktop/shell_live/chat_editor_test.exs` — single test only called + `function_exported?` for `ChatEditor`. The component was already fully tested + via `render_component` in `shell_live_test.exs`. **Deleted.** + +- `test/bds/desktop/shell_live/import_editor_test.exs` — single test only called + `Code.ensure_loaded?` + `function_exported?` for `ImportEditor`. The component + was already exercised in `import_shell_live_test.exs`. **Deleted.** + +Result after cleanup: 646 tests, 0 failures, 4 skipped. diff --git a/test/bds/desktop/shell_live/chat_editor_test.exs b/test/bds/desktop/shell_live/chat_editor_test.exs deleted file mode 100644 index 27b5161..0000000 --- a/test/bds/desktop/shell_live/chat_editor_test.exs +++ /dev/null @@ -1,9 +0,0 @@ -defmodule BDS.Desktop.ShellLive.ChatEditorTest do - use ExUnit.Case, async: false - - test "ChatEditor exports LiveComponent callbacks" do - assert function_exported?(BDS.Desktop.ShellLive.ChatEditor, :update, 2) - assert function_exported?(BDS.Desktop.ShellLive.ChatEditor, :handle_event, 3) - assert function_exported?(BDS.Desktop.ShellLive.ChatEditor, :render, 1) - end -end diff --git a/test/bds/desktop/shell_live/import_editor_test.exs b/test/bds/desktop/shell_live/import_editor_test.exs deleted file mode 100644 index 3f4d130..0000000 --- a/test/bds/desktop/shell_live/import_editor_test.exs +++ /dev/null @@ -1,11 +0,0 @@ -defmodule BDS.Desktop.ShellLive.ImportEditorTest do - use ExUnit.Case, async: false - - test "ImportEditor exports LiveComponent callbacks" do - module = BDS.Desktop.ShellLive.ImportEditor - assert Code.ensure_loaded?(module) - assert function_exported?(module, :update, 2) - assert function_exported?(module, :handle_event, 3) - assert function_exported?(module, :render, 1) - end -end