chore: remove redundant export-only tests, add test audit procedure
Deleted chat_editor_test.exs and import_editor_test.exs which only checked function_exported?/Code.ensure_loaded? without exercising any behavior — both components are already tested via LiveView rendering in shell_live_test.exs and import_shell_live_test.exs respectively. Added TESTAUDIT.md documenting the procedure for periodic test suite audits to catch non-behavioral tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
87
TESTAUDIT.md
Normal file
87
TESTAUDIT.md
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
# Test Audit Procedure
|
||||||
|
|
||||||
|
Periodic review of the unit test suite to ensure every test exercises production
|
||||||
|
code against real assumptions and behavior.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
All `*_test.exs` files under `test/`.
|
||||||
|
|
||||||
|
## What counts as a valid unit test
|
||||||
|
|
||||||
|
A valid unit test **calls at least one production function** from `lib/bds/` and
|
||||||
|
**asserts on its return value, side effects, or observable behavior**.
|
||||||
|
|
||||||
|
Acceptable patterns:
|
||||||
|
|
||||||
|
- Calling a production function and asserting its return value.
|
||||||
|
- Calling a production function with injected test doubles (fake HTTP clients,
|
||||||
|
fake runtimes) and asserting the production code's orchestration logic.
|
||||||
|
- Mounting a LiveView or rendering a LiveComponent and asserting HTML output
|
||||||
|
or database state after interactions.
|
||||||
|
- Sending events to a GenServer and asserting state transitions.
|
||||||
|
|
||||||
|
### Source-property tests (acceptable, not flagged)
|
||||||
|
|
||||||
|
Tests that verify structural properties of source code are acceptable and should
|
||||||
|
not be flagged during this audit. Examples:
|
||||||
|
|
||||||
|
- Checking that all public functions have `@spec` annotations (AST parsing).
|
||||||
|
- Asserting absence of `String.to_atom` or `cond do` in specific files.
|
||||||
|
- Verifying CSS/JS/template assets contain expected class names or imports.
|
||||||
|
- Checking that `API.md` matches the output of a documentation generator.
|
||||||
|
- Verifying database indexes exist via `EXPLAIN QUERY PLAN`.
|
||||||
|
- Asserting `.allium` spec files have consistent parameter signatures.
|
||||||
|
- Checking config files for expected values.
|
||||||
|
- Verifying function decomposition patterns in source.
|
||||||
|
|
||||||
|
These are linting/contract/consistency checks. They serve a purpose but are
|
||||||
|
distinct from behavioral tests.
|
||||||
|
|
||||||
|
## What gets flagged
|
||||||
|
|
||||||
|
1. **Export-existence-only tests** — tests that call `function_exported?/3` or
|
||||||
|
`Code.ensure_loaded?/1` without ever invoking the function. These verify
|
||||||
|
compilation, not behavior. They are redundant when the same module is already
|
||||||
|
tested via rendering or direct calls in another test file.
|
||||||
|
|
||||||
|
2. **Mock-only tests** — tests that define a fake/stub module and only assert
|
||||||
|
on that fake's behavior without routing through any production code path.
|
||||||
|
|
||||||
|
3. **Trivially-passing tests** — tests whose assertions succeed regardless of
|
||||||
|
whether the production code is correct (e.g., asserting on a hardcoded value
|
||||||
|
that never touches production logic).
|
||||||
|
|
||||||
|
## How to run the audit
|
||||||
|
|
||||||
|
Ask Claude Code to:
|
||||||
|
|
||||||
|
> Analyse the unit tests of the project and check if all of them actually call
|
||||||
|
> proper production code or if there are tests that essentially only test
|
||||||
|
> scaffolds, mocks and helper functions. Every unit test must test proper
|
||||||
|
> production code against assumptions and behaviour. Source-property tests
|
||||||
|
> (structure, @spec, asset presence, schema verification, doc staleness) are
|
||||||
|
> acceptable and should not be flagged.
|
||||||
|
|
||||||
|
The audit should:
|
||||||
|
|
||||||
|
1. Read every `*_test.exs` file under `test/` in full.
|
||||||
|
2. For each test block, identify which production function (if any) is called.
|
||||||
|
3. Flag any test that falls into the categories above.
|
||||||
|
4. Report flagged tests with file path, line number, and explanation.
|
||||||
|
|
||||||
|
## Audit log
|
||||||
|
|
||||||
|
### 2026-05-11
|
||||||
|
|
||||||
|
Reviewed all 71 test files (69 after cleanup). Found 2 redundant files:
|
||||||
|
|
||||||
|
- `test/bds/desktop/shell_live/chat_editor_test.exs` — single test only called
|
||||||
|
`function_exported?` for `ChatEditor`. The component was already fully tested
|
||||||
|
via `render_component` in `shell_live_test.exs`. **Deleted.**
|
||||||
|
|
||||||
|
- `test/bds/desktop/shell_live/import_editor_test.exs` — single test only called
|
||||||
|
`Code.ensure_loaded?` + `function_exported?` for `ImportEditor`. The component
|
||||||
|
was already exercised in `import_shell_live_test.exs`. **Deleted.**
|
||||||
|
|
||||||
|
Result after cleanup: 646 tests, 0 failures, 4 skipped.
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
defmodule BDS.Desktop.ShellLive.ChatEditorTest do
|
|
||||||
use ExUnit.Case, async: false
|
|
||||||
|
|
||||||
test "ChatEditor exports LiveComponent callbacks" do
|
|
||||||
assert function_exported?(BDS.Desktop.ShellLive.ChatEditor, :update, 2)
|
|
||||||
assert function_exported?(BDS.Desktop.ShellLive.ChatEditor, :handle_event, 3)
|
|
||||||
assert function_exported?(BDS.Desktop.ShellLive.ChatEditor, :render, 1)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
defmodule BDS.Desktop.ShellLive.ImportEditorTest do
|
|
||||||
use ExUnit.Case, async: false
|
|
||||||
|
|
||||||
test "ImportEditor exports LiveComponent callbacks" do
|
|
||||||
module = BDS.Desktop.ShellLive.ImportEditor
|
|
||||||
assert Code.ensure_loaded?(module)
|
|
||||||
assert function_exported?(module, :update, 2)
|
|
||||||
assert function_exported?(module, :handle_event, 3)
|
|
||||||
assert function_exported?(module, :render, 1)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
Reference in New Issue
Block a user