Return normalized token usage from one-shot AI calls #15

New Issue

hugo · 2026-07-18T20:22:03Z

hugo commented

2026-07-18 20:22:03 +00:00

Goal

Return provider token accounting with every successful core one-shot AI result.

Current state

crates/bds-core/src/engine/ai.rs::run_one_shot parses choices[0].message.content and ignores the response usage object.
The six one-shot operations already work: taxonomy analysis, image analysis, post analysis, language detection, post translation, and media translation.
A Diesel migration now adds nullable chat_messages.token_usage_input and chat_messages.token_usage_output beside the existing cache counters.
specs/ai.allium, UsageReturnedWithSuccessfulResult, is normative.

Required behavior

Every successful one-shot call returns its existing operation result plus a usage value with nullable:

input_tokens
output_tokens
cache_read_tokens
cache_write_tokens

Normalize OpenAI-compatible fields exactly as bDS2 does:

usage.prompt_tokens -> input_tokens
usage.completion_tokens -> output_tokens
usage.prompt_tokens_details.cached_tokens -> cache_read_tokens
usage.completion_tokens_details.cached_tokens -> cache_write_tokens

Missing usage or missing individual counters produces None for those counters and does not fail an otherwise valid AI result.

One-shot calls do not create a chat conversation and do not persist ChatMessage rows. Persisting normalized counters on assistant chat messages belongs to the extension chat implementation; the schema fields are present for that future work.

Implementation notes

Keep one shared usage parser at the HTTP response boundary and reuse it for all six operations.
Choose the smallest response-shape change that lets all callers access the usage without duplicating fields/parsing.
Update UI and engine callers without changing their current visible result behavior.
Follow red/green TDD.
Compare with ../bDS2/lib/bds/ai/openai_compatible_runtime.ex, one_shot.ex, and chat.ex when the spec is silent.

Acceptance criteria

Tests cover a complete usage object, a partial usage object, no usage object, and malformed/non-numeric counters.
All six operations return normalized usage while retaining their current payloads.
Existing callers continue to work and do not persist one-shot usage in chat tables.
Migration tests prove input, output, cache-read, and cache-write chat columns are nullable and writable.
allium check specs/*.allium, cargo fmt --all -- --check, cargo build --workspace, and cargo test --workspace pass.

## Goal Return provider token accounting with every successful core one-shot AI result. ## Current state - `crates/bds-core/src/engine/ai.rs::run_one_shot` parses `choices[0].message.content` and ignores the response `usage` object. - The six one-shot operations already work: taxonomy analysis, image analysis, post analysis, language detection, post translation, and media translation. - A Diesel migration now adds nullable `chat_messages.token_usage_input` and `chat_messages.token_usage_output` beside the existing cache counters. - `specs/ai.allium`, `UsageReturnedWithSuccessfulResult`, is normative. ## Required behavior Every successful one-shot call returns its existing operation result plus a usage value with nullable: - `input_tokens` - `output_tokens` - `cache_read_tokens` - `cache_write_tokens` Normalize OpenAI-compatible fields exactly as bDS2 does: - `usage.prompt_tokens` -> `input_tokens` - `usage.completion_tokens` -> `output_tokens` - `usage.prompt_tokens_details.cached_tokens` -> `cache_read_tokens` - `usage.completion_tokens_details.cached_tokens` -> `cache_write_tokens` Missing usage or missing individual counters produces `None` for those counters and does not fail an otherwise valid AI result. One-shot calls do not create a chat conversation and do not persist `ChatMessage` rows. Persisting normalized counters on assistant chat messages belongs to the extension chat implementation; the schema fields are present for that future work. ## Implementation notes - Keep one shared usage parser at the HTTP response boundary and reuse it for all six operations. - Choose the smallest response-shape change that lets all callers access the usage without duplicating fields/parsing. - Update UI and engine callers without changing their current visible result behavior. - Follow red/green TDD. - Compare with `../bDS2/lib/bds/ai/openai_compatible_runtime.ex`, `one_shot.ex`, and `chat.ex` when the spec is silent. ## Acceptance criteria - Tests cover a complete usage object, a partial usage object, no usage object, and malformed/non-numeric counters. - All six operations return normalized usage while retaining their current payloads. - Existing callers continue to work and do not persist one-shot usage in chat tables. - Migration tests prove input, output, cache-read, and cache-write chat columns are nullable and writable. - `allium check specs/*.allium`, `cargo fmt --all -- --check`, `cargo build --workspace`, and `cargo test --workspace` pass.

hugo added the enhancement label 2026-07-18 20:22:03 +00:00

hugo referenced this issue from a commit

2026-07-18 21:15:05 +00:00

feat: return normalized token usage from one-shot AI calls (closes #15)

hugo closed this issue

2026-07-18 21:15:05 +00:00

hugo commented

2026-07-18 21:15:15 +00:00

Implemented in 33929fd.

run_one_shot now returns (OneShotResponse, TokenUsage); one shared parse_token_usage at the HTTP response boundary normalizes prompt_tokens -> input_tokens, completion_tokens -> output_tokens, prompt_tokens_details.cached_tokens -> cache_read_tokens, completion_tokens_details.cached_tokens -> cache_write_tokens. Missing or non-numeric counters stay null without failing the result.
All six one-shot operations return usage through the single call path; engine and UI callers destructure and ignore it, so visible behavior is unchanged and nothing is persisted to chat tables.
Tests: parser unit tests for complete/partial/absent/malformed usage; the six per-operation tests assert normalized usage from mocked responses. The existing migration test already proves all four chat_messages token columns are nullable and writable.
allium check, cargo fmt --check, cargo build --workspace, cargo test --workspace all pass.

Implemented in 33929fd. - `run_one_shot` now returns `(OneShotResponse, TokenUsage)`; one shared `parse_token_usage` at the HTTP response boundary normalizes `prompt_tokens` -> `input_tokens`, `completion_tokens` -> `output_tokens`, `prompt_tokens_details.cached_tokens` -> `cache_read_tokens`, `completion_tokens_details.cached_tokens` -> `cache_write_tokens`. Missing or non-numeric counters stay null without failing the result. - All six one-shot operations return usage through the single call path; engine and UI callers destructure and ignore it, so visible behavior is unchanged and nothing is persisted to chat tables. - Tests: parser unit tests for complete/partial/absent/malformed usage; the six per-operation tests assert normalized usage from mocked responses. The existing migration test already proves all four chat_messages token columns are nullable and writable. - `allium check`, `cargo fmt --check`, `cargo build --workspace`, `cargo test --workspace` all pass.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/RuDS#15