fix(batch): prevent duplicate spend updates on batch retrieve

[colesmcintosh]

Title

fix(batch): prevent duplicate spend updates on batch retrieve

Relevant issues

Fixes #12084

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

Problem

Every time retrieve_batch is called, the batch spend is calculated and updated in the database, causing duplicate spend entries for the same batch job.

Solution

• Added an in-memory cache (_PROCESSED_BATCH_COSTS) to track which batch IDs have already had their costs calculated
• Check the cache before processing batch costs - if the batch ID is already in the cache, skip the cost calculation
• Add the batch ID to the cache after successful cost calculation
• This ensures each batch's cost is only calculated and logged once, regardless of how many times retrieve_batch is called

Testing

Added unit tests in tests/test_litellm/test_batch_spend_cache.py to verify:

• Cache correctly tracks processed batch IDs
• Cache prevents duplicate processing
• Cache operations (add, check, clear) work as expected

Test Results

poetry run pytest tests/test_litellm/test_batch_spend_cache.py -v
============================= test session starts ==============================
platform darwin -- Python 3.11.13, pytest-7.4.4, pluggy-1.5.0
collected 2 items

tests/test_litellm/test_batch_spend_cache.py::test_batch_cost_caching_logic PASSED [ 50%]
tests/test_litellm/test_batch_spend_cache.py::test_processed_batch_costs_cache PASSED [100%]

========================= 2 passed, 1 warning in 0.11s =========================

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 30, 2025 4:40pm