Skip to content

Conversation

@drempapis
Copy link
Contributor

Circuit breaker bytes were being released too early in the fetch phase. The bytes were released immediately after building SearchHits in FetchPhase.buildSearchHits(), but before the response was serialized and sent to the coordinator node. This created a timing window where:

  1. `SearchHits are still in memory, consuming resources
  2. Circuit breaker has already released the bytes, believing the memory is freed
  3. This leads to circuit breaker undercounting, potentially allowing OOM conditions, especially for small setups with low heap resources.

This PR fixes the timing issue by delaying circuit breaker release until after the response has been successfully sent to the coordinator. We achieve this by:

  1. Storing circuit breaker bytes in FetchSearchResult when hits are built
  2. Releasing after transmission via listener wrappers that free bytes after listener.onResponse() completes
  3. Handling exceptions safely by releasing immediately if an error occurs before the result is created

@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label v9.3.0 labels Dec 9, 2025
@drempapis drempapis added >non-issue Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch :Search Foundations/Search Catch all for Search Foundations and removed needs:triage Requires assignment of a team area label labels Dec 9, 2025
@drempapis drempapis changed the title Fix/fetch phase breaker lifecycle Delay circuit breaker release until fetch response is sent Dec 9, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-foundations (Team:Search Foundations)

@drempapis
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/part-1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>non-issue :Search Foundations/Search Catch all for Search Foundations Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants