Skip to content

[Star tree] star tree off heap changes #14815

Closed
@bharath-techie

Description

@bharath-techie

Is your feature request related to a problem? Please describe

Sort and aggregation of segment documents when building star tree takes up lots of resources if they are stored in in-memory structures like Lists. So this issue tracks the off heap implementation where we store and retrieve the star tree documents in off heap method using temporary files.

Describe the solution you'd like

We will store the offsets in array and swap the offsets when we go through the files. We offload the entire sort and aggregation of segment documents and construction of star tree documents to off-heap.

Related component

Other

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Labels

IndexingIndexing, Bulk Indexing and anything related to indexingIndexing:PerformanceOtherenhancementEnhancement or improvement to existing feature or requestv2.17.0

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions