Skip to content

ES|QL: Replace RRF with FUSE #130693

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 8, 2025
Merged

ES|QL: Replace RRF with FUSE #130693

merged 1 commit into from
Jul 8, 2025

Conversation

ioanatia
Copy link
Contributor

@ioanatia ioanatia commented Jul 7, 2025

Related #130434
meta issue: #123389

Will merge only after Kibana merges a change to change the autocomplete we have for RRF to be the autocomplete functionality for FUSE (elastic/kibana#226743)

@ioanatia ioanatia added >non-issue :Search Relevance/Search Catch all for Search Relevance Team:Search - Relevance The Search organization Search Relevance team v9.2.0 labels Jul 7, 2025
@@ -263,25 +262,10 @@ sampleCommand
: SAMPLE probability=constant
;

//
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved fork, completion and change_point before the // In development section

);

return new OrderBy(source, dedup, order);
return new Dedup(source, new RrfScoreEval(source, input, scoreAttr, forkAttr), aggregates, groupings);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will be changed in a follow up PR where we continue to work on FUSE - we called this Dedup initially because we thought that it would be its own command - now that command is actually FUSE.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we rename Dedup to FusePlan or similar so it's easier to follow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep done in a follow up - I have a lot more changes on the planning side - did not want to add them here where we just remove RRF

| rrf
| rrf
"""));
assertThat(e.getMessage(), containsString("RRF can only be used after FORK, but found RRF"));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FUSE does not have the restriction of only following fork - so we removed that validation and tests

@ioanatia ioanatia marked this pull request as ready for review July 7, 2025 10:01
@elasticsearchmachine elasticsearchmachine added Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch and removed Team:Search - Relevance The Search organization Search Relevance team labels Jul 7, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

Copy link
Member

@kderusso kderusso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

import static org.elasticsearch.xpack.esql.common.Failure.fail;

public class RrfScoreEval extends UnaryPlan implements PostAnalysisVerificationAware, LicenseAware {
public class RrfScoreEval extends UnaryPlan implements LicenseAware {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we renaming this class as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not 100 % sure yet which is why I haven't touched it yet - we could have a FuseScoreEval to do both RRF and linear, or have two classes RrfScoreEval and LinearScoreEval.

Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

);

return new OrderBy(source, dedup, order);
return new Dedup(source, new RrfScoreEval(source, input, scoreAttr, forkAttr), aggregates, groupings);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we rename Dedup to FusePlan or similar so it's easier to follow?

@ioanatia ioanatia merged commit 8f21ade into elastic:main Jul 8, 2025
32 checks passed
@ioanatia ioanatia deleted the replace_rrf branch July 8, 2025 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>non-issue :Search Relevance/Search Catch all for Search Relevance Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants