Fixed score value as null for single shard for sorting

[owaiskazi19]

Description

Fixed score value as null for single shard for sorting when sort field is not score. This mimics the current OpenSearch behavior

{
    "took": 3291,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 5,
            "relation": "eq"
        },
        "max_score": null,
        "hits": [
            {
                "_index": "my-test1",
                "_id": "5",
                "_score": null,
                "_source": {
                    "content_text": "zoo exhibits",
                    "content_vector": [
                        0.11,
                        0.44
                    ]
                },
                "sort": [
                    "5"
                ]
            },
            {
                "_index": "my-test1",
                "_id": "4",
                "_score": null,
                "_source": {
                    "content_text": "wildlife park",
                    "content_vector": [
                        0.14,
                        0.42
                    ]
                },
                "sort": [
                    "4"
                ]
            },
            {
                "_index": "my-test1",
                "_id": "3",
                "_score": null,
                "_source": {
                    "content_text": "zoo keeper",
                    "content_vector": [
                        0.15,
                        0.48
                    ]
                },
                "sort": [
                    "3"
                ]
            },
            {
                "_index": "my-test1",
                "_id": "2",
                "_score": null,
                "_source": {
                    "content_text": "aquarium fish",
                    "content_vector": [
                        0.12,
                        -0.34
                    ]
                },
                "sort": [
                    "2"
                ]
            },
            {
                "_index": "my-test1",
                "_id": "1",
                "_score": null,
                "_source": {
                    "content_text": "zoo animals",
                    "content_vector": [
                        0.13,
                        0.45
                    ]
                },
                "sort": [
                    "1"
                ]
            }
        ]
    }
}

Related Issues

Resolves #1274

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[vibrantvarun]

At high level, we already one DTO object HybridQueryContext. Instead of passing isSingleShard and sort seperatly as arguments, Can we just pass hybrid query context?

Overall the logic in code is correct.

[owaiskazi19]

At high level, we already one DTO object HybridQueryContext. Instead of passing isSingleShard and sort seperatly as arguments, Can we just pass hybrid query context?

HybridQueryContext is part of the query phase when building the HybridQuery. The CombineScoresDTO is passed across the combination and normalization phase where is exactly we want sort and isSingleShard field.

[vibrantvarun]

At high level, we already one DTO object HybridQueryContext. Instead of passing isSingleShard and sort seperatly as arguments, Can we just pass hybrid query context?

HybridQueryContext is part of the query phase when building the HybridQuery. The CombineScoresDTO is passed across the combination and normalization phase where is exactly we want sort and isSingleShard field.

Oh yeah sorry my bad. Please ignore the comment.

[martin-gaievski]

Can you please add unit test too, I think ScoreCombinationTechniqueTests class is a good place for it.

[owaiskazi19]

Can you please add unit test too, I think ScoreCombinationTechniqueTests class is a good place for it.

We have ITs for sorting that's why I added the same.

[martin-gaievski]

Can you please add unit test too, I think ScoreCombinationTechniqueTests class is a good place for it.

We have ITs for sorting that's why I added the same.

Right, but that not suppose to test all possible options/permutations of input, that should be done with unit tests.

[martin-gaievski]

On this note, can you please share results of customer scenario described in original issue #1274 and with your change

[owaiskazi19]

On this note, can you please share results of customer scenario described in original issue #1274 and with your change

Now returns with the change

{
    "took": 615,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": null,
        "hits": [
            {
                "_index": "my-test2",
                "_id": "1",
                "_score": null,
                "_source": {
                    "content_text": "foo",
                    "content_vector": [
                        0.12,
                        -0.34
                    ]
                },
                "sort": [
                    "1"
                ]
            },
            {
                "_index": "my-test2",
                "_id": "2",
                "_score": null,
                "_source": {
                    "content_text": "bar",
                    "content_vector": [
                        0.13,
                        0.45
                    ]
                },
                "sort": [
                    "2"
                ]
            }
        ]
    }
}

[martin-gaievski]

Looks good, will merge once CI is completed

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 0.00%. Comparing base (6cb2049) to head (0953ff7).
Report is 4 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #1277       +/-   ##
============================================
- Coverage     81.34%       0   -81.35%     
============================================
  Files           127       0      -127     
  Lines          5672       0     -5672     
  Branches        916       0      -916     
============================================
- Hits           4614       0     -4614     
+ Misses          702       0      -702     
+ Partials        356       0      -356     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.