Skip to content

Commit 5264e08

Browse files
zhichao-awsYeonghyeonKO
authored andcommitted
Add IT for neural sparse query + bert-uncased mbert-uncased analyzer (opensearch-project#1279)
* add it Signed-off-by: zhichao-aws <[email protected]> * change log Signed-off-by: zhichao-aws <[email protected]> --------- Signed-off-by: zhichao-aws <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]>
1 parent 93b9fd6 commit 5264e08

File tree

2 files changed

+19
-1
lines changed

2 files changed

+19
-1
lines changed

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
## [Unreleased 3.x](https://github.com/opensearch-project/neural-search/compare/main...HEAD)
77

88
### Features
9-
- Implement analyzer based neural sparse query ([#1088](https://github.com/opensearch-project/neural-search/pull/1088))
9+
- Implement analyzer based neural sparse query ([#1088](https://github.com/opensearch-project/neural-search/pull/1088) [#1279](https://github.com/opensearch-project/neural-search/pull/1279))
1010
- [Semantic Field] Add semantic field mapper. ([#1225](https://github.com/opensearch-project/neural-search/pull/1225)).
1111

1212
### Enhancements

src/test/java/org/opensearch/neuralsearch/query/NeuralSparseQueryIT.java

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,24 @@ public void testBasicQueryUsingQueryTextAndAnalyzer() {
100100
assertEquals("1", firstInnerHit.get("_id"));
101101
float expectedScore = 2 * computeExpectedScore(testRankFeaturesDoc, Map.of("hello", 1f, "world", 1f, "a", 1f, "b", 1f));
102102
assertEquals(expectedScore, objectToFloat(firstInnerHit.get("_score")), DELTA);
103+
104+
sparseEncodingQueryBuilder.analyzer("bert-uncased");
105+
Map<String, Object> searchResponseAsMap2 = search(TEST_BASIC_INDEX_NAME, sparseEncodingQueryBuilder, 1);
106+
Map<String, Object> firstInnerHit2 = getFirstInnerHit(searchResponseAsMap2);
107+
expectedScore = 2 * computeExpectedScore(
108+
testRankFeaturesDoc,
109+
Map.of("hello", 6.937756538391113f, "world", 3.4208686351776123f, "a", 0.4022789001464844f, "b", 3.7727110385894775f)
110+
);
111+
assertEquals(expectedScore, objectToFloat(firstInnerHit2.get("_score")), DELTA);
112+
113+
sparseEncodingQueryBuilder.analyzer("mbert-uncased");
114+
Map<String, Object> searchResponseAsMap3 = search(TEST_BASIC_INDEX_NAME, sparseEncodingQueryBuilder, 1);
115+
Map<String, Object> firstInnerHit3 = getFirstInnerHit(searchResponseAsMap3);
116+
expectedScore = 2 * computeExpectedScore(
117+
testRankFeaturesDoc,
118+
Map.of("hello", 3.848525285720825f, "world", 1.5188000202178955f, "a", 1.054316759109497f, "b", 2.0035440921783447f)
119+
);
120+
assertEquals(expectedScore, objectToFloat(firstInnerHit3.get("_score")), DELTA);
103121
} finally {
104122
wipeOfTestResources(TEST_BASIC_INDEX_NAME, null, null, null);
105123
}

0 commit comments

Comments
 (0)