Added release notes for 3.0 beta1 #1252

martin-gaievski · 2025-03-31T18:48:07Z

Description

Creating release notes for 3.0-beta1 release.

Related Issues

Resolves #225

Check List

~~[ ] New functionality includes testing.~~
~~[ ] New functionality has been documented.~~
~~[ ] API changes companion pull request created.~~
Commits are signed per the DCO using --signoff.
~~[ ] Public documentation issue/PR created.~~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

owaiskazi19

LGTM

owaiskazi19 · 2025-03-31T18:51:54Z

release-notes/opensearch-neural-search.release-notes-3.0.0.0-beta1.md

+- Support semantic sentence highlighter ([#1193](https://github.com/opensearch-project/neural-search/pull/1193))
+- Optimize embedding generation in Text Embedding Processor ([#1191](https://github.com/opensearch-project/neural-search/pull/1191))
+- Optimize embedding generation in Sparse Encoding Processor ([#1246](https://github.com/opensearch-project/neural-search/pull/1246))
+### Bug Fixes


Nit: Can we have some spacing before the heading?

will add it, in few our recent PRs related to releases we didn't do that, but core is doing it so we can be consistent with core.

owaiskazi19 · 2025-03-31T18:52:01Z

release-notes/opensearch-neural-search.release-notes-3.0.0.0-beta1.md

+- Optimize embedding generation in Sparse Encoding Processor ([#1246](https://github.com/opensearch-project/neural-search/pull/1246))
+### Bug Fixes
+- Remove validations for unmapped fields (text and image) in TextImageEmbeddingProcessor ([#1230](https://github.com/opensearch-project/neural-search/pull/1230))
+### Infrastructure


minalsha

lgtm

martin-gaievski · 2025-03-31T20:04:05Z

we may be missing one feature, @vibrantvarun should we wait for #1247 PR?

q-andy · 2025-03-31T23:19:13Z

#1208 is also waiting on merge to main pending app sec

heemin32 · 2025-04-01T00:07:56Z

Please add this as well. #1249

martin-gaievski · 2025-04-01T16:36:57Z

Please add this as well. #1249

added

vibrantvarun · 2025-04-01T18:09:36Z

we may be missing one feature, @vibrantvarun should we wait for #1247 PR?

yes I have raised the PR. #1253

vibrantvarun · 2025-04-01T19:58:14Z

~~(Please do not merge this PR now.)~~

Signed-off-by: Martin Gaievski <[email protected]>

codecov · 2025-04-03T22:51:20Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 0.00%. Comparing base (3ab220e) to head (bcc790d).
Report is 1 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #1252       +/-   ##
============================================
- Coverage     81.28%       0   -81.29%     
============================================
  Files           127       0      -127     
  Lines          5664       0     -5664     
  Branches        915       0      -915     
============================================
- Hits           4604       0     -4604     
+ Misses          704       0      -704     
+ Partials        356       0      -356

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

* Added release notes for 3.0 beta1 Signed-off-by: Martin Gaievski <[email protected]>

* Added release notes for 3.0 beta1 Signed-off-by: Martin Gaievski <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]>

#1342) * Implement Optimized embedding generation in text embedding processor (#1238) * implement single document update scenario for text embedding processor (#1191) Signed-off-by: Will Hwang <[email protected]> * implement batch document update scenario for text embedding processor (#1217) Signed-off-by: Will Hwang <[email protected]> --------- Signed-off-by: Will Hwang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Going from alpha1 to beta1 for 3.0 release (#1245) Signed-off-by: yeonghyeonKo <[email protected]> * Implement Optimized embedding generation in sparse encoding processor (#1246) Signed-off-by: Will Hwang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Implement Optimized embedding generation in text and image embedding processor (#1249) Signed-off-by: will-hwang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Inner hits support with hybrid query (#1253) * Inner Hits in Hybrid query Signed-off-by: Varun Jain <[email protected]> * Inner hits support with hybrid query Signed-off-by: Varun Jain <[email protected]> * Add changelog Signed-off-by: Varun Jain <[email protected]> * fix integ tests Signed-off-by: Varun Jain <[email protected]> * Modify comment Signed-off-by: Varun Jain <[email protected]> * Explain test case Signed-off-by: Varun Jain <[email protected]> * Optimize inner hits count calculation method Signed-off-by: Varun Jain <[email protected]> --------- Signed-off-by: Varun Jain <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Support custom tags in semantic highlighter (#1254) Signed-off-by: yeonghyeonKo <[email protected]> * Add neural stats API (#1256) * Add neural stats API Signed-off-by: Andy Qin <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Added release notes for 3.0 beta1 (#1252) * Added release notes for 3.0 beta1 Signed-off-by: Martin Gaievski <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Update semantic highlighter test model (#1259) Signed-off-by: Junqiu Lei <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Fix the edge case when the value of a fieldMap key in ingestDocument is empty string (#1257) Signed-off-by: Chloe Gao <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Hybrid query should call rewrite before creating weight (#1268) * Hybrid query should call rewrite before creating weight Signed-off-by: Harsha Vamsi Kalluri <[email protected]> * Awaits fix Signed-off-by: Harsha Vamsi Kalluri <[email protected]> * Rewrite with searcher Signed-off-by: Harsha Vamsi Kalluri <[email protected]> * Feature flag issue Signed-off-by: Harsha Vamsi Kalluri <[email protected]> --------- Signed-off-by: Harsha Vamsi Kalluri <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Support phasing off SecurityManager usage in favor of Java Agent (#1265) Signed-off-by: Gulshan <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Fix multi node transport issue on NeuralKNNQueryBuilder originalQueryText (#1272) Signed-off-by: Junqiu Lei <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Add semantic field mapper. (#1225) Signed-off-by: Bo Zhang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Increment version to 3.0.0-SNAPSHOT (#1286) Signed-off-by: opensearch-ci-bot <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Remove beta1 qualifier (#1292) Signed-off-by: Peter Zhu <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Fix for merging scoreDocs when totalHits are greater than 1 and fieldDocs are 0 (#1295) (#1296) (cherry picked from commit 6f3aabb) Co-authored-by: Varun Jain <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * add release notes for 3.0 (#1287) Signed-off-by: will-hwang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Allow maven to publish to all versions (#1300) (#1301) Signed-off-by: Peter Zhu <[email protected]> (cherry picked from commit c5625db) Co-authored-by: Peter Zhu <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * [FEAT] introduce new FixedStringLengthChunker Signed-off-by: yeonghyeonKo <[email protected]> * [TEST] initial test cases for FixedStringLengthChunker Signed-off-by: yeonghyeonKo <[email protected]> * [FIX] gradlew spotlessApply Signed-off-by: yeonghyeonKo <[email protected]> * [REFACTOR] remove unnecessary comments Signed-off-by: yeonghyeonKo <[email protected]> * [Performance Improvement] Add custom bulk scorer for hybrid query (2-3x faster) (#1289) Signed-off-by: Martin Gaievski <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Add TextChunkingProcessor stats (#1308) * Add TextChunkingProcessor stats Signed-off-by: Andy Qin <[email protected]> # Conflicts: # CHANGELOG.md * Update unit and integ tests Signed-off-by: Andy Qin <[email protected]> --------- Signed-off-by: Andy Qin <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Update Lucene dependencies (#1336) * Update Lucene dependencies Signed-off-by: Ryan Bogan <[email protected]> * Add changelog entry Signed-off-by: Ryan Bogan <[email protected]> * Update model request body for bwc and integ tests Signed-off-by: Ryan Bogan <[email protected]> --------- Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * [REFACTOR] modify algorithm name and related parts Signed-off-by: yeonghyeonKo <[email protected]> * [REFACTOR] update test codes along with the change in CharacterLengthChunker Signed-off-by: yeonghyeonKo <[email protected]> * [REFACTOR] remove defensive check to prevent adding redundant code lines Signed-off-by: yeonghyeonKo <[email protected]> * Update CharacterLengthChunker to FixedCharLengthChunker Signed-off-by: Marcel Yeonghyeon Ko <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Update ChunkerFactory Signed-off-by: Marcel Yeonghyeon Ko <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Update CharacterLengthChunkerTests to FixedCharLengthChunkerTests Signed-off-by: Marcel Yeonghyeon Ko <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * [FIX] handle a corner case where the content is shorter than charLimit Signed-off-by: yeonghyeonKo <[email protected]> * [TEST] Add integration test codes for fixed_char_length chunking algorithm Signed-off-by: yeonghyeonKo <[email protected]> * [TEST] integration test code for cascaded pipeline Signed-off-by: yeonghyeonKo <[email protected]> * Support analyzer-based neural sparse query (#1088) * merge main; add analyzer impl Signed-off-by: zhichao-aws <[email protected]> * two phase adaption Signed-off-by: zhichao-aws <[email protected]> * two phase adaption Signed-off-by: zhichao-aws <[email protected]> * remove analysis Signed-off-by: zhichao-aws <[email protected]> * lint Signed-off-by: zhichao-aws <[email protected]> * update Signed-off-by: zhichao-aws <[email protected]> * address comments Signed-off-by: zhichao-aws <[email protected]> * tests Signed-off-by: zhichao-aws <[email protected]> * modify plugin security policy Signed-off-by: zhichao-aws <[email protected]> * change log Signed-off-by: zhichao-aws <[email protected]> * address comments Signed-off-by: zhichao-aws <[email protected]> * modify to package-private Signed-off-by: zhichao-aws <[email protected]> --------- Signed-off-by: zhichao-aws <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Fixed score value as null for single shard for sorting (#1277) * Fixed score value as null for single shard for sorting Signed-off-by: Owais <[email protected]> * Addressed comment Signed-off-by: Owais <[email protected]> * Addressed more comments Signed-off-by: Owais <[email protected]> * Added UT Signed-off-by: Owais <[email protected]> --------- Signed-off-by: Owais <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Add IT for neural sparse query + bert-uncased mbert-uncased analyzer (#1279) * add it Signed-off-by: zhichao-aws <[email protected]> * change log Signed-off-by: zhichao-aws <[email protected]> --------- Signed-off-by: zhichao-aws <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Add WithFieldName implementation to QueryBuilders (#1285) Signed-off-by: Owais <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * [AUTO] Increment version to 3.1.0-SNAPSHOT (#1288) * Increment version to 3.1.0-SNAPSHOT Signed-off-by: opensearch-ci-bot <[email protected]> * Update build.gradle Signed-off-by: Peter Zhu <[email protected]> --------- Signed-off-by: opensearch-ci-bot <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Peter Zhu <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * add release notes for 3.0 (#1298) Signed-off-by: will-hwang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Return bad request for invalid stat parameters in stats API (#1291) Signed-off-by: Andy Qin <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Add semantic mapping transformer. (#1276) Signed-off-by: Bo Zhang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Add semantic ingest processor. (#1309) Signed-off-by: Bo Zhang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * [Performance Improvement] Add custom bulk scorer for hybrid query (2-3x faster) (#1289) Signed-off-by: Martin Gaievski <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Implement the query logic for the semantic field. (#1315) Signed-off-by: Bo Zhang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Support custom weights params in RRF (#1322) * Support Weights params in RRF Signed-off-by: Varun Jain <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * add validation for invalid nested hybrid query (#1305) * add validation for nested hybrid query Signed-off-by: will-hwang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Add stats tracking for semantic highlighting (#1327) * Add stats tracking for semantic highlighting Signed-off-by: Junqiu Lei <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Update Lucene dependencies (#1336) * Update Lucene dependencies Signed-off-by: Ryan Bogan <[email protected]> * Add changelog entry Signed-off-by: Ryan Bogan <[email protected]> * Update model request body for bwc and integ tests Signed-off-by: Ryan Bogan <[email protected]> --------- Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Enhance semantic field to allow to enable/disable chunking. (#1337) * Implement the query logic for the semantic field. Signed-off-by: Bo Zhang <[email protected]> * Enhance semantic field to allow to enable/disable chunking. Signed-off-by: Bo Zhang <[email protected]> --------- Signed-off-by: Bo Zhang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * [REFACTOR] modify algorithm name and related parts Signed-off-by: yeonghyeonKo <[email protected]> * Update CHANGELOG.md Signed-off-by: Marcel Yeonghyeon Ko <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * [FEAT] Add fixed_char_length chunking algorithm to STAT manager Signed-off-by: yeonghyeonKo <[email protected]> * [TEST] Add integration test codes for fixed_char_length chunking algorithm Signed-off-by: yeonghyeonKo <[email protected]> * [TEST] integration test code for cascaded pipeline Signed-off-by: yeonghyeonKo <[email protected]> * Going from alpha1 to beta1 for 3.0 release (#1245) Signed-off-by: yeonghyeonKo <[email protected]> * Fix multi node transport issue on NeuralKNNQueryBuilder originalQueryText (#1272) Signed-off-by: Junqiu Lei <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Add semantic field mapper. (#1225) Signed-off-by: Bo Zhang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Add semantic mapping transformer. (#1276) Signed-off-by: Bo Zhang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> * Fix multi node transport issue on NeuralKNNQueryBuilder originalQueryText (#1272) Signed-off-by: Junqiu Lei <[email protected]> * Add semantic field mapper. (#1225) Signed-off-by: Bo Zhang <[email protected]> * Add semantic mapping transformer. (#1276) Signed-off-by: Bo Zhang <[email protected]> * [FIX] minor typo Signed-off-by: yeonghyeonKo <[email protected]> * [REFACTOR] adopt FixedTokenLengthChunker's loop strategy for robust final chunking Signed-off-by: yeonghyeonKo <[email protected]> * [TEST] sum the number of processors and their executions correctly in TextChunkingProcessorIT Signed-off-by: yeonghyeonKo <[email protected]> * [REFACTOR] gradlew spotlessApply Signed-off-by: yeonghyeonKo <[email protected]> --------- Signed-off-by: Will Hwang <[email protected]> Signed-off-by: yeonghyeonKo <[email protected]> Signed-off-by: will-hwang <[email protected]> Signed-off-by: Varun Jain <[email protected]> Signed-off-by: Andy Qin <[email protected]> Signed-off-by: Martin Gaievski <[email protected]> Signed-off-by: Junqiu Lei <[email protected]> Signed-off-by: Chloe Gao <[email protected]> Signed-off-by: Harsha Vamsi Kalluri <[email protected]> Signed-off-by: Gulshan <[email protected]> Signed-off-by: Bo Zhang <[email protected]> Signed-off-by: opensearch-ci-bot <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Marcel Yeonghyeon Ko <[email protected]> Signed-off-by: zhichao-aws <[email protected]> Signed-off-by: Owais <[email protected]> Co-authored-by: Will Hwang <[email protected]> Co-authored-by: Martin Gaievski <[email protected]> Co-authored-by: Varun Jain <[email protected]> Co-authored-by: Junqiu Lei <[email protected]> Co-authored-by: Andy <[email protected]> Co-authored-by: Chloe Gao <[email protected]> Co-authored-by: Harsha Vamsi Kalluri <[email protected]> Co-authored-by: Gulshan <[email protected]> Co-authored-by: Bo Zhang <[email protected]> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Peter Zhu <[email protected]> Co-authored-by: Ryan Bogan <[email protected]> Co-authored-by: zhichao-aws <[email protected]> Co-authored-by: Owais Kazi <[email protected]>

martin-gaievski added Maintenance Add support for new versions of OpenSearch/Dashboards from upstream Infrastructure v3.0.0 v3.0.0 labels Mar 31, 2025

martin-gaievski requested review from heemin32, navneet1v, VijayanB, vamshin, jmazanec15, naveentatikonda, junqiu-lei, sean-zheng-amazon, model-collapse, zane-neo, vibrantvarun, zhichao-aws, yuye-aws and minalsha as code owners March 31, 2025 18:48

github-actions bot added the release label Mar 31, 2025

ryanbogan approved these changes Mar 31, 2025

View reviewed changes

owaiskazi19 approved these changes Mar 31, 2025

View reviewed changes

heemin32 approved these changes Mar 31, 2025

View reviewed changes

martin-gaievski force-pushed the added_release_notes_3.0_beta branch from 965b016 to 04dfe15 Compare March 31, 2025 19:04

minalsha approved these changes Mar 31, 2025

View reviewed changes

martin-gaievski force-pushed the added_release_notes_3.0_beta branch from 04dfe15 to dc51e99 Compare April 1, 2025 16:35

martin-gaievski force-pushed the added_release_notes_3.0_beta branch from 298321e to e63dacc Compare April 3, 2025 17:39

vibrantvarun approved these changes Apr 3, 2025

View reviewed changes

martin-gaievski added 2 commits April 3, 2025 14:14

Added release notes for 3.0 beta1

e629c6d

Signed-off-by: Martin Gaievski <[email protected]>

Added PR 1249, 1254 and 1253

bcc790d

Signed-off-by: Martin Gaievski <[email protected]>

martin-gaievski force-pushed the added_release_notes_3.0_beta branch from e63dacc to bcc790d Compare April 3, 2025 21:18

martin-gaievski merged commit 3e32d54 into opensearch-project:main Apr 3, 2025
51 checks passed

ryanbogan pushed a commit to ryanbogan/neural-search that referenced this pull request Apr 10, 2025

Added release notes for 3.0 beta1 (opensearch-project#1252)

aa801da

* Added release notes for 3.0 beta1 Signed-off-by: Martin Gaievski <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Added release notes for 3.0 beta1 #1252

Added release notes for 3.0 beta1 #1252

Uh oh!

martin-gaievski commented Mar 31, 2025

Uh oh!

owaiskazi19 left a comment

Uh oh!

owaiskazi19 Mar 31, 2025

Uh oh!

martin-gaievski Mar 31, 2025

Uh oh!

owaiskazi19 Mar 31, 2025

Uh oh!

minalsha left a comment

Uh oh!

martin-gaievski commented Mar 31, 2025

Uh oh!

q-andy commented Mar 31, 2025

Uh oh!

heemin32 commented Apr 1, 2025 •

edited

Loading

Uh oh!

martin-gaievski commented Apr 1, 2025

Uh oh!

vibrantvarun commented Apr 1, 2025

Uh oh!

vibrantvarun commented Apr 1, 2025 •

edited

Loading

Uh oh!

Uh oh!

codecov bot commented Apr 3, 2025

Uh oh!

Uh oh!

Added release notes for 3.0 beta1 #1252

Added release notes for 3.0 beta1 #1252

Uh oh!

Conversation

martin-gaievski commented Mar 31, 2025

Description

Related Issues

Check List

Uh oh!

owaiskazi19 left a comment

Choose a reason for hiding this comment

Uh oh!

owaiskazi19 Mar 31, 2025

Choose a reason for hiding this comment

Uh oh!

martin-gaievski Mar 31, 2025

Choose a reason for hiding this comment

Uh oh!

owaiskazi19 Mar 31, 2025

Choose a reason for hiding this comment

Uh oh!

minalsha left a comment

Choose a reason for hiding this comment

Uh oh!

martin-gaievski commented Mar 31, 2025

Uh oh!

q-andy commented Mar 31, 2025

Uh oh!

heemin32 commented Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

martin-gaievski commented Apr 1, 2025

Uh oh!

vibrantvarun commented Apr 1, 2025

Uh oh!

vibrantvarun commented Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Apr 3, 2025

Codecov Report

Uh oh!

Uh oh!

heemin32 commented Apr 1, 2025 •

edited

Loading

vibrantvarun commented Apr 1, 2025 •

edited

Loading