Description
Describe the bug
I have a query that uses a hybrid search + rag + rrf search pipeline, the query has the generative_qa_parameters extension block in order to answer the question with a llm model. It works like a charm. Nevertheless, when I put this query inside a search template and invoke this, I get only the result hits and not the block with the RAG/LLM answer.
The query - correct answer. I rendered this query with the search template that doesn't work right, so the query should be identical (rendered with {{openSearchUrl}}/_render/template):
GET {{openSearchUrl}}/{{openSearchIndex}}/_search?search_pipeline=hybrid-search-rag-pipeline_rrf&explain=false
{
"_source": {
"include": [
"tns_body",
"tns_body_chunked2048"
],
"exclude": [
"embeddingnested768_tns_body_chunked2048_chunked1024",
"embeddingnested768_tns_body_chunked1024"
]
},
"size": 200,
"sort": [],
"query": {
"hybrid": {
"queries": [
{
"bool": {
"should": [
{
"match": {
"tns_body_chunked2048": {
"query": "Kindermuseum",
"boost": 1
}
}
},
{
"match": {
"tns_body_chunked2048.ngram": {
"query": "Kindermuseum",
"boost": 1
}
}
}
],
"filter": []
}
},
{
"bool": {
"should": [
{
"nested": {
"score_mode": "max",
"path": "embeddingnested768_tns_body_chunked2048_chunked1024",
"query": {
"neural": {
"embeddingnested768_tns_body_chunked2048_chunked1024.knn": {
"query_text": "Kindermuseum",
"model_id": "ueeYJJYBtReV65DZrK_Y",
"k": 200
}
}
}
}
}
],
"filter": []
}
}
]
}
},
"ext": {
"generative_qa_parameters": {
"llm_question": "Kindermuseum",
"llm_model": "meta-llama-3.3-70b-instruct-fp8",
"context_size": 15,
"message_size": 50,
"timeout": 1024
}
}
}The search pipeline:
"hybrid-search-rag-pipeline_rrf": {
"description": "RAGnarok - Post and response processor for hybrid search and RAG, with reciprocal rank fusion. Using the LLM model: SDS internal LLM",
"phase_results_processors": [
{
"score-ranker-processor": {
"combination": {
"technique": "rrf",
"rank_constant": 60
}
}
}
],
"response_processors": [
{
"retrieval_augmented_generation": {
"tag": "ragnarok_rag_pipeline",
"description": "RAGnarok - RAG response processor the LLM: SDS internal LLM",
"model_id": "veeYJJYBtReV65DZra9L",
"context_field_list": [
"tns_body_chunked2048"
],
"system_prompt": "You are a system that generates a concise and informative answer in less than 100 words for the given question, which is marked with QUESTION. Use the given search results marked with SEARCH RESULT as only knowledge base. Answer questions in the language of the question, i.e. german questions in german and only english questions in english.",
"user_instructions": "Answer questions in german language in german language only, answer each question in the language of the question."
}
}
]
}The search template that returns no RAG answer:
"RAGnarok_searchTemplateHybridRAG": {
"lang": "mustache",
"source": "{\"_source\":{\"include\":{{#toJson}}_source.include{{/toJson}},\"exclude\":{{#toJson}}_source.exclude{{/toJson}}},\"size\":200,\"sort\":{{#toJson}}sort{{/toJson}},\"query\":{\"hybrid\":{\"queries\":[{\"bool\":{\"should\":[{\"match\":{\"tns_body_chunked2048\":{\"query\":\"{{query}}\",\"boost\":1.0}}},{\"match\":{\"tns_body_chunked2048.ngram\":{\"query\":\"{{query}}\",\"boost\":1.0}}}],\"filter\":{{#toJson}}filter{{/toJson}}}},{\"bool\":{\"should\":[{\"nested\":{\"score_mode\":\"max\",\"path\":\"embeddingnested768_tns_body_chunked2048_chunked1024\",\"query\":{\"neural\":{\"embeddingnested768_tns_body_chunked2048_chunked1024.knn\":{\"query_text\":\"{{query}}\",\"model_id\":\"ueeYJJYBtReV65DZrK_Y\",\"k\":200}}}}}],\"filter\":{{#toJson}}filter{{/toJson}}}}]}},\"ext\":{\"generative_qa_parameters\":{\"llm_question\":\"{{query}}\",\"llm_model\":\"meta-llama-3.3-70b-instruct-fp8\",\"context_size\":15,\"message_size\":50,\"timeout\":1024}}}"
}And finally, my search request using the search template, lacking the $.ext.retrieval_augmented_generation.answer block. The body is the same used for rendering the query that works when invoked manually:
GET {{openSearchUrl}}/{{openSearchIndex}}/_search/template?search_pipeline=hybrid-search-rag-pipeline_rrf
GET {{openSearchUrl}}/_render/template
{
"id": "RAGnarok_searchTemplateHybridRAG",
"params": {
"_source.include": [
"tns_body",
"tns_body_chunked2048"
],
"_source.exclude": [
"embeddingnested768_tns_body_chunked2048_chunked1024",
"embeddingnested768_tns_body_chunked1024"
],
"sort": [],
"filter": [],
"query": "Kindermuseum"
}
}Related component
Other
Expected behavior
I assume this is a bug, because the search template renders the query right, as verified with /_render/template. I assume the result should then be identical, which is not the case.
Additional Details
Host/Environment (please complete the following information):
- OS: Manjaro Linux
- Version: current
OpenSearch version "2.19.1" (also current)