Cache names of indices between requests

[CementWork]

Why does it query the index no matter what I input, instead of searching the content within the index?

[jedrazb]

Hey @CementWork can you provide reproduction steps for the issue you encounter?

[wildemat]

@CementWork Seems like you are asking about some cache implementation?

From my understanding (and from the blog post), the LLM can only determine your intent when it has all the context of what resources are available on the MCP server. With each input, the model needs to get the latest set of indices, see if there's one there that matches what you might want, then further see if that index has attributes that match up with what you're looking for. With those 2 bits of info it can then build a valid Query DSL to search.

I don't think it would work to skip over providing it with available indices. But caching list_indices results sounds like a reasonable option.

[JohnTitor2036v2]

It is inconvenient and inefficient for when there are 1500 indices or more. I was using Claude Desktop and as soon as it starts to list indices, it seems that it spends all the context tokens. What can be done to solve this issue?

[jedrazb]

@JohnTitor2036v2 If you already know the index you want to query, you can have the LLM search that index directly, skipping the index name lookup.

Alternatively, you could use an API key that’s restricted to a specific subset of indices, limiting the search space to just the ones you want to chat with. This could be a workaround until we improve the server’s ability to better handle thousands of indices.

[JoshMock]

Changing this to an enhancement request to cache index names between requests. A small in-memory cache is typically pretty easy to support, as long as the TTL of the cache is short-lived and can be circumvented by tools that might depend on a fresh list.