Skip to content

Resolve execution subagent model by exact CAPI id before family fallback#324859

Draft
piyushmadan wants to merge 1 commit into
microsoft:mainfrom
piyushmadan:piyushmadan/subagent_orchestration
Draft

Resolve execution subagent model by exact CAPI id before family fallback#324859
piyushmadan wants to merge 1 commit into
microsoft:mainfrom
piyushmadan:piyushmadan/subagent_orchestration

Conversation

@piyushmadan

Copy link
Copy Markdown

Summary

The github.copilot.chat.executionSubagent.model setting was resolved only by CAPI capabilities.family, never by exact model id. When multiple model ids share a single family, family-only resolution always selects the first model in that family (in /models response order), so a specific id could not be targeted.

This caused MAI routing failures:

  • executionSubagent.model: mai-code-1-flash-picker did not match a family key, so it silently fell back to the main agent model.
  • executionSubagent.model: oswe-vscode-modelD resolved the family, but always selected the first entry (lark-debug-picker), not the intended MAI model.

This change adds exact model-id resolution first, keeping the existing family resolution as a fallback.

Changes

  • modelMetadataFetcher.ts: added getChatModelFromCapiId(id) to IModelMetadataFetcher and ModelMetadataFetcher. It uses the same fetch guard as the other methods, iterates the family map, and returns the first chat model whose id matches exactly (or undefined on miss). getChatModelFromCapiFamily is unchanged.
  • endpointProvider.ts: added getChatEndpointById(id): Promise<IChatEndpoint | undefined> to IEndpointProvider.
  • endpointProviderImpl.ts: ProductionEndpointProvider.getChatEndpointById resolves via getChatModelFromCapiId and wraps with getOrCreateChatEndpointInstance, returning undefined on miss. getChatEndpoint(...) is unchanged.
  • testEndpointProvider.ts: implemented getChatEndpointById against its existing model metadata flow.
  • executionSubagentToolCallingLoop.ts: the non-proxy path now attempts exact-id resolution first, then falls back to family resolution, then to the main agent endpoint (with trace logs on fallback). The agentic-proxy path is untouched.
  • Updated the various mock/null IEndpointProvider / IModelMetadataFetcher implementations to satisfy the widened interfaces.

Tests

  • modelMetadataFetcher.spec.ts (new): exact-id resolution wins over the first-in-family model; family resolution still selects the first member; unknown id returns undefined.
  • endpointProviderImpl.spec.ts (new): getChatEndpointById resolves the exact id and returns undefined on miss; family fallback via getChatEndpoint unchanged.
  • Updated copilotChatEndpoint.spec.ts fake to implement the new method.

Validation

  • npx tsgo --noEmit passes for both tsconfig.json and tsconfig.worker.json.
  • Endpoint unit tests pass (366 tests).

Non-goals

  • No changes to the agentic-proxy path, settings, or getChatModelFromCapiFamily behavior.

Copilot AI review requested due to automatic review settings July 8, 2026 04:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a model-routing bug for the github.copilot.chat.executionSubagent.model setting. Previously the subagent model was resolved only by CAPI capabilities.family, which always selects the first model in a family (in /models order), so a specific model id sharing a family with others could never be targeted (breaking MAI routing such as mai-code-1-flash-picker). It adds an exact-model-id resolution path that runs before the existing family fallback.

Changes:

  • Adds getChatModelFromCapiId(id) to IModelMetadataFetcher/ModelMetadataFetcher (chat-model-only, undefined on miss) and getChatEndpointById(id) to IEndpointProvider/ProductionEndpointProvider.
  • Updates ExecutionSubagentToolCallingLoop.getEndpoint to try exact-id resolution first, then family resolution, then the main-agent endpoint, with trace logs on fallback (agentic-proxy path untouched).
  • Adds unit tests (modelMetadataFetcher.spec.ts, endpointProviderImpl.spec.ts) and updates all mock/null endpoint-provider and model-fetcher implementations to satisfy the widened interfaces.
Show a summary per file
File Description
platform/endpoint/node/modelMetadataFetcher.ts Adds getChatModelFromCapiId (interface + impl) iterating _familyMap for an exact chat-model id match
platform/endpoint/common/endpointProvider.ts Adds getChatEndpointById to IEndpointProvider with JSDoc
extension/prompt/vscode-node/endpointProviderImpl.ts Implements getChatEndpointById via the fetcher, wrapping with getOrCreateChatEndpointInstance
extension/prompt/node/executionSubagentToolCallingLoop.ts Exact-id-first resolution in getEndpoint, with tool-call-support and error fallbacks + trace logs
platform/endpoint/test/node/modelMetadataFetcher.spec.ts New tests: exact-id wins over first-in-family, family unchanged, unknown id → undefined
extension/prompt/vscode-node/test/endpointProviderImpl.spec.ts New tests for getChatEndpointById id resolution / miss and family fallback
platform/endpoint/test/node/testEndpointProvider.ts Implements getChatEndpointById against existing metadata flow
platform/endpoint/test/node/copilotChatEndpoint.spec.ts Fake fetcher implements getChatModelFromCapiId
extension/test/vscode-node/endpoints.test.ts Two fake fetchers implement getChatModelFromCapiId
lib/node/chatLibMain.ts NullEndpointProvider.getChatEndpointById returns undefined
lib/vscode-node/test/getInlineCompletions.spec.ts, extension/tools/node/test/searchToolTestUtils.ts, extension/intents/node/test/searchSubagentGating.spec.ts, extension/prompts/node/panel/test/fileVariable.spec.ts, extension/prompts/node/base/test/promptRenderer.spec.ts, extension/prompt/node/test/feedbackGenerator.spec.ts, extension/chatSessions/claude/node/test/claudeCodeModels.spec.ts Mock providers updated to satisfy the widened IEndpointProvider interface

Review details

  • Files reviewed: 17/17 changed files
  • Comments generated: 1
  • Review effort level: Medium

Comment on lines +135 to +143
const byId = await this.endpointProvider.getChatEndpointById(modelName);
if (byId) {
if (byId.supportsToolCalls) {
return byId;
}
// Resolved model does not support tool calls, fallback to main agent endpoint
this._logService.trace(`[ExecutionSubagent] Model '${modelName}' does not support tool calls; falling back to main agent endpoint.`);
return await this.endpointProvider.getChatEndpoint(this.options.request);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants