Skip to content

fix(mcp): re-read server config from disk on every respawn#71116

Open
dportier1021-crypto wants to merge 1 commit into
NousResearch:mainfrom
dportier1021-crypto:fix/mcp-respawn-config-refresh
Open

fix(mcp): re-read server config from disk on every respawn#71116
dportier1021-crypto wants to merge 1 commit into
NousResearch:mainfrom
dportier1021-crypto:fix/mcp-respawn-config-refresh

Conversation

@dportier1021-crypto

Copy link
Copy Markdown

Symptom

An MCP stdio server whose credentials are rotated in config.yaml by external tooling never receives the new values. Concrete case: a cron pre-run script unlocks a Bitwarden vault and writes a fresh BW_SESSION into mcp_servers.bitwarden.env.BW_SESSION, then kills the stale mcp-server-bitwarden child so the gateway respawns it. The respawned child still comes up with the old token — verified live via ps eww on three independent long-lived hosts (gateway, webui, desktop-serve), each holding a different stale token, none matching the just-written config value. The only escapes were a full process restart or /reload-mcp.

Root cause

MCPServerTask.run() captures config once (self._config = config) and the reconnect loop's while True: respawns via _run_stdio(config) / _run_http(config) using that same snapshot forever. _load_mcp_config() (and the mtime-aware load_config() beneath it) is only called from discover_mcp_tools() / status paths — never on the respawn path. Killing the child therefore can't propagate rotated env vars, because the auto-reconnect reuses the first-connect snapshot.

Fix

Re-read _load_mcp_config() at the top of every (re)spawn iteration inside the reconnect loop, and adopt the fresh config when it differs. This one site covers every respawn path (auto-reconnect after failure, parked revival, lazy stdio recycle, auth-recovery rebuild) because they all funnel through this loop. load_config() is mtime-aware, so the refresh is cheap when the file is unchanged. If the refresh raises or the server was removed from config, the loop falls back to the last known config — a config-refresh failure can never break reconnection.

Test

tests/tools/test_mcp_config_refresh.py (2 tests):

  • respawn after simulated transport death receives the refreshed on-disk env (MARKER one → two)
  • a server removed from config keeps its last known config and the loop doesn't crash

Existing MCP suites still pass: test_mcp_tool.py, test_mcp_bridge_single_failure.py, test_mcp_reload_rev.py — 231 passed.

MCPServerTask.run() captured the server config once at first connect
and reused that snapshot for every respawn in the reconnect loop.
Credentials rotated into config.yaml by external tooling (e.g. a
pre-run script refreshing BW_SESSION) never reached respawned
children — the only escape was a full process restart.

The loop now re-reads _load_mcp_config() (mtime-aware via
load_config()) at the top of every (re)spawn iteration, covering
auto-reconnect, parked revival, and lazy recycle paths. Falls back
to the last known config if the refresh fails or the server was
removed from config.

Regression test: respawn after simulated transport death picks up
the refreshed env marker; removed-from-config server keeps last
known config without crashing.
Copilot AI review requested due to automatic review settings July 25, 2026 01:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes MCP server respawn behavior so that when an MCP stdio/HTTP child process is restarted (e.g., after being killed to force a reconnect), the server’s configuration is re-read from disk each spawn iteration, allowing externally-rotated credentials in config.yaml (like BW_SESSION) to take effect without requiring a full process restart or manual /reload-mcp.

Changes:

  • Refresh MCP server config from disk at the top of each respawn iteration in MCPServerTask.run().
  • Update in-memory task config used for transport startup to reflect the refreshed on-disk config.
  • Add regression tests covering refreshed env propagation on respawn and resilience when a server is removed from config.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
tools/mcp_tool.py Re-reads MCP server config on every (re)spawn iteration before launching transport.
tests/tools/test_mcp_config_refresh.py Adds tests asserting respawn picks up refreshed config and tolerates server removal.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tools/mcp_tool.py
Comment on lines +3109 to +3123
try:
fresh_config = _load_mcp_config().get(self.name)
if isinstance(fresh_config, dict) and fresh_config:
if fresh_config != config:
logger.info(
"MCP server '%s': config changed on disk, "
"(re)spawning with fresh config",
self.name,
)
config = fresh_config
self._config = fresh_config
except Exception:
# Never let a config-refresh failure break the reconnect
# loop — fall back to the last known config.
pass
Comment on lines +32 to +34
# Keep backoff sleeps negligible.
monkeypatch.setattr(mcp_tool, "_MAX_BACKOFF_SECONDS", 0.01)
monkeypatch.setattr(mcp_tool, "_CONNECT_RETRY_BASE_BACKOFF_SEC", 0.01)
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists tool/mcp MCP client and OAuth area/config Config system, migrations, profiles labels Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles P2 Medium — degraded but workaround exists tool/mcp MCP client and OAuth type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants