Skip to content

Add chat terminal sandbox smoke tests#324848

Draft
dileepyavan wants to merge 2 commits into
microsoft:mainfrom
dileepyavan:DileepY/smoke_tests
Draft

Add chat terminal sandbox smoke tests#324848
dileepyavan wants to merge 2 commits into
microsoft:mainfrom
dileepyavan:DileepY/smoke_tests

Conversation

@dileepyavan

Copy link
Copy Markdown
Member

Summary

  • add Linux and macOS smoke coverage for chat.agent.sandbox.enabled
  • verify sandboxed terminal execution, workspace and temporary-directory writes, outside-workspace read/write behavior, and default network denial
  • use deterministic mock LLM tool calls and a local HTTP probe rather than external network dependencies
  • register the suite for desktop, non-remote smoke runs

Validation

  • cd test/smoke && npm run compile
  • npm run precommit

The end-to-end suite has not completed successfully in the current local environment because Chat did not become ready during the smoke-test warm-up, so this is opened as a draft for CI validation and iteration.

Copilot AI review requested due to automatic review settings July 7, 2026 23:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Electron smoke-test suite to exercise the Chat terminal sandbox (chat.agent.sandbox.enabled) on Linux/macOS using deterministic mock-LLM tool calls plus a local HTTP probe, and wires the suite into the desktop (non-remote) smoke run.

Changes:

  • Register a new “Chat Sandbox” smoke-test suite in the main Electron smoke runner.
  • Add end-to-end sandbox validations for: terminal execution, workspace vs outside-workspace write behavior, outside-workspace reads, TMPDIR writes, and default network denial.
  • Use a mock LLM server + echo-last-message tool-result echoing to assert against the tool output payload.
Show a summary per file
File Description
test/smoke/src/main.ts Registers the new Chat Sandbox smoke suite for Electron non-remote runs.
test/smoke/src/areas/chat/chatSandbox.test.ts New Linux/macOS smoke suite that drives chat tool calls to validate terminal sandbox filesystem + network restrictions.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment thread test/smoke/src/areas/chat/chatSandbox.test.ts
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants