[safe-output-health] Safe Output Health Report - 2026-05-16

[github-actions[bot]]

Executive Summary

• Period: Last 24 hours (ending 2026-05-16 05:16 UTC)
• Runs Analyzed: 62
• Active Workflows: ~30
• Safe-output jobs succeeded: 51
• Safe-output jobs skipped (no activation): 9 (out of scope)
• Safe-output jobs cancelled (collateral): 1 (out of scope)
• Safe-output jobs failed (actionable): 1 ⚠️

One actionable safe-output job failure detected today: Scout run §25950059255, in its add_labels step. Root cause looks like a field-name mismatch between the agent's emitted output and the handler's expected schema.

Safe Output Job Statistics

Job Outcome	Count	Notes
success	51	Normal completion
failure (actionable)	1	Scout 25950059255 — add_labels
cancelled (collateral)	1	Smoke CI 25953080331 — whole workflow cancelled, out of scope
skipped (no activation)	9	Activation gates didn't fire; safe-output job never ran
in-progress / queued	2	This audit run + a queued Issue Monster

Safe-output tool calls observed across the day: safeoutputs.add_comment, safeoutputs.add_labels, safeoutputs.noop, safeoutputs.missing_data, safeoutputs.missing_tool, safeoutputs.create_discussion, safeoutputs.create_pull_request, safeoutputs.submit_pull_request_review.

Critical Issues

Cluster 1 — add_labels step failed with "No issue/PR number available" (1 occurrence)

• Run: Scout — §25950059255
• Trigger: workflow_dispatch via /scout slash command on issue Lockdown check fails on private cross-repo workflow_call: Contents API uses caller's GITHUB_TOKEN, not callee-accessible token #32312
• Pipeline: pre_activation ✅ → activation ✅ → agent ✅ → detection ✅ → safe_outputs ❌ (14.0s) → update_cache_memory ✅ → conclusion ✅
• Engine: Claude (claude-opus-4-7)
• Items emitted: 2 (add_comment, add_labels)
• Items recorded: 1 (add_comment succeeded — comment 4465144782 was posted)

Root cause analysis

The agent emitted (from agent_output.json / safeoutputs.jsonl):

{"issue_number":"32312","labels":["scout"],"type":"add_labels"}

But the add_labels tool schema in pkg/workflow/js/safe_outputs_tools.json:622 declares the field as item_number (number type), not issue_number. The handler in actions/setup/js/add_labels.cjs:67 reads message.item_number and has no fallback for issue_number:

if (message.item_number !== undefined) { ... } else {
  itemNumber = context.payload?.issue?.number ?? context.payload?.pull_request?.number;
}
if (!itemNumber || isNaN(itemNumber)) {
  return { success: false, error: "No issue/PR number available" };
}

Under workflow_dispatch the context.payload.issue is absent, so the fallback also yields undefined — and the step fails. The MCP server accepted the agent's tool call (validation in mcp-logs/safeoutputs/server.log shows both handlers "returned"), which is why the agent never saw an error.

• safe-outputs-items/safe-output-items.jsonl contains only the add_comment line (with its success URL); no add_labels entry.
• add_comment in the same payload correctly uses issue_number: 32312 (number) — note that handler does accept issue_number, so the two handlers are inconsistent.

Recommendations

Bug Fix (Recommended — Small)

• File: actions/setup/js/add_labels.cjs:67
• Problem: Only reads message.item_number; ignores agent's issue_number even though sibling add_comment accepts it.
• Fix: Normalize before the undefined check, e.g.:

  const itemNumberRaw = message.item_number ?? message.issue_number;
  if (itemNumberRaw !== undefined) { ... }

• Risk: Low — purely additive; brings add_labels in line with add_comment's tolerant key handling.

Schema Tightening (Optional — Defensive)

• File: pkg/workflow/js/safe_outputs_tools.json (add_labels.inputSchema)
• Change: Set additionalProperties: false so the safeoutputs MCP server rejects unknown keys like issue_number at agent time, giving the agent immediate, actionable feedback rather than silently producing an unusable item.
• Risk: Medium — verify no other agents/workflows currently rely on extra keys being tolerated.

Documentation

• File: pkg/workflow/js/safe_outputs_tools.json add_labels description.
• Change: Clarify that the parameter is item_number (not issue_number) and that it is required under workflow_dispatch, schedule, push, or any trigger without an issue/PR payload.

Work Item Plan

Work Item 1: Make add_labels accept issue_number alias

• Type: Bug fix
• Priority: Medium (1 occurrence; affects any agent that has internalized issue_number from sibling tools)
• Acceptance criteria:
  • add_labels handler accepts either item_number (number) or issue_number (number or string) and parses to integer.
  • Unit test in actions/setup/js/add_labels.test.cjs covers an item that uses issue_number under workflow_dispatch context.
  • Documentation in the tool description notes both names while encouraging item_number.
• Technical approach: Single change at add_labels.cjs:67; add ?? message.issue_number and Number()-coerce before the NaN check.
• Effort: Small (~30 lines including the test).

Trends

Compared to yesterday's audit (2026-05-15, 57 runs, 0 failures):

• Run volume up ~9% (57 → 62).
• Engine mix shifted slightly: claude 9→15, copilot 34→33, codex 0→2, pi 0→1.
• This is the first observed safe-output job failure since the daily audit cadence started. Pattern is now tracked in cache-memory/safe-output-health/error-patterns.json under add_labels_wrong_field_name; will escalate to recurring if seen again.

Out-of-Scope Findings (logged, not actioned)

Cancelled / skipped runs

• Smoke CI §25953080331 — agent + safe_outputs both cancelled; the whole workflow was cancelled, so safe_outputs is collateral, not a job-level fault. error_count=3 reflects the cancellation, not internal errors.
• 9 "skipped" runs across Q, Deployment Incident Monitor, and others — activation gates didn't fire (e.g., skip-if-match); the safe-output job never ran. Healthy by design.

References

• Failing run: https://github.com/github/gh-aw/actions/runs/25950059255
• Cancelled (out of scope): https://github.com/github/gh-aw/actions/runs/25953080331
• Audit run: https://github.com/github/gh-aw/actions/runs/25953642286

Note

🔒 Integrity filter blocked 1 item

The following item was blocked because it doesn't meet the GitHub integrity level.

• Lockdown check fails on private cross-repo workflow_call: Contents API uses caller's GITHUB_TOKEN, not callee-accessible token #32312 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by 🔒 Safe Output Health Monitor · ● 20.5M · ◷

• expires on May 17, 2026, 5:27 AM UTC

[github-actions[bot]]

💥 WHOOSH! ⚡ The Smoke-Test Agent zipped through here on run §25953787843! 🦸♂️

KAPOW! 💥 Claude engine: nominal!
ZAP! ⚡ MCPs: humming!
BOOM! 💣 Safe outputs: firing on all cylinders!

Onward to the next adventure! 🚀✨

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · ● 3.9M · ◷

[github-actions[bot]]

💥 KA-POW! 🦸 The Claude smoke test agent ZOOMED through here! ⚡️ WHOOSH! 💨 All systems nominal — to infinity and beyond! 🚀✨

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · ● 5.9M · ◷