feat(llm): add opt-in Responses API mode

[dtownsel]

Summary

• add opt-in Responses API mode to the existing OpenAI LLM backend
• convert chat history, system instructions, tool declarations, tool calls, and tool results to Responses input
• support strict Pydantic structured output, usage normalization, reasoning details, and streaming
• reconstruct output from streamed events for Codex-compatible endpoints whose response.completed event omits assembled output
• allow providers to opt out of max_output_tokens when their Responses endpoint rejects it
• preserve existing Chat Completions behavior unless provider_params.api_mode is explicitly responses

Validation

• pytest -n 0 tests/llm -q — 231 passed
• focused OpenAI backend suite — 53 passed
• ruff check — passed
• basedpyright src/llm/backends/openai.py — 0 errors, 0 warnings
• live authenticated gpt-5.6-luna bridge:
  • strict structured output parsed successfully
  • required function tool call normalized successfully
  • streaming returned exactly STREAM_OK
• end-to-end Honcho attribution canary produced only the explicitly stated user preference and rejected quoted biography, assistant actions, Reid output, contradictory logs, and malformed tool fragments

Compatibility

Responses mode is opt-in through model provider_params; existing OpenAI Chat Completions callers retain the original code path.

Summary by CodeRabbit

• New Features

  • Added support for OpenAI’s Responses API for standard and streaming completions.
  • Supports structured JSON output, reasoning controls, tool calls, and prior tool interactions.
  • Preserves text, reasoning details, completion status, and token usage in responses.
  • Streaming now provides incremental text updates followed by a final completion event.

• Tests

  • Added coverage for Responses API requests, structured output, tool handling, token limits, and streaming.

[coderabbitai]

Walkthrough

OpenAIBackend now supports OpenAI Responses API requests through selectable completion and streaming paths, including message/tool conversion, structured outputs, reasoning details, usage normalization, and terminal stream chunks. Tests cover request construction, tool history, token options, normalization, and streaming.

Changes

Responses API support

Layer / File(s)	Summary
Responses request construction src/llm/backends/openai.py, tests/llm/test_backends/test_openai.py	Responses mode routes completion and streaming calls through dedicated paths, constructs inputs and tools, configures structured output, and validates optional passthrough parameters.
Response execution and normalization src/llm/backends/openai.py, tests/llm/test_backends/test_openai.py	Responses events and payloads become normalized text, tool calls, reasoning details, usage values, finish reasons, and final stream chunks.
Responses stream test fixtures tests/llm/test_backends/test_openai.py	Fake asynchronous streams and final payload helpers support completion and streaming assertions.

Estimated code review effort: 4 (Complex) | ~45 minutes

Poem

I’m a rabbit with a whiskered API,
Hopping through responses, tokens flying by.
Tools in my basket, schemas neat and bright,
Streaming little text chunks through the night.
“Done!” says the final carrot—what a sight!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding an opt-in OpenAI Responses API mode.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[dtownsel]

Independent review follow-up

A separate read-only reviewer audited the staged changes after implementation.

Verdict: no blocking security, correctness, or compatibility issues.

Verified by the reviewer:

• Responses mode is opt-in and leaves the existing Chat Completions path unchanged;
• streamed text and output-item reconstruction is internally consistent;
• structured output and tool/tool-history conversion are covered;
• no credential leakage was found in the reviewed path;
• focused OpenAI backend suite: 53 passed.

Non-blocking notes considered:

• response.completed maps to finish_reason="stop", matching Responses API completed-state semantics;
• the Codex proxy intentionally exposes /responses, not /chat/completions; Honcho explicitly opts into Responses mode.

No follow-up code change was required.

[coderabbitai]

🧹 Nitpick comments (3)

src/llm/backends/openai.py (2)

153-164: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

temperature, stop, and thinking_budget_tokens are silently dropped in Responses mode.

Both the complete() (here) and stream() (Lines 267-280) responses branches forward only a subset of the caller-provided arguments. The Responses API does accept temperature/stop-equivalent parameters, so callers setting them will find them silently ignored once api_mode="responses". If this omission is intentional (e.g. reasoning-only usage), a brief comment documenting it would prevent confusion; otherwise these should be threaded into _build_responses_params.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/llm/backends/openai.py` around lines 153 - 164, Update the Responses-mode
branches in complete() and stream() to preserve temperature, stop, and
thinking_budget_tokens from the caller, threading them through
_complete_responses and _build_responses_params as supported by the Responses
API. If any parameter is intentionally unsupported, document that explicitly in
a concise comment instead of silently dropping it.

---

10-10: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Avoid the private OpenAI helper import openai.lib._pydantic.to_strict_json_schema is an internal SDK symbol, so a routine OpenAI upgrade can break this backend at import time. Prefer a public schema helper or a small compatibility wrapper around the import.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/llm/backends/openai.py` at line 10, Replace the direct private import of
to_strict_json_schema in the OpenAI backend with a compatibility wrapper that
prefers a public schema-generation API and falls back safely when needed; update
all call sites to use the wrapper so OpenAI SDK upgrades cannot break module
import.

tests/llm/test_backends/test_openai.py (1)

1103-1121: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider adding coverage for a named-function tool_choice.

The tool tests only exercise tool_choice="required", which passes through unchanged. A named-function choice (e.g. tool_choice="lookup") would surface the Chat-Completions-vs-Responses shape mismatch flagged in src/llm/backends/openai.py (Lines 468-472). Adding that assertion would guard against the regression.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/llm/test_backends/test_openai.py` around lines 1103 - 1121, Add a test
case in the OpenAI Responses tool-choice coverage that passes a named function
such as tool_choice="lookup" through the relevant test setup, then assert
client.responses.stream receives the Responses API function-choice object with
the expected function name rather than the Chat Completions shape. Keep the
existing required-choice assertions and use the existing test symbols and
call-capture pattern.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/llm/backends/openai.py`:
- Around line 153-164: Update the Responses-mode branches in complete() and
stream() to preserve temperature, stop, and thinking_budget_tokens from the
caller, threading them through _complete_responses and _build_responses_params
as supported by the Responses API. If any parameter is intentionally
unsupported, document that explicitly in a concise comment instead of silently
dropping it.
- Line 10: Replace the direct private import of to_strict_json_schema in the
OpenAI backend with a compatibility wrapper that prefers a public
schema-generation API and falls back safely when needed; update all call sites
to use the wrapper so OpenAI SDK upgrades cannot break module import.

In `@tests/llm/test_backends/test_openai.py`:
- Around line 1103-1121: Add a test case in the OpenAI Responses tool-choice
coverage that passes a named function such as tool_choice="lookup" through the
relevant test setup, then assert client.responses.stream receives the Responses
API function-choice object with the expected function name rather than the Chat
Completions shape. Keep the existing required-choice assertions and use the
existing test symbols and call-capture pattern.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 94cdb010-abda-46be-815a-3f59993fe707

📥 Commits

Reviewing files that changed from the base of the PR and between 73453f8 and 17c80ff.

📒 Files selected for processing (2)

• src/llm/backends/openai.py
• tests/llm/test_backends/test_openai.py

[dtownsel]

All LLM tests, Ruff, basedpyright, live structured/tool/streaming probes, and the attribution canary pass. The author has requested immediate merge; this PR is mergeable but requires maintainer review and merge permission.