fix(openai): fallback to json_object when json_schema structured output fails#914
Conversation
…ut fails\n\nOpenAI-compatible providers (e.g. Kimi, MiMo) often accept\nchat.completions.parse() / json_schema but return empty or unparseable\ncontent. Instead of immediately raising or returning empty, retry the\nrequest with structured_output_mode=json_object, which injects the schema\ninto the prompt and lets json_repair recover loose provider JSON.\n\nIf json_object mode also fails, the original exception is re-raised so\nthe retry/fallback chain can still engage as before.
WalkthroughOpenAI structured-output handling now retries with ChangesOpenAI structured-output fallback
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/llm/backends/openai.py`:
- Around line 271-274: Update the exception re-raise in the fallback handler to
use explicit exception chaining suppression, changing the raise of
structured_exc to raise it from None. Preserve the existing exception types and
retry/fallback behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 22225f30-bb52-4000-a668-5681727155d1
📒 Files selected for processing (2)
src/llm/backends/openai.pytests/llm/test_backends/test_openai.py
| except (ValidationException, StructuredOutputError, ValidationError): | ||
| # json_object mode also failed; raise the original exception | ||
| # so the retry/fallback chain can engage as before. | ||
| raise structured_exc |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file="src/llm/backends/openai.py"
echo "== Outline =="
ast-grep outline "$file" --view expanded || true
echo
echo "== Relevant lines =="
sed -n '230,285p' "$file" | cat -n
echo
echo "== Search for similar raise patterns =="
rg -n "raise structured_exc|from None|B904|json_object" "$file" src/llm -g '!**/__pycache__/**' || trueRepository: plastic-labs/honcho
Length of output: 10361
Suppress the fallback exception context
raise structured_exc keeps the JSON-object parsing error in the traceback. Use raise structured_exc from None so the original failure stays cleanly preserved and Ruff B904 is avoided.
Proposed fix
- raise structured_exc
+ raise structured_exc from None📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| except (ValidationException, StructuredOutputError, ValidationError): | |
| # json_object mode also failed; raise the original exception | |
| # so the retry/fallback chain can engage as before. | |
| raise structured_exc | |
| except (ValidationException, StructuredOutputError, ValidationError): | |
| # json_object mode also failed; raise the original exception | |
| # so the retry/fallback chain can engage as before. | |
| raise structured_exc from None |
🧰 Tools
🪛 Ruff (0.15.21)
[warning] 274-274: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/llm/backends/openai.py` around lines 271 - 274, Update the exception
re-raise in the fallback handler to use explicit exception chaining suppression,
changing the raise of structured_exc to raise it from None. Preserve the
existing exception types and retry/fallback behavior.
Source: Linters/SAST tools
|
Finn-loop review of e457e79 CI: no required checks configured — escalate to human ReviewSummary: Adds a fallback from json_schema to json_object mode when OpenAI-compatible providers (Kimi, MiMo) return unparseable structured output. Well-tested with 3 new test cases covering empty, malformed-repairable, and unrepairable content. 1. Must fix before merge[DEFECT] Missing method dependency — The PR must either:
[CI] No required CI configured + stale branch protection 2. Should fix soonDependency on unreviewed fork infrastructure No linked issue 3. Safe to mergeNo — the missing method dependency is a hard blocker. Code quality of the actual change is good: clean error handling, clear test coverage, and sensible re-raise of the original exception on double-failure.
|
3 participants
OpenAI-compatible providers (e.g. Kimi, MiMo) often accept chat.completions.parse() / json_schema but return empty or unparseable content.\n\nThis change retries the request with structured_output_mode=json_object when parse() produces no parseable content. json_object injects the schema into the prompt and lets json_repair recover loose provider JSON.\n\nIf json_object mode also fails, the original exception is re-raised so the retry/fallback chain can still engage.\n\n- Added fallback logic in OpenAIBackend.complete()\n- Added tests for empty content, repairable malformed JSON, and unrepairable content\n\nRelates to: issues where providers return key-must-be-a-string or empty JSON from structured outputs.
Summary by CodeRabbit