fix(bedrock): wrong system prompt transformation

[hewliyang]

Title

fix(bedrock): wrong system transformation

Relevant issues

resolves #9969

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on (make test-unit)[https://docs.litellm.ai/docs/extras/contributing_code]
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Bug Description

for m in message["content"]:
    if m["type"] == "text":
        _system_content_block = SystemContentBlock(text=m["text"])
        _cache_point_block   = self._get_cache_point_block(m, block_type="system")
# …after the loop…
if _system_content_block is not None:
    system_content_blocks.append(_system_content_block)
if _cache_point_block is not None:
    system_content_blocks.append(_cache_point_block)

the original code would only append the last iterable content block, meaning a input like this:

messages = [
    {
        "role": "system",
        "content": [
            {
                "type": "text",
                "text": "The user's name is Li Yang and he is 5 years old." * 100,
                "cache_control": {"type": "ephemeral"},
            }, # <- this block is lost
            {"type": "text", "text": "He likes baking."},
        ],
    },
    {
        "role": "user",
        "content": "What is my name?"
    },
]

would result in the following request payload

{"messages": [{"role": "user", "content": [{"text": "What is my name?"}]}], "additionalModelRequestFields": {}, "system": [{"text": "He likes baking."}], "inferenceConfig": {}}'

Changes

• Append blocks correctly to the final output
• Added a unit test for this method

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 19, 2025 6:30pm