[Feature]: Bedrock Guardrails: Allow for Masked Sensitive Information Prompts to Continue

[marty-sullivan]

The Feature

When configuring a Bedrock Guardrail for Sensitive Information (PII) to be masked rather than blocked, triggering the guardrail will cause LiteLLM to throw a 400 error due to the triggered Guardrail, even though the prompt is designed to be masked and the request to continue.

To reproduce:

1. Configure a Bedrock Guardrail to block Sensitive Information (cloudformation below)
2. Configure the guardrail in your config.yaml
3. Attempt to trigger the guardrail by providing a prompt containing fake name, fake ssn and fake phone number.
4. The current behavior is to block if any guardrail is triggered rather than to continue the process of prompting, or to return the result with PII masked.

Demonstration of current behavior:

CloudFormation for Bedrock Guardrail:

 MaskPiiGuardrail:
    Type: 'AWS::Bedrock::Guardrail'
    Properties:
      Name: 'mask-pii'
      Description: 'Guardrail for detecting and masking PII data'
      BlockedInputMessaging: 'Your prompt has triggered a content filter, please check for PII data in your prompt and try again.'
      BlockedOutputsMessaging: 'The model response has triggered a content filter, please check for PII data in your prompt and try again.'
      SensitiveInformationPolicyConfig:
        PiiEntitiesConfig:
          - Type: 'ADDRESS'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'AGE'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'AWS_ACCESS_KEY'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'AWS_SECRET_KEY'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'CA_HEALTH_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'CA_SOCIAL_INSURANCE_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'CREDIT_DEBIT_CARD_CVV'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'CREDIT_DEBIT_CARD_EXPIRY'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'CREDIT_DEBIT_CARD_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'DRIVER_ID'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'EMAIL'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'INTERNATIONAL_BANK_ACCOUNT_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'IP_ADDRESS'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'LICENSE_PLATE'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'MAC_ADDRESS'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'NAME'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'PASSWORD'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'PHONE'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'PIN'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'SWIFT_CODE'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'UK_NATIONAL_HEALTH_SERVICE_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'UK_NATIONAL_INSURANCE_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'UK_UNIQUE_TAXPAYER_REFERENCE_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'URL'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'US_BANK_ACCOUNT_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'US_BANK_ROUTING_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'US_INDIVIDUAL_TAX_IDENTIFICATION_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'US_PASSPORT_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'US_SOCIAL_SECURITY_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'USERNAME'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true
          - Type: 'VEHICLE_IDENTIFICATION_NUMBER'
            Action: 'ANONYMIZE'
            InputAction: 'ANONYMIZE'
            InputEnabled: true
            OutputAction: 'ANONYMIZE'
            OutputEnabled: true

Config.yaml:

  - guardrail_name: 'pii'
    litellm_params:
      guardrail: 'bedrock'
      mode: 'pre_call'
      guardrailIdentifier: os.environ/BEDROCK_GUARDRAIL_MASK_PII_ID
      guardrailVersion: os.environ/BEDROCK_GUARDRAIL_MASK_PII_VERSION
      aws_region_name: 'us-east-1'

Python code to trigger the guardrail:

from openai import OpenAI, BadRequestError
from os import getenv

client = OpenAI(
    base_url=getenv('OPENAI_BASE_URL'),
    api_key=getenv('OPENAI_API_KEY'),
)

model_name = 'google.gemini-2.5-flash'

# Test PII Guardrail

pii_prompt = 'My name is John Smith and my SSN is 324-12-3212 and my phone number is 607-456-7890. Can you please repeat to me my name and phone number?'

try:
    response = client.chat.completions.create(
        model=model_name,
        messages=[
            {
                'role': 'user', 
                'content': pii_prompt,
            },
        ],
        extra_body=dict(
            guardrails=[
                'pii',
            ]),

    )

    print(response.model_dump_json(indent=2))

except BadRequestError as e:
    print(e)

Motivation, pitch

Many workloads may contain data that is expected to have high-risk fields and it makes sense to simply mask this data from prompts & completions rather than completely blocking the requests.

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response

[ishaan-jaff]

@marty-sullivan I fixed this on the latest release. Can you please test and confirm.

[ishaan-jaff]

fix here #11875

[marty-sullivan]

@ishaan-jaff Your update works well for mode: pre_call but it seems to have an issue with masking in mode: post_call. In the example below, it seems like it puts the prompt into the model response content rather than the model's actual response.

Example code:

pii_prompt = 'My name is John Smith and my SSN is 324-12-3212 and my phone number is 607-456-7890. Can you please repeat to me my name and phone number?'

try:
    response = client.chat.completions.create(
        model=model_name,
        messages=[
            {
                'role': 'user', 
                'content': pii_prompt,
            },
        ],
        extra_body=dict(
            guardrails=[
                # 'mask_pii_pre_call',
                'mask_pii_post_call',
            ]),

    )

    print(response.model_dump_json(indent=2))

except BadRequestError as e:
    print(e)

Example Response:

{
  "id": "-oJjaJahI-ftlOoP7eCg4QI",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "message": {
        "content": "My name is {NAME} and my SSN is {US_SOCIAL_SECURITY_NUMBER} and my phone number is {PHONE}. Can you please repeat to me my name and phone number?",
        "refusal": null,
        "role": "assistant",
        "annotations": null,
        "audio": null,
        "function_call": null,
        "tool_calls": null
      }
    }
  ],
  "created": 1751352058,
  "model": "gemini-2.5-flash",
  "object": "chat.completion",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "completion_tokens": 436,
    "prompt_tokens": 52,
    "total_tokens": 488,
    "completion_tokens_details": {
      "accepted_prediction_tokens": null,
      "audio_tokens": null,
      "reasoning_tokens": 400,
      "rejected_prediction_tokens": null
    },
    "prompt_tokens_details": {
      "audio_tokens": null,
      "cached_tokens": null
    }
  },
  "vertex_ai_grounding_metadata": [],
  "vertex_ai_url_context_metadata": [],
  "vertex_ai_safety_results": [],
  "vertex_ai_citation_metadata": []
}