Add /vllm and /mistral passthrough endpoints

[krrishdholakia]

• feat(llm_passthrough_endpoints.py): support mistral passthrough

Closes #9051

• feat(llm_passthrough_endpoints.py): initial commit for adding vllm passthrough route

• feat(vllm/common_utils.py): add new vllm model info route

make it possible to use vllm passthrough route via factory function

Example 1: Call Mistral OCR

curl -L -X POST 'http://0.0.0.0:4000/mistral/v1/ocr' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
    "model": "mistral-ocr-latest",
    "document": {
        "type": "image_url",
        "image_url": "https://raw.githubusercontent.com/mistralai/cookbook/refs/heads/main/mistral/ocr/receipt.png"
    }

}'

Example 2: Call VLLM /metrics endpoint

curl -L -X GET 'http://0.0.0.0:4000/vllm/metrics' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 15, 2025 5:06am