LiteLLM Stable release notes (#10919)

* docs(index/v1.70.1-stable): style improvements

* style: add style improvements to docs

* docs: cleanup docs

* docs: more style improvements

* docs: style improvements

* docs(gemini/realtime): add docs on realtime api via Google AI Studio

* docs: add openai example to anthropic web search docs

* docs: add missing doc links

* docs: doc cleanup

* docs: add more doc links

* fix: cleanup

* docs: add docker information

* docs: update doc links

* docs: add demo instance details to doc

s

Author: krrishdholakia

docs/my-website/docs/embedding/supported_embedding.md

@@ -225,36 +225,6 @@ response = embedding(
 | text-embedding-3-large | `embedding('text-embedding-3-large', input)` | `os.environ['OPENAI_API_KEY']`       |
 | text-embedding-ada-002 | `embedding('text-embedding-ada-002', input)` | `os.environ['OPENAI_API_KEY']`       |
 
-## Azure OpenAI Embedding Models
-
-### API keys
-This can be set as env variables or passed as **params to litellm.embedding()**
-```python
-import os
-os.environ['AZURE_API_KEY'] = 
-os.environ['AZURE_API_BASE'] = 
-os.environ['AZURE_API_VERSION'] = 
-```
-
-### Usage
-```python
-from litellm import embedding
-response = embedding(
-    model="azure/<your deployment name>",
-    input=["good morning from litellm"],
-    api_key=api_key,
-    api_base=api_base,
-    api_version=api_version,
-)
-print(response)
-```
-
-| Model Name           | Function Call                               |
-|----------------------|---------------------------------------------|
-| text-embedding-ada-002 | `embedding(model="azure/<your deployment name>", input=input)` |
-
-h/t to [Mikko](https://www.linkedin.com/in/mikkolehtimaki/) for this integration
-
 ## OpenAI Compatible Embedding Models
 Use this for calling `/embedding` endpoints on OpenAI Compatible Servers, example https://github.com/xorbitsai/inference
 

docs/my-website/docs/observability/phoenix_integration.md

@@ -1,6 +1,6 @@
 import Image from '@theme/IdealImage';
 
-# Phoenix OSS
+# Arize Phoenix OSS
 
 Open source tracing and evaluation platform
 

docs/my-website/docs/providers/anthropic.md

@@ -847,13 +847,50 @@ curl http://0.0.0.0:4000/v1/chat/completions \
 <TabItem value="web_search" label="Web Search">
 
 :::info
-
-Unified web search (same param across OpenAI + Anthropic) coming soon!
+Live from v1.70.1+
 :::
 
+LiteLLM maps OpenAI's `search_context_size` param to Anthropic's `max_uses` param.
+
+| OpenAI | Anthropic |
+| --- | --- |
+| Low | 1 | 
+| Medium | 5 | 
+| High | 10 | 
+
+
 <Tabs>
 <TabItem value="sdk" label="SDK">
 
+
+<Tabs>
+<TabItem value="openai" label="OpenAI Format">
+
+```python
+from litellm import completion
+
+model = "claude-3-5-sonnet-20241022"
+messages = [{"role": "user", "content": "What's the weather like today?"}]
+
+resp = completion(
+    model=model,
+    messages=messages,
+    web_search_options={
+        "search_context_size": "medium",
+        "user_location": {
+            "type": "approximate",
+            "approximate": {
+                "city": "San Francisco",
+            },
+        }
+    }
+)
+
+print(resp)
+```
+</TabItem>
+<TabItem value="anthropic" label="Anthropic Format">
+
 ```python
 from litellm import completion
 
@@ -873,8 +910,11 @@ resp = completion(
 
 print(resp)
 ```
+</TabItem>
 
+</Tabs>
 </TabItem>
+
 <TabItem value="proxy" label="PROXY">
 
 1. Setup config.yaml
@@ -894,22 +934,56 @@ litellm --config /path/to/config.yaml
 
 3. Test it! 
 
+<Tabs>
+<TabItem value="openai" label="OpenAI Format">
+
+
 ```bash
 curl http://0.0.0.0:4000/v1/chat/completions \
   -H "Content-Type: application/json" \
   -H "Authorization: Bearer $LITELLM_KEY" \
   -d '{
     "model": "claude-3-5-sonnet-latest",
-    "messages": [{"role": "user", "content": "There's a syntax error in my primes.py file. Can you help me fix it?"}],
-    "tools": [{"type": "web_search_20250305", "name": "web_search", "max_uses": 5}]
+    "messages": [{"role": "user", "content": "What's the weather like today?"}],
+    "web_search_options": {
+        "search_context_size": "medium",
+        "user_location": {
+            "type": "approximate",
+            "approximate": {
+                "city": "San Francisco",
+            },
+        }
+    }
+  }'
+```
+</TabItem>
+<TabItem value="anthropic" label="Anthropic Format">
+
+```bash
+curl http://0.0.0.0:4000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $LITELLM_KEY" \
+  -d '{
+    "model": "claude-3-5-sonnet-latest",
+    "messages": [{"role": "user", "content": "What's the weather like today?"}],
+    "tools": [{
+        "type": "web_search_20250305",
+        "name": "web_search",
+        "max_uses": 5
+    }]
   }'
 ```
+
+</TabItem>
+</Tabs>
 </TabItem>
 </Tabs>
 
 </TabItem>
 </Tabs>
 
+
+
 ## Usage - Vision 
 
 ```python

docs/my-website/docs/providers/azure.md renamed to docs/my-website/docs/providers/azure/azure.md

@@ -11,7 +11,7 @@ import TabItem from '@theme/TabItem';
 |-------|-------|
 | Description | Azure OpenAI Service provides REST API access to OpenAI's powerful language models including o1, o1-mini, GPT-4o, GPT-4o mini, GPT-4 Turbo with Vision, GPT-4, GPT-3.5-Turbo, and Embeddings model series |
 | Provider Route on LiteLLM | `azure/`, [`azure/o_series/`](#azure-o-series-models) |
-| Supported Operations | [`/chat/completions`](#azure-openai-chat-completion-models), [`/completions`](#azure-instruct-models), [`/embeddings`](../embedding/supported_embedding#azure-openai-embedding-models), [`/audio/speech`](#azure-text-to-speech-tts), [`/audio/transcriptions`](../audio_transcription), `/fine_tuning`, [`/batches`](#azure-batches-api), `/files`, [`/images`](../image_generation#azure-openai-image-generation-models) |
+| Supported Operations | [`/chat/completions`](#azure-openai-chat-completion-models), [`/completions`](#azure-instruct-models), [`/embeddings`](./azure_embedding), [`/audio/speech`](#azure-text-to-speech-tts), [`/audio/transcriptions`](../audio_transcription), `/fine_tuning`, [`/batches`](#azure-batches-api), `/files`, [`/images`](../image_generation#azure-openai-image-generation-models) |
 | Link to Provider Doc | [Azure OpenAI ↗](https://learn.microsoft.com/en-us/azure/ai-services/openai/overview)
 
 ## API Keys, Params

docs/my-website/docs/providers/azure/azure_embedding.md

@@ -0,0 +1,93 @@
+import Image from '@theme/IdealImage';
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Azure OpenAI Embeddings
+
+### API keys
+This can be set as env variables or passed as **params to litellm.embedding()**
+```python
+import os
+os.environ['AZURE_API_KEY'] = 
+os.environ['AZURE_API_BASE'] = 
+os.environ['AZURE_API_VERSION'] = 
+```
+
+### Usage
+```python
+from litellm import embedding
+response = embedding(
+    model="azure/<your deployment name>",
+    input=["good morning from litellm"],
+    api_key=api_key,
+    api_base=api_base,
+    api_version=api_version,
+)
+print(response)
+```
+
+| Model Name           | Function Call                               |
+|----------------------|---------------------------------------------|
+| text-embedding-ada-002 | `embedding(model="azure/<your deployment name>", input=input)` |
+
+h/t to [Mikko](https://www.linkedin.com/in/mikkolehtimaki/) for this integration
+
+
+## **Usage - LiteLLM Proxy Server**
+
+Here's how to call Azure OpenAI models with the LiteLLM Proxy Server
+
+### 1. Save key in your environment
+
+```bash
+export AZURE_API_KEY=""
+```
+
+### 2. Start the proxy 
+
+```yaml
+model_list:
+  - model_name: text-embedding-ada-002
+    litellm_params:
+      model: azure/my-deployment-name
+      api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
+      api_version: "2023-05-15"
+      api_key: os.environ/AZURE_API_KEY # The `os.environ/` prefix tells litellm to read this from the env.
+```
+
+### 3. Test it
+
+<Tabs>
+<TabItem value="Curl" label="Curl Request">
+
+```shell
+curl --location 'http://0.0.0.0:4000/embeddings' \
+  --header 'Content-Type: application/json' \
+  --data ' {
+  "model": "text-embedding-ada-002",
+  "input": ["write a litellm poem"]
+  }'
+```
+</TabItem>
+<TabItem value="openai" label="OpenAI v1.0.0+">
+
+```python
+import openai
+from openai import OpenAI
+
+# set base_url to your proxy server
+# set api_key to send to proxy server
+client = OpenAI(api_key="<proxy-api-key>", base_url="http://0.0.0.0:4000")
+
+response = client.embeddings.create(
+    input=["hello from litellm"],
+    model="text-embedding-ada-002"
+)
+
+print(response)
+
+```
+</TabItem>
+</Tabs>
+
+

docs/my-website/docs/providers/google_ai_studio/realtime.md

@@ -0,0 +1,92 @@
+# Gemini Realtime API - Google AI Studio
+
+| Feature | Description | Comments |
+| --- | --- | --- |
+| Proxy | ✅ |  |
+| SDK | ⌛️ | Experimental access via `litellm._arealtime`. |
+
+
+## Proxy Usage
+
+### Add model to config 
+
+```yaml
+model_list:
+  - model_name: "gemini-2.0-flash"
+    litellm_params:
+      model: gemini/gemini-2.0-flash-live-001
+    model_info:
+      mode: realtime
+```
+
+### Start proxy 
+
+```bash
+litellm --config /path/to/config.yaml 
+
+# RUNNING on http://0.0.0.0:8000
+```
+
+### Test 
+
+Run this script using node - `node test.js`
+
+```js
+// test.js
+const WebSocket = require("ws");
+
+const url = "ws://0.0.0.0:4000/v1/realtime?model=openai-gemini-2.0-flash";
+
+const ws = new WebSocket(url, {
+    headers: {
+        "api-key": `${LITELLM_API_KEY}`,
+        "OpenAI-Beta": "realtime=v1",
+    },
+});
+
+ws.on("open", function open() {
+    console.log("Connected to server.");
+    ws.send(JSON.stringify({
+        type: "response.create",
+        response: {
+            modalities: ["text"],
+            instructions: "Please assist the user.",
+        }
+    }));
+});
+
+ws.on("message", function incoming(message) {
+    console.log(JSON.parse(message.toString()));
+});
+
+ws.on("error", function handleError(error) {
+    console.error("Error: ", error);
+});
+```
+
+## Limitations 
+
+- Does not support audio transcription. 
+- Does not support tool calling 
+
+## Supported OpenAI Realtime Events
+
+- `session.created`
+- `response.created`
+- `response.output_item.added`
+- `conversation.item.created`
+- `response.content_part.added`
+- `response.text.delta`
+- `response.audio.delta`
+- `response.text.done`
+- `response.audio.done`
+- `response.content_part.done`
+- `response.output_item.done`
+- `response.done`
+
+
+
+## [Supported Session Params](https://github.com/BerriAI/litellm/blob/e87b536d038f77c2a2206fd7433e275c487179ee/litellm/llms/gemini/realtime/transformation.py#L155)
+
+## More Examples
+### [Gemini Realtime API with Audio Input/Output](../../../docs/tutorials/gemini_realtime_with_audio)

docs/my-website/docs/providers/litellm_proxy.md

@@ -163,9 +163,11 @@ LiteLLM Proxy works seamlessly with Langchain, LlamaIndex, OpenAI JS, Anthropic
 
 [Learn how to use LiteLLM proxy with these libraries →](../proxy/user_keys)
 
-## Flags to send requests to litellm proxy
+## Send all SDK requests to LiteLLM Proxy
 
-Use the following options to route all requests through your LiteLLM proxy, regardless of the model specified.
+Use this when calling LiteLLM Proxy from any library / codebase already using the LiteLLM SDK.
+
+These flags will route all requests through your LiteLLM proxy, regardless of the model specified.
 
 When enabled, requests will use `LITELLM_PROXY_API_BASE` with `LITELLM_PROXY_API_KEY` as the authentication.