Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.71.1-stable.patch1
Full Changelog: v1.71.1-nightly...v1.71.1-stable.patch1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.71.1-stable.patch1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 208.76252080483792 | 6.21531894936511 | 0.0 | 1860 | 0 | 167.59147999994184 | 1521.5310909999857 |
| Aggregated | Passed ✅ | 190.0 | 208.76252080483792 | 6.21531894936511 | 0.0 | 1860 | 0 | 167.59147999994184 | 1521.5310909999857 |
v1.71.2.dev2
Full Changelog: v1.71.2-nightly...v1.71.2.dev2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.2.dev2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 240.0 | 257.64197439032404 | 6.082707498887239 | 0.0 | 1819 | 0 | 211.94133999995302 | 1432.5384599999893 |
| Aggregated | Passed ✅ | 240.0 | 257.64197439032404 | 6.082707498887239 | 0.0 | 1819 | 0 | 211.94133999995302 | 1432.5384599999893 |
v1.71.2.dev1
What's Changed
- feat: Allow Adding MCP Servers Through LiteLLM UI by @ishaan-jaff in #11208
- Add price for gpt4o mini audio preview without date by @Jacobh2 in #11207
- Put "all proxy models" if no models are selected by @NANDINI-star in #11187
- Removed teamInfo call and used existing teamsList call by @NANDINI-star in #11051
- build(deps): bump next from 14.2.26 to 15.2.4 in /ui/litellm-dashboard by @dependabot in #11216
- Improve display of model tags on Keys, Teams and Org page by @NANDINI-star in #11022
- Update mistral-medium prices and context sizes by @regismesquita in #10729
- Add sentry sample rate by @Vinnie-Singleton-NN in #10283
- [Feat] Permission management vector stores on LiteLLM Key, Team, Orgs by @ishaan-jaff in #11213
- Fix: Remove duplicate object_permission field in LiteLLM_TeamTable by @trashhalo in #11219
- [UI] SCIM fixes SCIM token exposed on UI by @ishaan-jaff in #11220
New Contributors
- @Vinnie-Singleton-NN made their first contribution in #10283
- @trashhalo made their first contribution in #11219
Full Changelog: v1.71.2-nightly...v1.71.2.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.2.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 220.0 | 238.87666312181884 | 6.174193914087743 | 0.0033428229096306135 | 1847 | 1 | 50.488271999995504 | 1443.0136089999905 |
| Aggregated | Passed ✅ | 220.0 | 238.87666312181884 | 6.174193914087743 | 0.0033428229096306135 | 1847 | 1 | 50.488271999995504 | 1443.0136089999905 |
Contributors
regismesquita, trashhalo, and 5 other contributors
Assets 4
v1.71.2-nightly
What's Changed
- feat: Make gemini accept the openai parameter parallel_tool_calls by @aholmberg in #11125
- Fix #9295 docker-compose healthcheck test uses curl but curl is not in the image by @agajdosi in #9737
- [Feat] Add /image/edits support for Azure by @ishaan-jaff in #11160
- Fix deprecation_date value for llama groq models by @kiriloman in #11151
- [Fix] Rollback to httpx==0.27.0 by @ishaan-jaff in #11146
- Doc update for azure openai by @ketangangal in #11161
- Litellm fix GitHub action testing by @krrishdholakia in #11163
- [Feat - Contributor PR] Add Video support for Bedrock Converse by @ishaan-jaff in #11166
- [Fixes] Aiohttp transport fixes - add handling for
aiohttp.ClientPayloadErrorand ssl_verification settings by @ishaan-jaff in #11162 - prevent leaking sensitive keys to langfuse + support forwarding
/sso/key/generateto the server root path url by @krrishdholakia in #11165 - [Fix] - Duplicate maxTokens parameter being sent to Bedrock/Claude model with thinking by @ishaan-jaff in #11181
- Integration with Nebius AI Studio added by @Aktsvigun in #11143
- Codestral - return litellm latency overhead on
/v1/completions+ Add 'contains' support for ChatCompletionDeltaToolCall by @krrishdholakia in #10879 - Ollama Chat - parse tool calls on streaming by @krrishdholakia in #11171
- [Fix] Prometheus Metrics - Do not track end_user by default + expose flag to enable tracking end_user on prometheus by @ishaan-jaff in #11192
- [Fix]: Add cost tracking for image edits endpoint [OpenAI, Azure] by @ishaan-jaff in #11186
- VertexAI -
codeExecutiontool support + anyOf handling by @krrishdholakia in #11195 - Add Pangea provider to Guardrails hook by @ryanmeans in #10775
- Return anthropic thinking blocks on streaming + VertexAI Minor Fixes & Improvements (Thinking, Global regions, Parallel tool calling) by @krrishdholakia in #11194
- Azure OIDC provider improvements + OIDC audience bug fix by @nikoizs in #10054
- [Feat] Add well known MCP servers to LiteLLM by @ishaan-jaff in #11209
- Add missing
request_kwargstoget_available_deploymentcall by @Nitro963 in #11202
New Contributors
- @agajdosi made their first contribution in #9737
- @ketangangal made their first contribution in #11161
- @Aktsvigun made their first contribution in #11143
- @ryanmeans made their first contribution in #10775
- @nikoizs made their first contribution in #10054
- @Nitro963 made their first contribution in #11202
Full Changelog: v1.71.1-nightly...v1.71.2-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.2-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 230.0 | 296.34506371491256 | 6.07338921380297 | 0.0 | 1817 | 0 | 196.8864309999958 | 5947.878826000022 |
| Aggregated | Passed ✅ | 230.0 | 296.34506371491256 | 6.07338921380297 | 0.0 | 1817 | 0 | 196.8864309999958 | 5947.878826000022 |
Contributors
aholmberg, ryanmeans, and 8 other contributors
Assets 4
v1.71.1-stable
What's Changed
- add docs for spend logs by @jugaldb in #10913
- [QA/Guardrails] Allow editing guardrail entity configs by @ishaan-jaff in #10918
- [UI QA Guardrails] - Minor UI Fixes by @ishaan-jaff in #10920
- [CI/CD] Fix - linting cryptography error by @ishaan-jaff in #10921
- LiteLLM Stable release notes by @krrishdholakia in #10919
- docs apply guardrail endpoint by @ishaan-jaff in #10923
- [Docs] Pii masking with LiteLLM by @ishaan-jaff in #10924
- Add provider Featherless AI (access to 4200+ models) by @DarinVerheijke in #10596
- fix(vertex_llm_base.py): remove quota_project_id parameter from crede… by @krrishdholakia in #10915
- Enable structured JSON schema outputs in LM Studio by @estsauver in #10929
- update sambanova models and parameters by @jhpiedrahitao in #10900
- Fix Cohere Rerank Provider by @keyute in #10822
- add skip server startup flag to cli by @mohittalele in #10665
- Allow passed in vertex_ai credentials to be authorized_user type by @pselden in #10899
- fix: handle DB_USER, DB_PASSWORD, DB_HOST problem I faced, since this… by @unrealandychan in #10842
- Add keys and members count in team info by @NANDINI-star in #10950
- Update github.md by @dastaiger in #10946
- docs: add llms.txt and llms_full.txt for LLM-friendly scraping by @colesmcintosh in #10961
- [Fix] Invitation Email does not include the invitation link by @ishaan-jaff in #10958
- Enable key reassignment on UI + Show 'updated at' column for key in all keys table by @krrishdholakia in #10960
- Update existing model's model info on UI + Show model access groups when creating keys/teams on UI by @krrishdholakia in #10965
- Validate migrating keys to teams + Fix mistral image url on async translation by @krrishdholakia in #10966
- add cla to docs by @jugaldb in #10963
- [Fix] List Guardrails - Show config.yaml guardrails on litellm ui by @ishaan-jaff in #10959
- fix: vertex show clear exception on failed refresh by @ishaan-jaff in #10969
- [Feature] Add supports_computer_use to the model list by @slytechnical in #10881
- [Feat] - Add Support for Showing Passthrough endpoint Error Logs on LiteLLM UI by @ishaan-jaff in #10990
- added support to credential delete to support slashes in the curl by @jugaldb in #10987
- Add new gemini preview models + Fix cohere v2 embedding 'embedding_types' param mapping by @krrishdholakia in #10991
- Litellm add new gemini models by @krrishdholakia in #10998
- [Feat] Prometheus - Track
routeon proxy_* metrics by @ishaan-jaff in #10992 - fix: default role for JWT authentication by @ishaan-jaff in #10995
- fix(internal_user_endpoints.py): allow resetting spend/max budget on
/user/update+ Fix updating user budget across multiple instances by @krrishdholakia in #10993 - Improve response_id propagation logic and add tests for valid/empty I… by @daarko10 in #11006
- Support vertex_ai global endpoints for chat by @sorenmat in #10658
- Ollama wildcard support by @matthid in #10982
- Spend rounded to 4 decimal points on Teams page by @NANDINI-star in #11013
- Place organization and team create buttons at the top by @NANDINI-star in #10948
- Feat: add xai/grok-3 pricing by @ishaan-jaff in #11028
- [Feat] Add Image Edits Support to LiteLLM by @ishaan-jaff in #11020
- Updating the available VoyageAI models in the docs by @fzowl in #11003
- Fix UI - Call tool renders correctly when no arguments required by @wagnerjt in #11012
- Verbose error on admin add by @jugaldb in #10978
- Spend rounded to 4 for Organizations and Users page by @NANDINI-star in #11023
- Fix: Handle dict objects in Anthropic streaming response by @jgowdy-godaddy in #11032
- feat: add Databricks Llama 4 Maverick model cost by @bepotp in #11008
- Add LiteLLM Managed Files support for finetuning api
createendpoint by @krrishdholakia in #11039 - Fixes the InvitationLink Prisma find_many query by @jmorenoc-o in #11031
- Support passing
prompt_labelto langfuse + Log consistent modelParams to langfuse by @krrishdholakia in #11018 - Fix CI | Added closing tags for + indentation changes by @NANDINI-star in #11046
- Feat: add MCP to Responses API and bump openai python sdk by @ishaan-jaff in #11029
- Model filter on logs by @NANDINI-star in #11048
- [Feat] Add claude-4 model family by @ishaan-jaff in #11060
- Litellm managed file updates combined by @krrishdholakia in #11040
- [Fix] Reliability Fix - Removing code that was creating threads on errors by @ishaan-jaff in #11066
- [Feat] Add Azure AD certificate-based authentication by @ishaan-jaff in #11069
- Update feature_request.yml by @ishaan-jaff in #11078
- adds tzdata (#10796) by @krrishdholakia in #11052
- Fix proxy_cli.py: avoid overriding DATABASE_URL when it’s already provided by @martin-liu in #11076
- feat(helm): Add loadBalancerClass support for LoadBalancer services by @gunjan-solanki in #11064
- Add Azure Mistral Medium 25.05 by @emerzon in #11063
- fix:Databricks Claude 3.7 Sonnet output token cost: $17.85/M by @bepotp in #11007
- Fix/openrouter stream usage id 8913 by @daarko10 in #11004
- feat: add embeddings to CustomLLM by @tokoko in #10980
- Enable switching between custom auth and litellm api key auth + Fix
/customer/updatefor max budgets by @krrishdholakia in #11070 - Validate if user can call
retrieve+list+deletefile by @krrishdholakia in #11081 - [feature] ConfidentAI logging enabled for proxy and sdk by @spike-spiegel-21 in #10649
- Proper github images by @kreatoo in #10927
- Add devstral-small-2505 model to pricing and context window configuration by @xingyaoww in #11103
- [Ci/CD] use n 4 for mapped tests by @ishaan-jaff in #11109
- Fix/background health check by @baejooc in #10887
- [Feat] Add content policy violation error mapping for image edits by @ishaan-jaff in #11113
- Expose
/listand/infoendpoints for Audit Log events by @krrishdholakia in #11102 - [Feat] Use aiohttp transport by default - 97% lower median latency by @ishaan-jaff in #11097
- Logfire - fix(opentelemetry.py): Fix otel proxy server initialization + Return abbreviated key in key not found error (easier clientside debugging) + Ignore invalid deployments on router load by @krrishdholakia in #11091
- feat(handle_jwt.py): map user to team when added via jwt auth by @krrishdholakia in #11108
- fix(ui_sso.py): maintain backwards compatibility for older user id formats + fix existing user email w/ trailing whitespace check + ensure default_internal_user_settings runs on all user new calls by @krrishdholakia in #11106
- fix(route_llm_request.py): map team model from list in route llm request by @krrishdholakia in #11111
- Remove + Check for unsafe
enterprise/folder imports by @krrishdholakia in #11107 - Fix: Add Claude Sonnet 4 and Opus 4 support for reasoning_effort parameter by @keykbd in #1...
Contributors
dalssoft, sorenmat, and 32 other contributors
Assets 4
v1.71.1-nightly
What's Changed
- Logfire - fix(opentelemetry.py): Fix otel proxy server initialization + Return abbreviated key in key not found error (easier clientside debugging) + Ignore invalid deployments on router load by @krrishdholakia in #11091
- feat(handle_jwt.py): map user to team when added via jwt auth by @krrishdholakia in #11108
- fix(ui_sso.py): maintain backwards compatibility for older user id formats + fix existing user email w/ trailing whitespace check + ensure default_internal_user_settings runs on all user new calls by @krrishdholakia in #11106
- fix(route_llm_request.py): map team model from list in route llm request by @krrishdholakia in #11111
- Remove + Check for unsafe
enterprise/folder imports by @krrishdholakia in #11107 - Fix: Add Claude Sonnet 4 and Opus 4 support for reasoning_effort parameter by @keykbd in #11114
- fix(session): correctly place litellm_session_id at root level instead of metadata by @dalssoft in #11088
- fix(model_management_endpoints): clear cache and reload models after update by @jtong99 in #10853
- [Feat] Add /image/edits on LiteLLM by @ishaan-jaff in #11123
- Correctly delete team model alias when team only model is deleted (#… by @krrishdholakia in #11121
- fix: detect and return status codes in streaming responses by @aholmberg in #10962
- Fix passing standard optional params by @krrishdholakia in #11124
- UI QA fix: team viewer should not see create team by @ishaan-jaff in #11127
- [Chore]: feature flag aiohttp transport - users should opt into using aiohttp transport by @ishaan-jaff in #11132
- v1.71.1-stable - notes by @ishaan-jaff in #11133
- Litellm revert redis changes by @krrishdholakia in #11135
- Litellm fix multi instance checks on teams by @krrishdholakia in #11137
New Contributors
- @keykbd made their first contribution in #11114
- @dalssoft made their first contribution in #11088
- @jtong99 made their first contribution in #10853
Full Changelog: v1.71.0-nightly...v1.71.1-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.1-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 271.62186726419185 | 6.123952252359233 | 0.0 | 1832 | 0 | 215.75241199997208 | 1968.6522410000293 |
| Aggregated | Passed ✅ | 250.0 | 271.62186726419185 | 6.123952252359233 | 0.0 | 1832 | 0 | 215.75241199997208 | 1968.6522410000293 |
Contributors
dalssoft, aholmberg, and 4 other contributors
Assets 4
v1.71.0-nightly
What's Changed
- Litellm managed file updates combined by @krrishdholakia in #11040
- [Fix] Reliability Fix - Removing code that was creating threads on errors by @ishaan-jaff in #11066
- [Feat] Add Azure AD certificate-based authentication by @ishaan-jaff in #11069
- Update feature_request.yml by @ishaan-jaff in #11078
- adds tzdata (#10796) by @krrishdholakia in #11052
- Fix proxy_cli.py: avoid overriding DATABASE_URL when it’s already provided by @martin-liu in #11076
- feat(helm): Add loadBalancerClass support for LoadBalancer services by @gunjan-solanki in #11064
- Add Azure Mistral Medium 25.05 by @emerzon in #11063
- fix:Databricks Claude 3.7 Sonnet output token cost: $17.85/M by @bepotp in #11007
- Fix/openrouter stream usage id 8913 by @daarko10 in #11004
- feat: add embeddings to CustomLLM by @tokoko in #10980
- Enable switching between custom auth and litellm api key auth + Fix
/customer/updatefor max budgets by @krrishdholakia in #11070 - Validate if user can call
retrieve+list+deletefile by @krrishdholakia in #11081 - [feature] ConfidentAI logging enabled for proxy and sdk by @spike-spiegel-21 in #10649
- Proper github images by @kreatoo in #10927
- Add devstral-small-2505 model to pricing and context window configuration by @xingyaoww in #11103
- [Ci/CD] use n 4 for mapped tests by @ishaan-jaff in #11109
- Fix/background health check by @baejooc in #10887
- [Feat] Add content policy violation error mapping for image edits by @ishaan-jaff in #11113
- Expose
/listand/infoendpoints for Audit Log events by @krrishdholakia in #11102 - [Feat] Use aiohttp transport by default - 97% lower median latency by @ishaan-jaff in #11097
New Contributors
- @martin-liu made their first contribution in #11076
- @gunjan-solanki made their first contribution in #11064
- @tokoko made their first contribution in #10980
- @spike-spiegel-21 made their first contribution in #10649
- @kreatoo made their first contribution in #10927
- @baejooc made their first contribution in #10887
Full Changelog: v1.70.4-nightly...v1.71.0-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 310.0 | 321.9030490306742 | 5.995437687618243 | 4.166377779571852 | 1793 | 1246 | 259.8663090000173 | 771.521746000019 |
| Aggregated | Failed ❌ | 310.0 | 321.9030490306742 | 5.995437687618243 | 4.166377779571852 | 1793 | 1246 | 259.8663090000173 | 771.521746000019 |
Contributors
martin-liu, bepotp, and 10 other contributors
Assets 4
v1.70.2.dev6
Full Changelog: v1.70.2-nightly...v1.70.2.dev6
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.2.dev6
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 470.0 | 546.8056516133963 | 5.687025964404996 | 0.0 | 1702 | 0 | 432.45771799996646 | 2108.0635040000006 |
| Aggregated | Failed ❌ | 470.0 | 546.8056516133963 | 5.687025964404996 | 0.0 | 1702 | 0 | 432.45771799996646 | 2108.0635040000006 |
v1.70.4-nightly
Full Changelog: v1.70.2.dev5...v1.70.4-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.4-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 530.0 | 592.8290290780234 | 5.619046633443039 | 0.0 | 1679 | 0 | 480.5262289999632 | 1595.1236809999614 |
| Aggregated | Failed ❌ | 530.0 | 592.8290290780234 | 5.619046633443039 | 0.0 | 1679 | 0 | 480.5262289999632 | 1595.1236809999614 |
v1.70.2.dev5
What's Changed
- [Feature] Add supports_computer_use to the model list by @slytechnical in #10881
- [Feat] - Add Support for Showing Passthrough endpoint Error Logs on LiteLLM UI by @ishaan-jaff in #10990
- added support to credential delete to support slashes in the curl by @jugaldb in #10987
- Add new gemini preview models + Fix cohere v2 embedding 'embedding_types' param mapping by @krrishdholakia in #10991
- Litellm add new gemini models by @krrishdholakia in #10998
- [Feat] Prometheus - Track
routeon proxy_* metrics by @ishaan-jaff in #10992 - fix: default role for JWT authentication by @ishaan-jaff in #10995
- fix(internal_user_endpoints.py): allow resetting spend/max budget on
/user/update+ Fix updating user budget across multiple instances by @krrishdholakia in #10993 - Improve response_id propagation logic and add tests for valid/empty I… by @daarko10 in #11006
- Support vertex_ai global endpoints for chat by @sorenmat in #10658
- Ollama wildcard support by @matthid in #10982
- Spend rounded to 4 decimal points on Teams page by @NANDINI-star in #11013
- Place organization and team create buttons at the top by @NANDINI-star in #10948
- Feat: add xai/grok-3 pricing by @ishaan-jaff in #11028
- [Feat] Add Image Edits Support to LiteLLM by @ishaan-jaff in #11020
- Updating the available VoyageAI models in the docs by @fzowl in #11003
- Fix UI - Call tool renders correctly when no arguments required by @wagnerjt in #11012
- Verbose error on admin add by @jugaldb in #10978
- Spend rounded to 4 for Organizations and Users page by @NANDINI-star in #11023
- Fix: Handle dict objects in Anthropic streaming response by @jgowdy-godaddy in #11032
- feat: add Databricks Llama 4 Maverick model cost by @bepotp in #11008
- Add LiteLLM Managed Files support for finetuning api
createendpoint by @krrishdholakia in #11039 - Fixes the InvitationLink Prisma find_many query by @jmorenoc-o in #11031
- Support passing
prompt_labelto langfuse + Log consistent modelParams to langfuse by @krrishdholakia in #11018 - Fix CI | Added closing tags for + indentation changes by @NANDINI-star in #11046
- Feat: add MCP to Responses API and bump openai python sdk by @ishaan-jaff in #11029
- Model filter on logs by @NANDINI-star in #11048
- [Feat] Add claude-4 model family by @ishaan-jaff in #11060
New Contributors
- @slytechnical made their first contribution in #10881
- @daarko10 made their first contribution in #11006
- @sorenmat made their first contribution in #10658
- @matthid made their first contribution in #10982
- @jgowdy-godaddy made their first contribution in #11032
- @bepotp made their first contribution in #11008
- @jmorenoc-o made their first contribution in #11031
Full Changelog: v1.70.2-nightly...v1.70.2.dev5
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.2.dev5
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 490.0 | 556.7359308349169 | 5.626256137716844 | 0.0 | 1684 | 0 | 437.79858300001706 | 2137.070654000013 |
| Aggregated | Failed ❌ | 490.0 | 556.7359308349169 | 5.626256137716844 | 0.0 | 1684 | 0 | 437.79858300001706 | 2137.070654000013 |
Contributors
sorenmat, matthid, and 11 other contributors