Skip to content

[Feature] Graceful handling of non-existing lora_path in inference request #7447

Closed
@lifuhuang

Description

@lifuhuang

Checklist

Motivation

Creating an issue to track this TODO for myself (or anyone else who wants to help):

Currently when users call SGLang with a non-existing lora_path, SGLang server/engine would crash due to failed assertions in prepare_lora_batch. This is unideal as it imposes unnecessary burden for server owner to validate request params before they are passed to the SGLang backend.

Ideally, SGLang should have gracefully handled the exception and respond 4xx errors without crashing the server.

Related resources

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions