Feat/sync from upstream #18

jimoosciuc · 2025-04-17T03:55:28Z

Motivation

Modifications

Checklist

Format your code according to the Code Formatting with Pre-Commit.
Add unit tests as outlined in the Running Unit Tests.
Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

…-project#4882)

…#4530)

Co-authored-by: Sleepcoo <[email protected]>

…gl-project#4932)

Co-authored-by: ch-wan <[email protected]>

Co-authored-by: Lianmin Zheng <[email protected]>

…ions (sgl-project#5431)

Co-authored-by: hebiao064 <[email protected]>

Co-authored-by: Stefan He <[email protected]>

Co-authored-by: Yuhao Yang <[email protected]>

…gl-project#5113)

…sgl-project#5345) Co-authored-by: Chayenne <[email protected]>

Co-authored-by: yhyang201 <[email protected]> Co-authored-by: Liangsheng Yin <[email protected]> Co-authored-by: Chayenne <[email protected]> Co-authored-by: Yineng Zhang <[email protected]>

zhyncs and others added 30 commits March 28, 2025 17:23

fix sgl-kernel cu118 build (sgl-project#4872)

ec3ee02

[Feature] Support FA3 backend for MLA (sgl-project#4831)

20c90be

upgrade sgl-kernel 0.0.5.post4 (sgl-project#4873)

d8a136a

update torch compile doc (sgl-project#4874)

aa08aea

bump v0.4.4.post3 (sgl-project#4878)

19e96e5

Fix BadRequestError wrong arguments and remove openai dependency (sgl…

b1cfb4e

…-project#4882)

Improve stack trace of retry errors (sgl-project#4845)

8690c40

Tiny fix doc error (sgl-project#4795)

736502d

[Docs] Update DeepGEMM at README.md (sgl-project#4886)

05625b9

Update CODEOWNERS (sgl-project#4889)

400ad66

Delete test_deep_gemm.py (sgl-project#4891)

ddf8981

Add deepseek style fused moe group gate selection kernel (sgl-project…

45dcfc2

…#4530)

quick fix: add default for new kernel (sgl-project#4898)

8e7b315

remove setup for sgl-kernel (sgl-project#4899)

54b9a2d

[Misc] Clean m.def and add Development Tips (sgl-project#4890)

0d7fe86

fix allreduce test (sgl-project#4909)

5ec5eaf

Support page size > 1 + eagle (sgl-project#4908)

b26bc86

Fix retract for page size > 1 (sgl-project#4914)

4ede677

[Feature] use pytest for sgl-kernel (sgl-project#4896)

9fccda3

fix bmm fp8 (sgl-project#4926)

195a09f

Fix the timeout for unit-test-2-gpu in pr-test.yml (sgl-project#4927)

f842853

Fix 2-gpu CI test and suppress some warnings (sgl-project#4930)

9adf178

[feat] add fa3 in sgl-kernel (sgl-project#4902)

37c66ec

Co-authored-by: Sleepcoo <[email protected]>

Fix sglang frontend's incorrect dependency on torch (sgl-project#4931)

032f8fa

[Fix] avoid stream sync and torch compile in prefill for fa3 backend (s…

e62d60f

…gl-project#4932)

cleanup sgl-kernel (sgl-project#4933)

4814eca

[Fix] Improve Lora tests and reduce CI runtime (sgl-project#4925)

42873ea

Fix DeepSeek bug causing 2.2% MMLU drop when TP!=DP (sgl-project#4883)

a303325

Co-authored-by: ch-wan <[email protected]>

[Fix] Add torch compile for torch.clamp back (sgl-project#4936)

4a63bc3

Fix oom error for large page size (sgl-project#4913)

a169b9f

Co-authored-by: Lianmin Zheng <[email protected]>

merrymercy and others added 30 commits April 15, 2025 07:07

[minor] cleanup cmakelists.txt (sgl-project#5420)

838fa0f

bugfix: fix merge_state_v2 cuda graph (sgl-project#5419)

12ef7e3

chore: bump sgl-kernel v0.0.9.post1 (sgl-project#5430)

6f509d5

fix: solve release issue (sgl-project#5434)

88defc4

BLackwell cutlass mla: Add check for bad page size/block num combinat…

e8f62b2

…ions (sgl-project#5431)

feat: update model_specific_adjustment (sgl-project#5344)

fa909dc

Co-authored-by: hebiao064 <[email protected]>

chore: upgrade sgl-kernel 0.0.9.post1 (sgl-project#5436)

8ec0bb7

Fix ignore_eos parameter when loading a chat template (sgl-project#5264)

27a009b

add attention backend supporting matrix in the doc (sgl-project#5211)

3efc8e2

Co-authored-by: Stefan He <[email protected]>

Support BNB quantization for llama/mllama (sgl-project#5038)

bc24205

Co-authored-by: Yuhao Yang <[email protected]>

[Docs] Update start/install.md (sgl-project#5398)

b64b88e

[Minor] Move torch.compile patch to a better place (sgl-project#5397)

0769b14

[Bug fix] need record start time in pd mode (sgl-project#5425)

dd83e7e

Support MHA with chunked prefix cache for DeepSeek chunked prefill (s…

a42736b

…gl-project#5113)

chore: bump v0.4.5.post1 (sgl-project#5445)

5b5c723

Fix several minor issues in PD disaggregation (sgl-project#5444)

6aca583

[doc] Update benchmark_and_profiling.md (sgl-project#5449)

06a1656

Update cutlass dependency. (sgl-project#5447)

85ec044

add multi-lora feature in README.md (sgl-project#5463)

d7bc19a

Clean up imports (sgl-project#5467)

177320a

[verl] Modify the update_weights func to align with verl's resharding (…

90faf90

…sgl-project#5345) Co-authored-by: Chayenne <[email protected]>

[Model Support] unsloth/Phi-4-mini bnb model (sgl-project#4982)

8f783c1

Co-authored-by: yhyang201 <[email protected]> Co-authored-by: Liangsheng Yin <[email protected]> Co-authored-by: Chayenne <[email protected]> Co-authored-by: Yineng Zhang <[email protected]>

feat: support pd-disaggregation

1a95371

feat: scheduler separately get batch kv cache

8f44024

feat: support pd disaggregation

50fbff9

fix: debug

b1b626b

fix bug

05698fc

fix bug

6436509

fix: undefine req params

a163003

support startup

1fc3357

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feat/sync from upstream #18

Feat/sync from upstream #18

Uh oh!

jimoosciuc commented Apr 17, 2025

Uh oh!

Uh oh!

Feat/sync from upstream #18

Are you sure you want to change the base?

Feat/sync from upstream #18

Uh oh!

Conversation

jimoosciuc commented Apr 17, 2025

Motivation

Modifications

Checklist

Uh oh!

Uh oh!