[rebase]rebase grpo-latest #6354

flybird11111 · 2025-06-23T06:37:17Z

📌 Checklist before creating the PR

I have created an issue for this PR for traceability
The title follows the standard format: [doc/gemini/tensor/...]: A concise description
I have added relevant tags if possible for us to better distinguish different PRs
I have installed pre-commit: pip install pre-commit && pre-commit install

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

I have linked my PR to an issue (instruction)
My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
I have performed a self-review of my code
I have added thorough tests.
I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

🌝 Yes, I do.
🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

…into grpo-latest-dev

Co-authored-by: Tong Li <[email protected]>

for more information, see https://pre-commit.ci

[feat] Move prompt-level-filtering to buffer side

for more information, see https://pre-commit.ci

[feat] Support Code Generation RFT, Move Reward Calculation to Producer

Co-authored-by: Tong Li <[email protected]>

…achine [feat] Manually schedule resources and support auto master address assigning

…e_per_episode [fix] fix num update per episode

[Feat] optimize pp log_softmax memory usage

Co-authored-by: Tong Li <[email protected]>

* [fix] support npu * [feat] multinode 14B * [feat] enlarge seqlen * [fix] * [fix] ready to updated * [fix] ready to merge grpo-latest * [fix] rm comments * [feat] support msprof-analyze, add analsys result * [feat] support ColossalaiRL on Ascend * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [feat] rm comments in qwen modeling * [Doc] Drafted README.md * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [feat] fix ascend readme format * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [fix] fix readme * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [fix] fix readme * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [fix] fix Readme, rm irrelevant testcase * [fix] fix some adapt modification * [fix] rm comments in modeling qwen * [fix] rm comm, test and debug print * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: YeAnbang <[email protected]>

* update readme * [fix] add vllm & vllm-ascend installation --------- Co-authored-by: Tong Li <[email protected]> Co-authored-by: duanjunwen <[email protected]>

* [fix] fix colossalai ascend requirments * [fix] fix colossalai chat requirements * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [fix] fix requirments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* move prompt-level-filtering to buffer side * move prompt-level-filtering to buffer side * remove redundant code and fix bugs * fix metric calculation * fix missing tags parameter * address conversation * add overlength sample count (hpcaitech#6332) Co-authored-by: Tong Li <[email protected]> * address conversation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix typ and parameter description * [feat] Update requriments and set return logits False --------- Co-authored-by: YeAnbang <[email protected]> Co-authored-by: Tong Li <[email protected]> Co-authored-by: Tong Li <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

for more information, see https://pre-commit.ci

YeAnbang and others added 30 commits May 15, 2025 18:16

move prompt-level-filtering to buffer side

55eee12

move prompt-level-filtering to buffer side

a528921

remove redundant code and fix bugs

11a5854

merge reward and eval

116621d

fix metric calculation

3766338

merge grpo-latest

88e3b09

fix missing tags parameter

78a06f5

address conversation

4c36568

Merge branch 'grpo-latest' of https://github.com/hpcaitech/ColossalAI …

58f8c9b

…into grpo-latest-dev

add overlength sample count (hpcaitech#6332)

c8b368c

Co-authored-by: Tong Li <[email protected]>

address conversation

ee939d9

merge grpo-latest

7b921ac

[pre-commit.ci] auto fixes from pre-commit.com hooks

0d00811

for more information, see https://pre-commit.ci

fix typ and parameter description

96faf54

Merge pull request hpcaitech#6312 from hpcaitech/grpo-latest-dev

ceb7065

[feat] Move prompt-level-filtering to buffer side

support code generation tasks

dc3033e

fix bug, tested

3bed6ae

remove debug code

d0e12c5

[pre-commit.ci] auto fixes from pre-commit.com hooks

9ca920c

for more information, see https://pre-commit.ci

Merge pull request hpcaitech#6341 from hpcaitech/grpo-code

c308b42

[feat] Support Code Generation RFT, Move Reward Calculation to Producer

move out evaluation func (hpcaitech#6343)

bb6f5d9

Co-authored-by: Tong Li <[email protected]>

Manually schedule resources and support auto master address assigning

21d517d

modify readme

2559924

update readme

dc29c74

add ray timeout handling instruction

1330b57

fix pp memory issue (hpcaitech#6344)

8992def

Co-authored-by: Tong Li <[email protected]>

Update README.md

2f02a28

Merge pull request hpcaitech#6316 from hpcaitech/grpo-support-multi-m…

ac06935

…achine [feat] Manually schedule resources and support auto master address assigning

fix num_update_per_episode

51b7abe

Merge pull request hpcaitech#6347 from hpcaitech/hotfix/fix_num_updat…

0e69b98

…e_per_episode [fix] fix num update per episode

YeAnbang and others added 10 commits June 13, 2025 18:21

optimize pp log_softmax OOM

30a6859

implement memory efficient logprob

e3d56cb

fix small bug

6b06430

Merge pull request hpcaitech#6348 from hpcaitech/grpo_optimization

dd49444

[Feat] optimize pp log_softmax memory usage

add dp rank for multi-dp (hpcaitech#6351)

8880b83

Co-authored-by: Tong Li <[email protected]>

[Ascend] Update README (hpcaitech#6331)

2dd59c0

* update readme * [fix] add vllm & vllm-ascend installation --------- Co-authored-by: Tong Li <[email protected]> Co-authored-by: duanjunwen <[email protected]>

update to conform to json format

2783dde

flybird11111 requested a review from a team as a code owner June 23, 2025 06:37

flybird11111 and others added 4 commits June 23, 2025 14:39

Merge branch 'grpo-latest-ascend' into merge

9d7544a

[pre-commit.ci] auto fixes from pre-commit.com hooks

7891cc7

for more information, see https://pre-commit.ci

fix pp

bd8b571

[pre-commit.ci] auto fixes from pre-commit.com hooks

a14bd0b

for more information, see https://pre-commit.ci

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[rebase]rebase grpo-latest #6354

[rebase]rebase grpo-latest #6354

Uh oh!

flybird11111 commented Jun 23, 2025

Uh oh!

Uh oh!

[rebase]rebase grpo-latest #6354

Are you sure you want to change the base?

[rebase]rebase grpo-latest #6354

Uh oh!

Conversation

flybird11111 commented Jun 23, 2025

📌 Checklist before creating the PR

🚨 Issue number

📝 What does this PR do?

💥 Checklist before requesting a review

⭐️ Do you enjoy contributing to Colossal-AI?

Uh oh!

Uh oh!