[RL] Remove the w13 weight_scale and input_scale for UnquantizedEPMoE…

[zhuzilin]

…Method

Motivation

When doing RL training, we may release all the parameters with /release_memory_occupation to free the memory occupied by the inference engine, which will also released all the input_scales and weight_scales.

Modifications

The origin w13_weight_scale in UnquantizedEPMoEMethod does not support reloading (as the shape should be (num_experts_per_partition, 2)). And I found that for the UnquantizedEPMoEMethod, we don't need to instantiate w13_weight_scale and w13_input_scale, so removing them could be a better solution than allocating twice the origin memory.

And note that we do need to reload the w2_input_scale, because if we set that to None, it will be initialized to torch.ones during EpMoE.forward_normal. So I need to change the condition in _load_fp8_scale to allow loading w2_input_scale from a random value to 1.

Thank you for your time on reviewing this PR :)

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[fzyzcjy]

LGTM, indeed making it none has one extra benefit: when doing EPLB shuffling, we no longer need to send these params between ranks