[V1][eagle3] Support eagle3 proposer for v1

[yuancaoyaoHW]

What this PR does / why we need it?

This PR implements the Eagle Pososer feature for vLLM v1, which enables more efficient speculative decoding by using a draft model to predict potential future tokens.

• The implementation includes the core Eagle algorithm integration with vLLM's existing architecture, allowing for faster inference while maintaining output quality.
• This is needed to significantly improve the generation speed of large language models without compromising on the quality of generated text.

Does this PR introduce any user-facing change?

Yes, this PR introduces a new speculative decoding mode that can be enabled via configuration.

• Users can now choose to use Eagle Pososer by setting appropriate flags in the inference configuration.
• The API remains backward compatible, with the new functionality being opt-in.

How was this patch tested?

CI passed with new unit tests added for the Eagle Pososer functionality.

• Benchmark tests were conducted comparing generation speed and quality with and without Eagle Pososer.
• Integration tests were performed with various model architectures to ensure compatibility.
• Manual testing was done using different prompt scenarios to verify output quality remains consistent.
• we test accept rate on one Ascend 910B npu, The acceptance rate results are basically consistent with those shown here: [V1][Spec Decode] EAGLE-3 Support vllm#16937
• Currently, we support scenarios where num_spec_tokens <= 2. When num_spec_tokens > 2, issues such as insufficient GPU memory and operator computation errors may occur. We will address this in subsequent updates.
• We will add support for Eagle v1 in future updates.

Acceptance Test Script

SCRIPT="/offline/eagle.py"
DATASET="ShareGpt"
MODEL=Meta-Llama-3.1-8B-Instruct
DRAFT=EAGLE3-LLaMA3.1-Instruct-8B

CUDA_VISIBLE_DEVICES="0" VLLM_USE_V1=1 $PYTHON $SCRIPT \
    --dataset $DATASET \
    --num_spec_tokens 2 \
    --max_num_seqs 1 \
    --model_dir $MODEL \
    --eagle_dir $DRAFT \
    --tp 1 \
    --num_prompts 80

Acceptance Test Results

██████████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [21:22<00:00, 16.03s/it, est. speed input: 4.72 toks/s, output: 13.56 toks/s]
-------------------------------------------------------------------------------------
mean acceptance length: 1.63
-------------------------------------------------------------------------------------
total_counts: 8062
acceptance at token 0: 1.00 (8062 times)
acceptance at token 1: 0.70 (5612 times)
acceptance at token 2: 0.47 (3765 times)

Closes: #1004

[Yikun]

Thansk for contributions, you can git commit --amend -s amend the singned off by info and git push origin vllm-eagle3 -f

[Yikun]

Thanks for your contributions.

1. Please add How was this patch tested? in commit msh including vllm server or script to help others reproduce locally
2. Please change code note to english
3. Add e2e test: https://github.com/vllm-project/vllm-ascend/tree/main/tests/long_term/spec_decode/e2e , you can reference: https://github.com/vllm-project/vllm/blob/main/tests/spec_decode/e2e/test_eagle_correctness.py
4. Fix mypy and lint CI, you can refernece: https://vllm-ascend.readthedocs.io/en/latest/developer_guide/contributing.html

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[mengwei805]

please revise file header.
I think this file should be placed in the woker directory instead of creating a new spec_decode directory. @wangxiyuan

[mengwei805]

make sure this UT could run in v1 engine when CI running.
The UT in the current long_term directory is run by the v0 engine by default.
You can refer to test_v1_spec_decode.py or test_v1_mtp_correctness.py.
You may also have to modify .github/workflows/vllm_ascend_test_long_term.yaml

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[wangxiyuan]

@jianzs @ganyi1996ppo @Yikun please help review.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[Yikun]

I'm OK with this PR, I trend to merge this today.

[MengqingCao]

Please add egaer_mode=True to https://github.com/vllm-project/vllm-ascend/blob/main/tests/e2e/long_term/test_deepseek_v2_lite_tp2_accuracy.py#L41