Skip to content

[ROCm][CI]: add VLM PR CI for parity with NVIDIA #6130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

functionstackx
Copy link

@functionstackx functionstackx commented May 8, 2025

Motivation

get ROCm CI closer to NVIDIA CI coverage

added VLM ROCm CI similar to #6038

Checklist

@JustinTong0323
Copy link
Collaborator

Thank you for your contribution! Kindly refer to

if os.getenv("SGLANG_AMD_CI") == "1":
self.assertGreater(res["output_throughput"], 2000)
# TODO: not set yet, need AMD machine
else:
self.assertGreater(res["output_throughput"], 2500)

if os.getenv("SGLANG_AMD_CI") == "1":
self.assertLess(res["median_ttft_ms"], 150)
# TODO: not set yet, need AMD machine
else:
self.assertLess(res["median_ttft_ms"], 90)

Adjust the throughput and latency in accordance with your local test results.

@JustinTong0323 JustinTong0323 added the VLM vision language model label May 8, 2025
@JustinTong0323 JustinTong0323 changed the title [ROCm]: add VLM PR CI for parity with NVIDIA [ROCm][CI]: add VLM PR CI for parity with NVIDIA May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
VLM vision language model
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants