ITL metrics

[marquis-wang]

During performance analysis with sglang, after using the --enable-metrics option, no ITL-related metrics were found in the metrics output by the framework. How can I do it？

[Fridge003]

Hi @marquis-wang, --enable-metrics option is for prometheus.

To see ITL-related metrics, please use benchmark scripts as described in benchmark section of this doc, and the ITL related metrics will be printed.

[marquis-wang]

Hi @marquis-wang, --enable-metrics option is for prometheus.

To see ITL-related metrics, please use benchmark scripts as described in benchmark section of this doc, and the ITL related metrics will be printed.

Thank you for your response. We would like to use Prometheus to monitor and alert on ITL metrics for our online inference service in real-time. However, we couldn't find the relevant metrics in the metrics provided by sglang. Do you have any plans to provide these metrics in the future?

[Fridge003]

@marquis-wang OK, I will ask related developers of prometheus.

[PopSoda2002]

I will help to look at this

[zhaochenyang20]

@PopSoda2002 Great! How is the progress of grafana?

[PopSoda2002]

Still work in progress:(

[zhaochenyang20]

@PopSoda2002 Great. Thanks so muhc!

[marquis-wang]

"Is there any progress on this matter?"

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.

[PopSoda2002]

This problem is solved in #4718 you can take a look @marquis-wang, sorry for late reply