[Feature] grafana dashboard should work out of the box

[ziliangpeng]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

we should make prometheus metrics and grafana dashboard ready to use out of the box to avoid requiring any user manual set up. this will streamline the accessibility of observability and reduce efforts required to support users.

Related resources

task check list

• in the metrics doc, remind user to run sglang server with --enable-metrics

No response

[PopSoda2002]

I will take that together!

[blacker521]

I will take that together

[ziliangpeng]

What

From the example provided to display production metrics, a few things are missing on the grafana UI that require manual steps:

• missing data source to connect to the prometheus instance
• missing dashboards that requires manual importing

There might be more missing pieces (as we work along this issue) to have a completely manual-free grafana example.
The acceptance of this issue is to run docker compose -f compose.yaml -p monitoring up and production metrics is viewable on localhost:3000/{path to dashboard} showing sglang metrics.

How

We most likely need to create our own Dockerfil based off of the grafana base image, and manipulate the configuration in grafana image to the right state.

automating data source

this may be achieved by just putting a file in /etc/grafana/provisioning/datasources/.
an example (only example) of the config file would be something like

apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://localhost:9090
    jsonData:
      httpMethod: POST
      prometheusType: Prometheus
      prometheusVersion: 2.44.0
      exemplarTraceIdDestinations:
        - name: traceID
          datasourceUid: my_jaeger_uid
    version: 1
    editable: false

the exactly location and format of the file can be explored from official documents, or just find it in the container after manually creating a data source.

automating dashboard

similarly we can create dashboard by putting the right file in the right place.
TODO: will add more details when we're at this step.

[blacker521]

just like this one?
https://github.com/sgl-project/sglang/blob/main/docs/references/production_metrics.md

[zhaochenyang20]

@blacker521 will make it easier

[blacker521]

@zhaochenyang20 return this information in JSON? how can I contribute it?

[blacker521]

Is this feature going anywhere? Can I be a part of it?

[zhaochenyang20]

@blacker521 @PopSoda2002 and @ziliangpeng is on this part. We do not need more people right now. Thanks!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.

[PopSoda2002]

Fixed by PR: #4718