Skip to content

Add querier metrics for requests executed#3524

Merged
electron0zero merged 4 commits intografana:mainfrom
electron0zero:add_querier_metrics
Mar 27, 2024
Merged

Add querier metrics for requests executed#3524
electron0zero merged 4 commits intografana:mainfrom
electron0zero:add_querier_metrics

Conversation

@electron0zero
Copy link
Copy Markdown
Member

@electron0zero electron0zero commented Mar 27, 2024

What this PR does:

Adds two metrics to provide observability into read path and query processing

  1. tempo_querier_worker_request_executed_total is a counter, measuring the requests executed by the querier
  2. log the concurrency of a querier

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@electron0zero electron0zero changed the title Add metrics for concurrency and querier requests executed Add querier metrics for concurrency and requests executed Mar 27, 2024
Comment thread modules/querier/worker/worker.go Outdated
MaxBackoff: 1 * time.Second,
}

var metricWorkerRequests = promauto.NewCounter(prometheus.CounterOpts{
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't mind this but do you expect it to be significantly different than the querier request rates?

Copy link
Copy Markdown
Member Author

@electron0zero electron0zero Mar 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, I think request rate is for batches, and this is at job level (single request). we can rate the counter to capture how much work each querier is doing...

@electron0zero electron0zero changed the title Add querier metrics for concurrency and requests executed Add querier metrics for requests executed Mar 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants