Skip to content

Include metrics for configured limit overrides#1089

Merged
joe-elliott merged 8 commits intografana:mainfrom
zalegrala:overridesLimitsMetrics
Nov 18, 2021
Merged

Include metrics for configured limit overrides#1089
joe-elliott merged 8 commits intografana:mainfrom
zalegrala:overridesLimitsMetrics

Conversation

@zalegrala
Copy link
Copy Markdown
Contributor

@zalegrala zalegrala commented Oct 28, 2021

What this PR does:

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@zalegrala zalegrala force-pushed the overridesLimitsMetrics branch 3 times, most recently from e88e046 to 9cf9b2a Compare November 8, 2021 16:09
@zalegrala zalegrala marked this pull request as ready for review November 8, 2021 17:21
Comment thread CHANGELOG.md Outdated
Comment thread modules/overrides/overrides.go Outdated
Comment thread modules/overrides/overrides.go Outdated
@zalegrala zalegrala force-pushed the overridesLimitsMetrics branch from dd82ad8 to f210535 Compare November 9, 2021 17:24
@zalegrala
Copy link
Copy Markdown
Contributor Author

I've added a metric for default limits as well.

@zalegrala zalegrala force-pushed the overridesLimitsMetrics branch from 91b548a to 5c1d86f Compare November 10, 2021 16:46
@zalegrala
Copy link
Copy Markdown
Contributor Author

I've made an update to implement the Overrides module as a Collector. We could adjust to have the exporter be its own module /package like is done in the cortex code, but I think that might be a little more than we need for this. What do you think?

@zalegrala zalegrala force-pushed the overridesLimitsMetrics branch 2 times, most recently from 58e0c72 to 5a33173 Compare November 12, 2021 17:50
Comment thread modules/overrides/overrides.go Outdated
var manager *runtimeconfig.Manager
subservices := []services.Service(nil)

metricDefaultsLimits.WithLabelValues("max_local_traces_per_user").Set(float64(defaults.MaxLocalTracesPerUser))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make max_local_traces_per_user, etc. constants

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is addressed in the last commit.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still seeing individual static strings.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sorry, I thought you meant prometheus.MustNewConstMetric to make them constants. You mean make the metric name string a constant to be referenced by both the defaults and the overrides, right?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup. I just meant to make a constant string that they both use. I have no idea what MustNewConstMetric is but let's just follow whatever Cortex does here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep that's what they are doing. This promehteus.Collector pattern is new to me, but the const bit is document here: https://prometheus.io/docs/instrumenting/writing_exporters/#collectors

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Cortex the exporter is also a module, but as it is now we aren't doing that. If we want to do that, I think we just need to make a limits/exporter package and glue it up, but not sure how far to go with modules. It would be our first "exporter" module, but it could be a pattern if we want to structure the exporters as modules. Seems to me that they belong with the module they support, rather than a module in themselves. What do you think?

@zalegrala zalegrala force-pushed the overridesLimitsMetrics branch from 5faecd7 to 822c19c Compare November 16, 2021 21:53
@zalegrala zalegrala force-pushed the overridesLimitsMetrics branch from 822c19c to ef029c2 Compare November 17, 2021 21:58
Copy link
Copy Markdown
Collaborator

@joe-elliott joe-elliott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice! i think we're there!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants