Skip to content

feat: Added flag to disable KIP-714 Kafka client metrics#6046

Merged
oleg-kozlyuk-grafana merged 8 commits intografana:mainfrom
oleg-kozlyuk-grafana:kafka-error-fix
Dec 11, 2025
Merged

feat: Added flag to disable KIP-714 Kafka client metrics#6046
oleg-kozlyuk-grafana merged 8 commits intografana:mainfrom
oleg-kozlyuk-grafana:kafka-error-fix

Conversation

@oleg-kozlyuk-grafana
Copy link
Copy Markdown
Contributor

@oleg-kozlyuk-grafana oleg-kozlyuk-grafana commented Dec 9, 2025

What this PR does:

Disabled Kafka API client metrics as currently unused

Which issue(s) this PR fixes:

https://github.com/grafana/tempo-squad/issues/992

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@oleg-kozlyuk-grafana oleg-kozlyuk-grafana changed the title fix: Disabled client metrics causing noise in Kafka client fix: Disabled Kafka client metrics causing log noise Dec 9, 2025
Comment thread pkg/ingest/writer_client.go Outdated
Copy link
Copy Markdown
Contributor

@javiermolinar javiermolinar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to make this opt-out instead of opt-in? These metrics are very helpful for debugging producer/consumer issues (latency, retries, lag, broker connectivity), and disabling them by default makes Kafka significantly harder to operate. While we’ve replaced some of them with our own metrics, we don’t yet cover the full surface area that kgo provides. It might be better to keep them enabled by default and let users explicitly disable them if needed (e.g., for overhead or cardinality reasons).

@oleg-kozlyuk-grafana
Copy link
Copy Markdown
Contributor Author

oleg-kozlyuk-grafana commented Dec 10, 2025

Flipped the meaning of the flag so it's opt-out

@oleg-kozlyuk-grafana oleg-kozlyuk-grafana changed the title fix: Disabled Kafka client metrics causing log noise feat: Added flag to disable KIP-714 Kafka client metrics Dec 11, 2025
@oleg-kozlyuk-grafana oleg-kozlyuk-grafana merged commit 03ea9f3 into grafana:main Dec 11, 2025
22 checks passed
@oleg-kozlyuk-grafana oleg-kozlyuk-grafana deleted the kafka-error-fix branch December 11, 2025 12:10
joe-elliott pushed a commit to joe-elliott/tempo that referenced this pull request Dec 12, 2025
oleg-kozlyuk-grafana added a commit to oleg-kozlyuk-grafana/tempo that referenced this pull request Dec 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants