fix: target_info skipped when resource attributes have empty values#6774
Merged
carles-grafana merged 1 commit intografana:mainfrom Mar 25, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a regression where traces_target_info could be skipped when resource attributes include empty-string values, due to a mismatch between raw resource attribute counts and the final label set produced by the Prometheus label builder.
Changes:
- Replace the
target_infoemission guard from a label-count comparison to explicitjob/instancepresence checks. - Add a regression test that exercises empty-valued resource attributes while still expecting
traces_target_infoto be emitted.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| modules/generator/processor/spanmetrics/spanmetrics.go | Updates the target_info guard logic to avoid relying on label counts that can be affected by empty-valued labels being dropped. |
| modules/generator/processor/spanmetrics/spanmetrics_test.go | Adds a regression test for empty-valued resource attributes to ensure traces_target_info is still generated. |
carles-grafana
force-pushed
the
fix/target-info-empty-resource-attr
branch
from
c9efbf8 to
3f65492
Compare
carles-grafana
requested review from
electron0zero,
ie-pham,
javiermolinar,
mattdurham,
oleg-kozlyuk-grafana,
ruslan-mikhailov,
stoewer,
yvrhdn and
zalegrala
as code owners
| if jobName != "" { | ||
| identifyingLabels++ | ||
| } | ||
| if instanceID != "" && p.Cfg.EnableInstanceLabel { |
Contributor
There was a problem hiding this comment.
This two lines are duplicated above
| // resourceAttributesCount, because the Prometheus label builder drops empty-valued | ||
| // labels (Set("x","") calls Del("x")), which would cause a count mismatch. | ||
| identifyingLabels := 0 | ||
| if jobName != "" { |
Contributor
There was a problem hiding this comment.
This line is already calculated above, we can just coung the indentifiying labels in the already existing conditions
| // only register target info if at least (job or instance) AND one other attribute are present | ||
| // TODO - We can move this check to the top | ||
| if resourceAttributesCount > 0 && targetInfoRegistryLabelValues.Len() > resourceAttributesCount { | ||
| // Only register target_info if it has at least (job or instance) AND one other |
Contributor
There was a problem hiding this comment.
This comment is referencig the deleted variable resourceAttributesCount
carles-grafana
force-pushed
the
fix/target-info-empty-resource-attr
branch
from
3f65492 to
5c2d82d
Compare
The Prometheus labels.Builder deletes labels with empty values, but resourceAttributesCount was computed before building. This mismatch caused the guard check to fail, silently dropping target_info for any service with empty resource attributes (e.g. host.id="" from the OTel NodeJS SDK). Count identifying labels (job/instance) and compare against the built label set length to ensure at least one real resource attribute exists.
carles-grafana
force-pushed
the
fix/target-info-empty-resource-attr
branch
from
5c2d82d to
3b61bf3
Compare
Comment on lines
190
to
200
| // add job label only if job is not blank and target_info is enabled | ||
| identifyingLabels := 0 | ||
| if jobName != "" && p.Cfg.EnableTargetInfo { | ||
| builder.Add(gen.DimJob, jobName) | ||
| identifyingLabels++ | ||
| } | ||
| // add instance label only if instance is not blank and enabled and target_info is enabled | ||
| // add instance label only if instance is not blank and enabled and target_info is enabled | ||
| if instanceID != "" && p.Cfg.EnableTargetInfo && p.Cfg.EnableInstanceLabel { | ||
| builder.Add(gen.DimInstance, instanceID) | ||
| identifyingLabels++ | ||
| } |
There was a problem hiding this comment.
identifyingLabels is incremented when adding job/instance to the span metrics label builder, but it’s later used to decide whether to emit target_info. This couples two separate builders and could become incorrect if the conditions for adding job/instance diverge in the future. Consider computing this count (or two booleans for job/instance presence) alongside the targetInfoBuilder.Add(...) calls inside the EnableTargetInfo block, and use that for the guard.
carles-grafana
pushed a commit
to carles-grafana/tempo
that referenced
this pull request
Mar 27, 2026
Add 3 entries that were missing from the unreleased changelog: - [BUGFIX] grafana#6774: target_info skipped when resource attributes have empty values - [BUGFIX] grafana#6653: drain old series on metric replacement to prevent limiter leak - [ENHANCEMENT] grafana#6371: make trace_too_large log line an insight
3 tasks
carles-grafana
added a commit
that referenced
this pull request
Mar 27, 2026
Add 3 entries that were missing from the unreleased changelog: - [BUGFIX] #6774: target_info skipped when resource attributes have empty values - [BUGFIX] #6653: drain old series on metric replacement to prevent limiter leak - [ENHANCEMENT] #6371: make trace_too_large log line an insight Co-authored-by: Carles Grafana <carles@grafana.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Prometheus labels.Builder deletes labels with empty values, but resourceAttributesCount was computed before building. This mismatch caused the guard check to fail, silently dropping target_info for any service with empty resource attributes (e.g. host.id="" from the OTel NodeJS SDK).
Replace the count comparison with direct checks for job/instance.
What this PR does:
Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]