Skip to content

Metrics Generator Performance#4408

Merged
ie-pham merged 1 commit intografana:mainfrom
ie-pham:mgImprove
Dec 12, 2024
Merged

Metrics Generator Performance#4408
ie-pham merged 1 commit intografana:mainfrom
ie-pham:mgImprove

Conversation

@ie-pham
Copy link
Copy Markdown
Contributor

@ie-pham ie-pham commented Dec 2, 2024

What this PR does: Generator improvements based on @mdisibio 's notes for things relating to target_info.

cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
          │ before-mg.out │            after-mg.out             │
          │    sec/op     │   sec/op     vs base                │
PushSpans     3.006m ± 3%   1.308m ± 1%  -56.49% (p=0.000 n=10)

          │ before-mg.out │            after-mg.out            │
          │  heap_in_use  │ heap_in_use  vs base               │
PushSpans     15.73M ± 0%   15.79M ± 0%  +0.36% (p=0.000 n=10)

          │ before-mg.out │             after-mg.out             │
          │     B/op      │     B/op      vs base                │
PushSpans    705.9Ki ± 0%   568.6Ki ± 0%  -19.44% (p=0.000 n=10)

          │ before-mg.out │            after-mg.out             │
          │   allocs/op   │  allocs/op   vs base                │
PushSpans    18.582k ± 0%   9.044k ± 0%  -51.33% (p=0.000 n=10)

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Copy link
Copy Markdown
Contributor

@mdisibio mdisibio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice improvements 👍 I think it would be worth running the benchmarks with TargetInfoEnabled, and fixing the buffer reuse in that mode. Left a few surface-level comments too.

}

return keys, values
return keys[:count], values[:count]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the slices start with 0 length and are appended to, don't think the reslice or count tracking are needed.

func SanitizeLabelNameWithCollisions(name string, dimensions []string) string {
sanitized := strutil.SanitizeLabelName(name)

if IsIntrinsicDimension(sanitized, dimensions) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can probably replace all of IsIntrinsicDimension with slices.Contains

resourceValues = resourceValues[:0]
if p.Cfg.EnableTargetInfo {
resourceLabels, resourceValues = processor_util.GetTargetInfoAttributesValues(rs.Resource.Attributes, p.Cfg.TargetInfoExcludedDimensions)
resourceLabels, resourceValues = processor_util.GetTargetInfoAttributesValues(rs.Resource.Attributes, p.Cfg.TargetInfoExcludedDimensions, intrinsicLabels)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When TargetInfo is enabled, this will still realloc the labels and values on every loop iteration. It should be possible to pass them in to GetTargetInfoAttributesValues to reuse them, which would be a nice improvement in that mode.

@ie-pham
Copy link
Copy Markdown
Contributor Author

ie-pham commented Dec 3, 2024

latest benchmarks

target info disabled

cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
          │ before-tgdisabled-mg.out │      after-tgdisabled-mg.out       │
          │          sec/op          │   sec/op     vs base               │
PushSpans                2.102m ± 7%   1.969m ± 4%  -6.35% (p=0.000 n=10)

          │ before-tgdisabled-mg.out │      after-tgdisabled-mg.out       │
          │       heap_in_use        │ heap_in_use  vs base               │
PushSpans                15.86M ± 0%   16.00M ± 0%  +0.90% (p=0.000 n=10)

          │ before-tgdisabled-mg.out │    after-tgdisabled-mg.out     │
          │           B/op           │     B/op      vs base          │
PushSpans               540.2Ki ± 0%   540.2Ki ± 0%  ~ (p=0.063 n=10)

          │ before-tgdisabled-mg.out │    after-tgdisabled-mg.out    │
          │        allocs/op         │  allocs/op   vs base          │
PushSpans                15.36k ± 0%   15.36k ± 0%  ~ (p=0.839 n=10)

target info enabled

cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
          │ before-tgenabled-mg.out │       after-tgenabled-mg.out        │
          │         sec/op          │   sec/op     vs base                │
PushSpans              7.455m ± 10%   3.263m ± 5%  -56.23% (p=0.000 n=10)

          │ before-tgenabled-mg.out │       after-tgenabled-mg.out       │
          │       heap_in_use       │ heap_in_use  vs base               │
PushSpans               16.36M ± 0%   16.15M ± 0%  -1.23% (p=0.000 n=10)

          │ before-tgenabled-mg.out │        after-tgenabled-mg.out        │
          │          B/op           │     B/op      vs base                │
PushSpans              1.724Mi ± 0%   1.386Mi ± 0%  -19.58% (p=0.000 n=10)

          │ before-tgenabled-mg.out │       after-tgenabled-mg.out        │
          │        allocs/op        │  allocs/op   vs base                │
PushSpans               46.44k ± 0%   22.60k ± 0%  -51.33% (p=0.000 n=10)

@ie-pham ie-pham enabled auto-merge (squash) December 12, 2024 14:52
@ie-pham ie-pham disabled auto-merge December 12, 2024 14:52
@ie-pham ie-pham enabled auto-merge (squash) December 12, 2024 14:55
@ie-pham ie-pham merged commit b54844c into grafana:main Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants