Skip to content

Add cassandra tag filter#442

Merged
yurishkuro merged 7 commits intomasterfrom
add_tag_filter
Oct 12, 2017
Merged

Add cassandra tag filter#442
yurishkuro merged 7 commits intomasterfrom
add_tag_filter

Conversation

@black-adder
Copy link
Copy Markdown
Contributor

tag_filter allows the ability to decide which tags we want to index in cassandra. Some tags aren't worth indexing and this will allow you to customize which ones to filter out.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 100.0% when pulling aab9f9a on add_tag_filter into 82eeb27 on master.

@black-adder black-adder reopened this Sep 29, 2017
@black-adder
Copy link
Copy Markdown
Contributor Author

I guess I need to signoff on every commit?

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 100.0% when pulling aab9f9a on add_tag_filter into 82eeb27 on master.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 100.0% when pulling b14c75c on add_tag_filter into 5030513 on master.

)

// FilterTags returns span tags that should be inserted into cassandra.
type FilterTags func(span *model.Span) []TagInsertion
Copy link
Copy Markdown
Member

@yurishkuro yurishkuro Oct 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not too psyched with the data dependencies here. Filtering is unrelated to building TagInsertion.
Why not have the filter only depend on the domain model (e.g. Span and Tag for tags, and Span and Log for logs) and return true/false?

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 100.0% when pulling 48d6781 on add_tag_filter into 54151f8 on master.

@yurishkuro
Copy link
Copy Markdown
Member

yurishkuro commented Oct 11, 2017

@caniszczyk FOSSA complains about GPL license in some of the dependencies, is that valid? Looks like 4 of them come from spf13/cobra, which is a library used by Docker, Hugo, etc.

Copy link
Copy Markdown
Member

@yurishkuro yurishkuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't quite address my concern. The "Filter" is not just a filter, it's also a flattener, which doesn't have to be part of its function. That means we cannot chain the filters since the first one will flatten everything and deprive the next one from having access to semantics of each KeyValue. While it's a bit more work, but a cleaner model is to have independent filters for logs, tags, and process tags. It could be one interface with 3 methods, or three independent interfaces.

import "github.com/uber/jaeger/model"

// FilterLogTags returns a filter that filters span.Logs.
func FilterLogTags() FilterTags {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you mean "log fields"?

spanHb.collectorOpts.WriteCacheTTL,
spanHb.metricsFactory,
spanHb.logger,
casSpanstoreModel.DefaultTagFilter(),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would be better as a functional option, as the arg list is starting to get too long (and this one is clearly optional).

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0.09%) to 99.907% when pulling f9f6f53 on add_tag_filter into 54151f8 on master.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 100.0% when pulling 8013314 on add_tag_filter into 54151f8 on master.

Copy link
Copy Markdown
Member

@yurishkuro yurishkuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aside from one concern lgtm

allTags = append(allTags, process.Tags...)
// GetAllUniqueTags creates a list of all unique tags from a set of filtered tags.
func GetAllUniqueTags(span *model.Span, tagFilter TagFilter) []TagInsertion {
tags := tagFilter.FilterProcessTags(span.Process.Tags)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • allTags was a better name imo (and would cause fewer spurious changes in this diff)
  • are you sure you don't need to init to nil? if FilterProcessTags simply returns its arg, and the underlying array had extra capacity, then append might just add to that, creating potential side effects.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wrote some primitive code: https://play.golang.org/p/5MbPA7Siya looks like the existing process tags doesn't get touched even if it has extra capacity

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Signed-off-by: Won Jun Jang <wjang@uber.com>
Signed-off-by: Won Jun Jang <wjang@uber.com>
Signed-off-by: Won Jun Jang <wjang@uber.com>
Signed-off-by: Won Jun Jang <wjang@uber.com>
Signed-off-by: Won Jun Jang <wjang@uber.com>
Signed-off-by: Won Jun Jang <wjang@uber.com>
@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 100.0% when pulling c73737f on add_tag_filter into 0549efd on master.

Signed-off-by: Won Jun Jang <wjang@uber.com>
@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 100.0% when pulling c2c0a9b on add_tag_filter into 0549efd on master.

@yurishkuro yurishkuro merged commit c3e6e14 into master Oct 12, 2017
@ghost ghost removed the review label Oct 12, 2017
ideepika pushed a commit to ideepika/jaeger that referenced this pull request Oct 22, 2017
* Add cassandra tag filter

Signed-off-by: Won Jun Jang <wjang@uber.com>
isaachier pushed a commit to isaachier/jaeger that referenced this pull request Nov 1, 2017
* Add cassandra tag filter

Signed-off-by: Won Jun Jang <wjang@uber.com>
@yurishkuro yurishkuro deleted the add_tag_filter branch December 23, 2017 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants