Skip to content

fix: correct end time when a trace is outside the ingestion slack range#3954

Merged
joe-elliott merged 4 commits intografana:mainfrom
javiermolinar:fix-block-time-for-old-traces
Aug 12, 2024
Merged

fix: correct end time when a trace is outside the ingestion slack range#3954
joe-elliott merged 4 commits intografana:mainfrom
javiermolinar:fix-block-time-for-old-traces

Conversation

@javiermolinar
Copy link
Copy Markdown
Contributor

@javiermolinar javiermolinar commented Aug 9, 2024

What this PR does:
This PR addresses a bug where traces older than the ingestion slack are ingested. When that happened, the block start time exceeded the block end time. This led the compactor to remove those blocks, and the inability to search them by id.

Example of the current error
Trace data:

{
          "traceId": "5B8EFFF798038103D269B633813FC703",
          "spanId": "EEE19B7EC3C1B100",
          "name": "I am a span!",
          "startTimeUnixNano": 1689969302000000000,
          "endTimeUnixNano": 1689970000000000000,
          "kind": 2,
          "status": {
            "code": "STATUS_CODE_OK"
          },
          "attributes": [
          {
              "key": "my.span.attr",
              "value": {
                  "stringValue": "some value"
              }
}

The start and date are from 2023

Resulting block:

window: 2023-07-21T20:00:00Z
start: 2024-08-07T17:21:44+02:00
end: 2023-07-21T22:06:40+02:00
duration:  -9187h15m4s
age: 9203h48m32s

The start date is being set as now() whereas the end date is left as it is in the span data.

Example after the PR changes

{"format":"vParquet4","blockID":"be564c9d-f76d-49ee-9668-24b6cc024eed","minID":"W47/95gDgQPSabYzgT/HAw==","maxID":"W47/95gDgQPSabYzgT/HAw==","tenantID":"single-tenant","startTime":"2024-08-09T15:45:16.837762+02:00","endTime":"2024-08-09T15:47:18+02:00","totalObjects":2,"size":0,"compactionLevel":0,"encoding":"none","indexPageSize":0,"totalRecords":0,"dataEncoding":"","bloomShards":0,"footerSize":0}

Now, in the wall block, the start and end times are set to now() and there are no inconsistencies.

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@javiermolinar javiermolinar changed the title fix: correct end time when a trace is appended to a block if is smaller than start time after the adjustment fix: correct end time when a trace is outside the ingestion slack range Aug 9, 2024
// start/end are unix epoch seconds
func (b *BlockMeta) ObjectAdded(id []byte, start, end uint32) {
// start/end are unix epoch seconds, when 0 the start and the end are not applied.
func (b *BlockMeta) ObjectAdded(start, end uint32) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing id was missed in the min/max removal? nice cleanup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants