[processor/resourcedetection] Align detectors with opentelemetry-go SDK detectors

[dmitryax]

Component(s)

processor/resourcedetection

Is your feature request related to a problem? Please describe.

With open-telemetry/opentelemetry-collector#14311 landing, the collector's service::telemetry will gain support for the resource detectors from the opentelemetry-go SDK (via the detection/development schema — see open-telemetry/opentelemetry-collector#14426 and #15226). Meanwhile, processor/resourcedetectionprocessor maintains its own parallel set of detector implementations (host, system, process, container, ec2, gcp, azure, k8snode, env, …).

This leaves two independent detection paths in the same binary:

• SDK detectors — used by service::telemetry (and, by extension, the attributes reported by the OpAMP extension via include_resource_attributes).
• resourcedetectionprocessor detectors — used by user telemetry pipelines.

They can, and likely will, drift: different attribute keys, different detection semantics, different failure modes, different priorities. Users will see one set of resource attributes on their collector's self-telemetry/AgentDescription and a different set on the telemetry flowing through their pipelines — even with similar-looking configuration.

This came up in review on open-telemetry/opentelemetry-collector#14412:

I'm a bit concerned about two separate solutions in the same binary that could potentially diverge and produce different results. We probably should start using the sdk detectors libraries in the detection processor going forward.

Describe the solution you'd like

Migrate resourcedetectionprocessor to delegate to the opentelemetry-go SDK detectors wherever an equivalent exists, keeping only contrib-specific detectors as local implementations.

Concretely:

• Audit each detector under processor/resourcedetectionprocessor/internal/ against what the SDK currently offers (host, process, container, service; cloud detectors arriving incrementally).
• For detectors with an SDK equivalent, replace the implementation with a thin adapter over the SDK detector, preserving the existing processor configuration surface for backwards compatibility.
• Keep detectors that have no SDK equivalent (e.g. k8snode, kubeadm, openshift, heroku, consul, dynatrace, regional clouds without upstream coverage) as-is, but document which are SDK-backed and which are contrib-local.
• Ensure attribute keys and values produced by the migrated detectors match what the SDK emits so that a user enabling the same detector in service::telemetry and in the processor gets consistent output.

[github-actions]

Pinging code owners:

• processor/resourcedetection: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[iblancasa]

Thanks for creating this ticket. I have been thinking about this since the discussion in the collector core repository.

• For detectors with an SDK equivalent, replace the implementation with a thin adapter over the SDK detector, preserving the existing processor configuration surface for backwards compatibility.

Would it make sense to contribute to SDK those features not present there?

• Keep detectors that have no SDK equivalent (e.g. k8snode, kubeadm, openshift, heroku, consul, dynatrace, regional clouds without upstream coverage) as-is, but document which are SDK-backed and which are contrib-local.

What do you think about contributing those to the SDK?

[dashpole]

I think it is a good idea to contribute them to the SDK. But i'm a bit overstretched as its with the resourcedetection processor. I can sponsor new detectors in the Go SDK, but I would really prefer not to own even more artifacts. I would like for others who are interested in those detectors to take ownership.

cc @paulojmdias

[paulojmdias]

I agree with contributing those to the SDK, and I'm open to helping and keeping ownership of it on the SDK side.

Should we create an issue and sub-issues for each one in the opentelemetry-go?

[dashpole]

Yes, please do. I'll bring this up at the Go sig meeting later today

[dashpole]

The Go SIG is happy with this approach, as long as we have codeowners for all of the resource detectors.

[paulojmdias]

The Go SIG is happy with this approach, as long as we have codeowners for all of the resource detectors.

Awesome! I can own at the least the ones I current own but I'm also open to help/own other ones. 👍

If no one is planning to create the issues, I Maybe can start doing later today or tomorrow

[paulojmdias]

@dashpole This should live into go-contrib side right ?

There is some detectors already, should we also open an issue for moving/review some existing ones to check if some attributes are missing there ?

[dashpole]

Yes, they will live in contrib. We should audit and resolve any differences