Proposed strategy for making the collector an observable application

[codeboten]

This is a follow up to various conversations that have been had around this topic. I'd like to propose that we make the configuration for the collector follow the proposed configuration for SDKs (which is still under work, see sample configuration from the otep). Ideally I'd like to use the collector as a prototype of the configuration as it evolves.

I propose that most of the configuration parsing be done inside the collector for the time being, with the plan to leverage the Go SDK's implementation in the future. In order for this to work, there is quite a bit of work that needs to happen:

• Update telemetry code to support more than a single prometheus instance #7533
• Support additional exporters for exporting collector metrics #7641
• Support additional exporters for exporting collector traces #8106
• Support additional exporters for exporting collector logs #8118
• Collector telemetry stanza must support additional configuration parameters

Related issues:
#6629
#7223

[mattsains]

Hello, I am interested in helping on this initiative, specifically exporting internal spans. Please let me know what I can do.

For my own testing, I have been writing a hacky extension which registers itself as a span processor for internal traces, converts the sdk span to a ptrace span, and exports it over otlp. However, I think it would be nice to work on a more official feature which could help others as well.

[mattsains]

I think it would be worth adding a new issue here, for being able to turn off the existing telemetry metrics prometheus endpoint. Because even if you use the new export functionality for metrics, the collector still tries to open a prometheus port, and this behaviour cannot be disabled (only relocated to a different port). This causes problems if you try to run two collector instances on the same host

[kasun-bandara]

Hello, I am interested in helping on this initiative, specifically exporting internal spans. Please let me know what I can do.

For my own testing, I have been writing a hacky extension which registers itself as a span processor for internal traces, converts the sdk span to a ptrace span, and exports it over otlp. However, I think it would be nice to work on a more official feature which could help others as well.

Do we have the official processor support for internal traces/metrics yet?