Description
With JMX Insights, we have the ability to capture JMX metrics using a simple YAML configuration file.
As found while working on #14067 , some JMX MBeans have some technical details leak into their object names, which forces to capture extra "technical metric attributes" that are not relevant for the metric consumer.
MBeans are published in a tree-like structure with optional key/value pairs, the metric values are extracted from their MBean attributes.
JMX metrics are extracted from those using one or more MBean query that can have wildcards for those key/value pairs
For example, with the Jetty application server, when we need to capture the "select count" metric:
org.eclipse.jetty.io:context=*,type=managedselector,id=*
MBean query allows to get all the MBean instances providing theselectCount
MBean attribute- by default 10 instances of such MBean exist, each with a distinct value of
id
key. - the current JMX implementation will attempt to capture all of those as 10 distinct metrics with the same identity unless we map the
id
key value as a metric attribute, hence producing 10 distinct metrics. - we need to provide the aggregate sum of
selectCount
attribute over all those instances and capture it as a single metric - in this particular case, in addition the
context
key also needs to be discarded because there is often a single transient value (thus potentially high-cardinality over time).
Here is how it looks like with VisualVM:
The context
key value is equal to HTTP_1_1@524d6d96
, and the id
key have 10 distinct values from 0
to 9
.
Having the ability to discard some metrics attributes and perform relevant aggregation seems to be possible through the usage of metric views, so I think we should be able to leverage this feature and apply it for JMX metrics.
For the distinct values of id
having those as metric attributes is not relevant for the metric consumer.
For the context
key, we can consider this as an exception (but one we have to get by) as it goes against the recommendations for MBean names, in particular for this statement:
The name of the same object should not change between different executions of the application, so for example it should not contain items like a "JVM id" or the "MBean Server Id" that will change with each execution