Kafkametrics Receiver

Available in: contrib Maintainers: @dmitryax Source: opentelemetry-collector-contrib

Supported Telemetry

Overview

Note: This receiver was renamed from kafkametrics to kafka_metrics to match the snake_case naming convention. The deprecated component type kafkametrics is still accepted as an alias and will log a deprecation warning.

Prerequisites

This receiver supports Kafka versions:

Feature gates

See documentation.md for the complete list of feature gates supported by this processor. Feature gates can be enabled using the --feature-gates flag:

"--feature-gates=<feature-gate>"

Getting Started

Required settings (no defaults):

scrapers: any combination of the following scrapers can be enabled.
- topics
- consumers
- brokers

Metrics collected by the associated scraper are listed in metadata.yaml Optional Settings (with defaults):

cluster_alias: Alias name of the cluster. Adds kafka.cluster.alias resource attribute.
The cluster’s unique ID can be auto-discovered from cluster metadata and emitted as the kafka.cluster.id resource attribute. It is disabled by default; enable it via resource_attributes.kafka.cluster.id.enabled: true.
protocol_version (default = 2.1.0): Kafka protocol version
brokers (default = localhost:9092): the list of brokers to read from.
resolve_canonical_bootstrap_servers_only (default = false): whether to resolve then reverse-lookup broker IPs during startup.
topic_match (default = ^[^_].*$): regex pattern of topics to filter on metrics collection. The default filter excludes internal topics (starting with _).
group_match (default = .*): regex pattern of consumer groups to filter on for metrics.
client_id (default = otel-collector): consumer client id
collection_interval (default = 1m): frequency of metric collection/scraping.
initial_delay (default = 1s): defines how long this receiver waits before starting.
tls: see TLS Configuration Settings for the full set of available options.
auth (default none)
- plain_text (Deprecated in v0.123.0: use sasl with mechanism set to PLAIN instead.)
  - username: The username to use.
  - password: The password to use
- sasl
  - username: The username to use.
  - password: The password to use.
  - mechanism: The sasl mechanism to use (SCRAM-SHA-256, SCRAM-SHA-512, AWS_MSK_IAM_OAUTHBEARER, or PLAIN)
  - aws_msk
    - region: AWS Region in case of AWS_MSK_IAM_OAUTHBEARER mechanism
- tls ((Deprecated in v0.124.0: configure tls at the top level): this is an alias for tls at the top level.
- kerberos
  - service_name: Kerberos service name
  - realm: Kerberos realm
  - use_keytab: Use of keytab instead of password, if this is true, keytab file will be used instead of password
  - username: The Kerberos username used for authenticate with KDC
  - password: The Kerberos password used for authenticate with KDC
  - config_file: Path to Kerberos configuration. i.e /etc/krb5.conf
  - keytab_file: Path to keytab file. i.e /etc/security/kafka.keytab
  - disable_fast_negotiation: Disable PA-FX-FAST negotiation (Pre-Authentication Framework - Fast). Some common Kerberos implementations do not support PA-FX-FAST negotiation. This is set to false by default.
metadata
- full (default = true): Whether to maintain a full set of metadata. When disabled, the client does not make the initial request to broker at the startup.
- retry
  - max (default = 3): The number of retries to get metadata
  - backoff (default = 250ms): How long to wait between metadata retries

Examples:

Basic configuration with all scrapers:

receivers:
  kafka_metrics:
    scrapers:
      - brokers
      - topics
      - consumers

Configuration with more optional settings:

For this example:

A non-default broker is specified
cluster alias is set to “kafka-prod”
collection interval is 5 secs.
Kafka protocol version is 3.0.0
mTLS is configured

receivers:
  kafka_metrics:
    cluster_alias: kafka-prod
    brokers: ["10.10.10.10:9092"]
    protocol_version: 3.0.0
    scrapers:
      - brokers
      - topics
      - consumers
    tls:
      ca_file: ca.pem
      cert_file: cert.pem
      key_file: key.pem
    collection_interval: 5s

Metrics

Metric Name	Description	Unit	Type	Attributes
❌ `kafka.broker.log_retention_period`	log retention time (s) of a broker.	s	Gauge	broker
✅ `kafka.brokers`	Number of brokers in the cluster.	{brokers}	UpDownCounter
✅ `kafka.consumer_group.lag`	Current approximate lag of consumer group at partition of topic	1	Gauge	group, topic, partition
✅ `kafka.consumer_group.lag_sum`	Current approximate sum of consumer group lag across all partitions of topic	1	Gauge	group, topic
✅ `kafka.consumer_group.members`	Count of members in the consumer group	{members}	UpDownCounter	group
✅ `kafka.consumer_group.offset`	Current offset of the consumer group at partition of topic	1	Gauge	group, topic, partition
✅ `kafka.consumer_group.offset_sum`	Sum of consumer group offset across partitions of topic	1	Gauge	group, topic
✅ `kafka.partition.current_offset`	Current offset of partition of topic.	1	Gauge	topic, partition
✅ `kafka.partition.oldest_offset`	Oldest offset of partition of topic	1	Gauge	topic, partition
✅ `kafka.partition.replicas`	Number of replicas for partition of topic	{replicas}	UpDownCounter	topic, partition
✅ `kafka.partition.replicas_in_sync`	Number of synchronized replicas of partition	{replicas}	UpDownCounter	topic, partition
❌ `kafka.topic.log_retention_period`	log retention period of a topic (s).	s	Gauge	topic
❌ `kafka.topic.log_retention_size`	log retention size of a topic in Bytes, The value (-1) indicates infinite size.	By	Gauge	topic
❌ `kafka.topic.min_insync_replicas`	minimum in-sync replicas of a topic.	{replicas}	Gauge	topic
✅ `kafka.topic.partitions`	Number of partitions in topic.	{partitions}	UpDownCounter	topic
❌ `kafka.topic.replication_factor`	replication factor of a topic.	1	Gauge	topic

Attributes

Attribute Name	Description	Type
`broker`	The ID of the kafka broker	string
`group`	The ID (string) of a consumer group	string
`partition`	The number (integer) of the partition	int
`topic`	The ID (integer) of a topic	string

Resource Attributes

Attribute Name	Description	Type	Enabled
`kafka.cluster.alias`	The alias name (string) of the cluster	string	❌
`kafka.cluster.id`	The unique ID (string) of the Kafka cluster, as reported in cluster metadata.	string	❌

Configuration

Example Configuration

kafka_metrics:
  cluster_alias: kafka-test
  brokers: 10.10.10.10:9092
  scrapers:
    - brokers
    - topics
    - consumers
  metadata:
    # Set a non-default value in metadata to ensure
    # the refresh_frequency -> refresh_interval alias
    # does not destroy any other config.
    full: false
  refresh_frequency: 1
  topic_match: test_\w+
  group_match: test_\w+

Last generated: 2026-08-24

​Kafkametrics Receiver

​Supported Telemetry

​Overview

​Prerequisites

​Feature gates

​Getting Started

​Examples:

​Metrics

​Attributes

​Resource Attributes

​Configuration

​Example Configuration