Kafkametrics Receiver

Available in: contrib Maintainers: @dmitryax Source: opentelemetry-collector-contrib

Supported Telemetry

Overview

Note: This receiver was renamed from kafkametrics to kafka_metrics to match the snake_case naming convention. The deprecated component type kafkametrics is still accepted as an alias and will log a deprecation warning.

Prerequisites

This receiver supports Kafka versions 2.x, 3.x and 4.x (including KRaft mode).

Note: Compatibility is determined by the underlying Kafka protocol support in the driver (e.g., franz-go or sarama). Basic metrics collection remains compatible with Kafka 4.x brokers.

Feature gates

See documentation.md for the complete list of feature gates supported by this processor. Feature gates can be enabled using the --feature-gates flag:

"--feature-gates=<feature-gate>"

Getting Started

Required settings (no defaults):

scrapers: any combination of the following scrapers can be enabled.
- topics
- consumers
- brokers

Metrics collected by the associated scraper are listed in metadata.yaml Optional Settings (with defaults):

cluster_alias: Alias name of the cluster. Adds kafka.cluster.alias resource attribute.
The cluster’s unique ID can be auto-discovered from cluster metadata and emitted as the kafka.cluster.id resource attribute. It is disabled by default; enable it via resource_attributes.kafka.cluster.id.enabled: true.
protocol_version (default = 2.1.0): Kafka protocol version
brokers (default = localhost:9092): the list of brokers to read from.
resolve_canonical_bootstrap_servers_only (default = false): whether to resolve then reverse-lookup broker IPs during startup.
topic_match (default = ^[^_].*$): regex pattern of topics to filter on metrics collection. The default filter excludes internal topics (starting with _).
group_match (default = .*): regex pattern of consumer groups to filter on for metrics.
client_id (default = otel-collector): consumer client id
collection_interval (default = 1m): frequency of metric collection/scraping.
initial_delay (default = 1s): defines how long this receiver waits before starting.
tls: see TLS Configuration Settings for the full set of available options.
auth (default none)
- plain_text (Deprecated in v0.123.0: use sasl with mechanism set to PLAIN instead.)
  - username: The username to use.
  - password: The password to use
- sasl
  - username: The username to use.
  - password: The password to use.
  - mechanism: The sasl mechanism to use (SCRAM-SHA-256, SCRAM-SHA-512, AWS_MSK_IAM_OAUTHBEARER, or PLAIN)
  - aws_msk
    - region: AWS Region in case of AWS_MSK_IAM_OAUTHBEARER mechanism
- tls ((Deprecated in v0.124.0: configure tls at the top level): this is an alias for tls at the top level.
- kerberos
  - service_name: Kerberos service name
  - realm: Kerberos realm
  - use_keytab: Use of keytab instead of password, if this is true, keytab file will be used instead of password
  - username: The Kerberos username used for authenticate with KDC
  - password: The Kerberos password used for authenticate with KDC
  - config_file: Path to Kerberos configuration. i.e /etc/krb5.conf
  - keytab_file: Path to keytab file. i.e /etc/security/kafka.keytab
  - disable_fast_negotiation: Disable PA-FX-FAST negotiation (Pre-Authentication Framework - Fast). Some common Kerberos implementations do not support PA-FX-FAST negotiation. This is set to false by default.
metadata
- full (deprecated): No longer has any effect. Metadata is only fetched for the topics the client scrapes.
- retry
  - max (default = 20): The number of times to retry a retriable request. Applies to the admin and offset fetch requests issued by the scrapers. It does not apply to the client’s internal metadata refresh, which caps itself at 3 retries.
  - backoff (default = 250ms): The minimum time to wait before retrying a request. Each successive retry doubles the wait, with jitter applied, capped at max(5s, backoff).

Examples:

Basic configuration with all scrapers:

receivers:
  kafka_metrics:
    scrapers:
      - brokers
      - topics
      - consumers

Configuration with more optional settings:

For this example:

A non-default broker is specified
cluster alias is set to “kafka-prod”
collection interval is 5 secs.
Kafka protocol version is 3.0.0
mTLS is configured

receivers:
  kafka_metrics:
    cluster_alias: kafka-prod
    brokers: ["10.10.10.10:9092"]
    protocol_version: 3.0.0
    scrapers:
      - brokers
      - topics
      - consumers
    tls:
      ca_file: ca.pem
      cert_file: cert.pem
      key_file: key.pem
    collection_interval: 5s

Metrics

Metric Name	Description	Unit	Type	Attributes
❌ `kafka.broker.log_retention_period`	log retention time (s) of a broker.	s	Gauge	broker
✅ `kafka.brokers`	Number of brokers in the cluster.	{brokers}	UpDownCounter
✅ `kafka.consumer_group.lag`	Current approximate lag of consumer group at partition of topic	1	Gauge	group, topic, partition
✅ `kafka.consumer_group.lag_sum`	Current approximate sum of consumer group lag across all partitions of topic	1	Gauge	group, topic
✅ `kafka.consumer_group.members`	Count of members in the consumer group	{members}	UpDownCounter	group
✅ `kafka.consumer_group.offset`	Current offset of the consumer group at partition of topic	1	Gauge	group, topic, partition
✅ `kafka.consumer_group.offset_sum`	Sum of consumer group offset across partitions of topic	1	Gauge	group, topic
✅ `kafka.partition.current_offset`	Current offset of partition of topic.	1	Gauge	topic, partition
✅ `kafka.partition.oldest_offset`	Oldest offset of partition of topic	1	Gauge	topic, partition
✅ `kafka.partition.replicas`	Number of replicas for partition of topic	{replicas}	UpDownCounter	topic, partition
✅ `kafka.partition.replicas_in_sync`	Number of synchronized replicas of partition	{replicas}	UpDownCounter	topic, partition
❌ `kafka.topic.log_retention_period`	log retention period of a topic (s).	s	Gauge	topic
❌ `kafka.topic.log_retention_size`	log retention size of a topic in Bytes, The value (-1) indicates infinite size.	By	Gauge	topic
❌ `kafka.topic.min_insync_replicas`	minimum in-sync replicas of a topic.	{replicas}	Gauge	topic
✅ `kafka.topic.partitions`	Number of partitions in topic.	{partitions}	UpDownCounter	topic
❌ `kafka.topic.replication_factor`	replication factor of a topic.	1	Gauge	topic

Attributes

Attribute Name	Description	Type
`broker`	The ID of the kafka broker	string
`group`	The ID (string) of a consumer group	string
`partition`	The number (integer) of the partition	int
`topic`	The ID (integer) of a topic	string

Resource Attributes

Attribute Name	Description	Type	Enabled
`kafka.cluster.alias`	The alias name (string) of the cluster	string	❌
`kafka.cluster.id`	The unique ID (string) of the Kafka cluster, as reported in cluster metadata.	string	❌

Configuration

Example Configuration

kafka_metrics:
  cluster_alias: kafka-test
  brokers: 10.10.10.10:9092
  scrapers:
    - brokers
    - topics
    - consumers
  metadata:
    # Set a non-default value in metadata to ensure
    # the refresh_frequency -> refresh_interval alias
    # does not destroy any other config.
    full: false
  refresh_frequency: 1
  topic_match: test_\w+
  group_match: test_\w+

Last generated: 2026-10-05

​Kafkametrics Receiver

​Supported Telemetry

​Overview

​Prerequisites

​Feature gates

​Getting Started

​Examples:

​Metrics

​Attributes

​Resource Attributes

​Configuration

​Example Configuration