> ## Documentation Index > Fetch the complete documentation index at: https://otel.fyi/llms.txt > Use this file to discover all available pages before exploring further. # Kubeletstats > OpenTelemetry receiver for Kubeletstats # Kubeletstats Receiver ![Status](https://img.shields.io/badge/status-beta-yellow) **Available in:** `contrib`, `k8s` **Maintainers:** [@dmitryax](https://github.com/dmitryax), [@TylerHelmuth](https://github.com/TylerHelmuth), [@ChrsMark](https://github.com/ChrsMark) **Source:** [opentelemetry-collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kubeletstatsreceiver) ## Supported Telemetry ![Metrics](https://img.shields.io/badge/metrics-beta-green) ## Overview ## Metrics Details about the metrics produced by this receiver can be found in [metadata.yaml](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kubeletstatsreceiver/metadata.yaml) with further documentation in [documentation.md](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kubeletstatsreceiver/documentation.md) ## Compatibility ### Kubernetes Versions This receiver is tested against the Kubernetes versions specified in the [e2e-tests.yml](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/.github/workflows/e2e-tests.yml#L97-L99) workflow. These tested versions represent the officially supported Kubernetes versions for this component. ## Configuration A kubelet runs on a kubernetes node and has an API server to which this receiver connects. To configure this receiver, you have to tell it how to connect and authenticate to the API server and how often to collect data and send it to the next consumer. Kubelet Stats Receiver supports both secure Kubelet endpoint exposed at port 10250 by default and read-only Kubelet endpoint exposed at port 10255. If `auth_type` set to `none`, the read-only endpoint will be used. The secure endpoint will be used if `auth_type` set to any of the following values: * `tls` tells the receiver to use TLS for auth and requires that the fields `ca_file`, `key_file`, and `cert_file` also be set. * `serviceAccount` tells this receiver to use the default service account token to authenticate to the kubelet API along with the default certificate which is signed by the cluster's root CA cert: * `/var/run/secrets/kubernetes.io/serviceaccount/token` * `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt` * `kubeConfig` tells this receiver to use the kubeconfig file (KUBECONFIG env variable or \~/.kube/config) to authenticate and use API server proxy to access the kubelet API. * `initial_delay` (default = `1s`): defines how long this receiver waits before starting. ### TLS Example ```yaml theme={null} receivers: kubelet_stats: collection_interval: 20s initial_delay: 1s auth_type: "tls" ca_file: "/path/to/ca.crt" key_file: "/path/to/apiserver.key" cert_file: "/path/to/apiserver.crt" endpoint: "https://192.168.64.1:10250" insecure_skip_verify: true exporters: file: path: "fileexporter.txt" service: pipelines: metrics: receivers: [kubelet_stats] exporters: [file] ``` ### Service Account Authentication Example Although it's possible to use kubernetes' hostNetwork feature to talk to the kubelet api from a pod, the preferred approach is to use the downward API. Make sure the pod spec sets the node name as follows: ```yaml theme={null} env: - name: K8S_NODE_NAME valueFrom: fieldRef: fieldPath: spec.nodeName ``` Then the otel config can reference the `K8S_NODE_NAME` environment variable: ```yaml theme={null} receivers: kubelet_stats: collection_interval: 20s auth_type: "serviceAccount" endpoint: "https://${env:K8S_NODE_NAME}:10250" insecure_skip_verify: true exporters: file: path: "fileexporter.txt" service: pipelines: metrics: receivers: [kubelet_stats] exporters: [file] ``` Note: a missing or empty `endpoint` will cause the hostname on which the collector is running to be used as the endpoint. If the hostNetwork flag is set, and the collector is running in a pod, this hostname will resolve to the node's network namespace. #### Custom CA The service account client, by default, uses the CA certificate located at `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt` to validate the kubelet certificate. If the kubelet server uses a certificate issued by a different CA, specify the custom CA certificate path using the `ca_file` option. ##### AKS Custom CA example This use case applies to AKS cluster, where the kubelet certificate is issued by `/etc/kubernetes/certs/kubeletserver.crt` ```yaml theme={null} receivers: kubelet_stats: collection_interval: 20s auth_type: "serviceAccount" endpoint: "https://${env:K8S_NODE_NAME}:10250" ca_file: "/etc/kubernetes/certs/kubeletserver.crt" ``` ### Read Only Endpoint Example The following config can be used to collect Kubelet metrics from read-only endpoint: ```yaml theme={null} receivers: kubelet_stats: collection_interval: 20s auth_type: "none" endpoint: "http://${env:K8S_NODE_NAME}:10255" exporters: file: path: "fileexporter.txt" service: pipelines: metrics: receivers: [kubelet_stats] exporters: [file] ``` ### Kubeconfig example The following config can be used to collect Kubelet metrics from read-only endpoint, proxied by the API server: ```yaml theme={null} receivers: kubelet_stats: collection_interval: 20s auth_type: "kubeConfig" context: "my-context" insecure_skip_verify: true endpoint: "${env:K8S_NODE_NAME}" exporters: file: path: "fileexporter.txt" service: pipelines: metrics: receivers: [kubelet_stats] exporters: [file] ``` Note that using `auth_type` `kubeConfig`, the endpoint should only be the node name as the communication to the kubelet is proxied by the API server configured in the `kubeConfig`. `insecure_skip_verify` still applies by overriding the `kubeConfig` settings. If no `context` is specified, the current context or the default context is used. ### Extra metadata labels By default, all produced metrics get resource labels based on what kubelet /stats/summary endpoint provides. For some use cases it might be not enough. So it's possible to leverage other endpoints to fetch additional metadata entities and set them as extra labels on metric resource. Currently supported metadata include the following: * `container.id` - to augment metrics with Container ID label obtained from container statuses exposed via `/pods`. * `k8s.volume.type` - to collect volume type from the Pod spec exposed via `/pods` and have it as a label on volume metrics. If there's more information available from the endpoint than just volume type, those are synced as well depending on the available fields and the type of volume. For example, `aws.volume.id` would be synced from `awsElasticBlockStore` and `gce.pd.name` is synced for `gcePersistentDisk`. > \[!WARNING] > `gce.pd.name`, `aws.volume.id`, `glusterfs.path`, and `glusterfs.endpoints.name` resource attributes are all deprecated and will be removed soon. If you want to have `container.id` label added to your metrics, use `extra_metadata_labels` field to enable it, for example: ```yaml theme={null} receivers: kubelet_stats: collection_interval: 10s auth_type: "serviceAccount" endpoint: "${env:K8S_NODE_NAME}:10250" insecure_skip_verify: true extra_metadata_labels: - container.id ``` If `extra_metadata_labels` is not set, no additional API calls is done to fetch extra metadata. #### Collecting Additional Volume Metadata When dealing with Persistent Volume Claims, it is possible to optionally sync metdadata from the underlying storage resource rather than just the volume claim. This is achieved by talking to the Kubernetes API. Below is an example, configuration to achieve this. ```yaml theme={null} receivers: kubelet_stats: collection_interval: 10s auth_type: "serviceAccount" endpoint: "${env:K8S_NODE_NAME}:10250" insecure_skip_verify: true extra_metadata_labels: - k8s.volume.type k8s_api_config: auth_type: serviceAccount ``` If `k8s_api_config` is set, the receiver will attempt to collect metadata from underlying storage resources for Persistent Volume Claims. For example, if a Pod is using a PVC backed by an EBS instance on AWS, the receiver would set the `k8s.volume.type` label to be `awsElasticBlockStore` rather than `persistentVolumeClaim`. > \[!WARNING] > awsElasticBlockStore resource attribute `aws.volume.id` is deprecated and will be removed soon **Important**: When using `k8s_api_config`, the service account must have `get` permissions for `persistentvolumeclaims` and `persistentvolumes` resources. See [Role-based access control](#role-based-access-control) for the required RBAC configuration. ### Metric Groups A list of metric groups from which metrics should be collected. By default, metrics from containers, pods and nodes will be collected. If `metric_groups` is set, only metrics from the listed groups will be collected. Valid groups are `container`, `pod`, `node` and `volume`. For example, if you're looking to collect only `node` and `pod` metrics from the receiver use the following configuration. ```yaml theme={null} receivers: kubelet_stats: collection_interval: 10s auth_type: "serviceAccount" endpoint: "${env:K8S_NODE_NAME}:10250" insecure_skip_verify: true metric_groups: - node - pod ``` ### Network metrics from all interfaces for Node and Pod By default, `k8s.[node|pod].network.*` metrics are collected only for the default network interface (e.g. `eth0`). To enable network IO/error metrics collection from all available interfaces on Node/Pod level - you can use `collect_all_network_interfaces` configuration parameters. Please be aware that enabling this options will increase the amount of produced network metrics and increase network metrics cardinality, because of `interface` attribute. For example, if you would like to have network IO/error metrics from all network interfaces for both Pod and Node level you can use the following configuration. ```yaml theme={null} receivers: kubelet_stats: collection_interval: 10s auth_type: "serviceAccount" endpoint: "${env:K8S_NODE_NAME}:10250" insecure_skip_verify: true collect_all_network_interfaces: pod: true node: true ``` ### Collect `k8s.{container,pod}.{cpu,memory}.node.utilization` as ratio of total node's capacity In order to calculate the `k8s.container.cpu.node.utilization`, `k8s.pod.cpu.node.utilization`, `k8s.container.memory.node.utilization` and `k8s.pod.memory.node.utilization` metrics, the information of the node's capacity must be retrieved from the k8s API. In this, the `k8s_api_config` needs to be set. In addition, the node name must be identified properly. The `K8S_NODE_NAME` env var can be set using the downward API inside the collector pod spec as follows: ```yaml theme={null} env: - name: K8S_NODE_NAME valueFrom: fieldRef: fieldPath: spec.nodeName ``` Then set `node` value to `${env:K8S_NODE_NAME}` in the receiver's configuration: ```yaml theme={null} receivers: kubelet_stats: collection_interval: 10s auth_type: 'serviceAccount' endpoint: '${env:K8S_NODE_NAME}:10250' node: '${env:K8S_NODE_NAME}' k8s_api_config: auth_type: serviceAccount metrics: k8s.container.cpu.node.utilization: enabled: true k8s.pod.cpu.node.utilization: enabled: true k8s.container.memory.node.utilization: enabled: true k8s.pod.memory.node.utilization: enabled: true ``` ### Optional parameters The following parameters can also be specified: * `collection_interval` (default = `10s`): The interval at which to collect data. * `insecure_skip_verify` (default = `false`): Whether or not to skip certificate verification. The full list of settings exposed for this receiver are documented in [config.go](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kubeletstatsreceiver/config.go) with detailed sample configurations in [testdata/config.yaml](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/kubeletstatsreceiver/testdata/config.yaml). ### Role-based access control The Kubelet Stats Receiver needs `get` permissions on the `nodes/stats` resources. Additionally, when using `extra_metadata_labels` or any of the `{request|limit}_utilization` metrics the receiver also needs `get` permissions for `nodes/pods` resources. When using `k8s_api_config` to collect detailed volume metadata from PersistentVolumeClaims (as described in [Collecting Additional Volume Metadata](#collecting-additional-volume-metadata)), the receiver also needs `get` permissions for `persistentvolumeclaims` and `persistentvolumes` resources. ```yaml theme={null} apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: otel-collector rules: - apiGroups: [""] resources: ["nodes/stats"] verbs: ["get"] # Only needed if you are using extra_metadata_labels or # are collecting the request/limit utilization metrics - apiGroups: [""] resources: ["nodes/pods"] verbs: ["get"] # Only needed if you are using k8s_api_config to collect # detailed volume metadata from PersistentVolumeClaims - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get"] - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get"] ``` ## Metrics | Metric Name | Description | Unit | Type | Attributes | | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- | ------ | ------------- | -------------------- | | ✅ `container.cpu.time` | Total cumulative CPU time (sum of all cores) spent by the container/pod/node since its creation | s | Counter | | | ✅ `container.cpu.usage` | Total CPU usage (sum of all cores per second) averaged over the sample window | \{cpu} | Gauge | | | ✅ `container.filesystem.available` | Container filesystem available | By | Gauge | | | ✅ `container.filesystem.capacity` | Container filesystem capacity | By | Gauge | | | ✅ `container.filesystem.usage` | Container filesystem usage | By | Gauge | | | ✅ `container.memory.available` | Container memory available | By | Gauge | | | ✅ `container.memory.major_page_faults` | Container memory major\_page\_faults | 1 | Gauge | | | ✅ `container.memory.page_faults` | Container memory page\_faults | 1 | Gauge | | | ✅ `container.memory.rss` | Container memory rss | By | Gauge | | | ✅ `container.memory.usage` | Container memory usage | By | Gauge | | | ✅ `container.memory.working_set` | Container memory working\_set | By | Gauge | | | ❌ `container.uptime` | The time since the container started | s | Counter | | | ❌ `k8s.container.cpu.node.utilization` | Container cpu utilization as a ratio of the node's capacity | 1 | Gauge | | | ❌ `k8s.container.cpu_limit_utilization` | Container cpu utilization as a ratio of the container's limits | 1 | Gauge | | | ❌ `k8s.container.cpu_request_utilization` | Container cpu utilization as a ratio of the container's requests | 1 | Gauge | | | ❌ `k8s.container.ephemeral_storage.usage` | Ephemeral storage used by the container. | By | UpDownCounter | fs.type | | ❌ `k8s.container.memory.node.utilization` | Container memory utilization as a ratio of the node's capacity | 1 | Gauge | | | ❌ `k8s.container.memory_limit_utilization` | Container memory utilization as a ratio of the container's limits | 1 | Gauge | | | ❌ `k8s.container.memory_request_utilization` | Container memory utilization as a ratio of the container's requests | 1 | Gauge | | | ✅ `k8s.node.cpu.time` | Total cumulative CPU time (sum of all cores) spent by the container/pod/node since its creation | s | Counter | | | ✅ `k8s.node.cpu.usage` | Total CPU usage (sum of all cores per second) averaged over the sample window | \{cpu} | Gauge | | | ✅ `k8s.node.filesystem.available` | Node filesystem available | By | Gauge | | | ✅ `k8s.node.filesystem.capacity` | Node filesystem capacity | By | Gauge | | | ✅ `k8s.node.filesystem.usage` | Node filesystem usage | By | Gauge | | | ✅ `k8s.node.memory.available` | Node memory available | By | Gauge | | | ✅ `k8s.node.memory.major_page_faults` | Node memory major\_page\_faults | 1 | Gauge | | | ✅ `k8s.node.memory.page_faults` | Node memory page\_faults | 1 | Gauge | | | ✅ `k8s.node.memory.rss` | Node memory rss | By | Gauge | | | ✅ `k8s.node.memory.usage` | Node memory usage | By | Gauge | | | ✅ `k8s.node.memory.working_set` | Node memory working\_set | By | Gauge | | | ✅ `k8s.node.network.errors` | Node network errors | 1 | Counter | interface, direction | | ✅ `k8s.node.network.io` | Node network IO | By | Counter | interface, direction | | ❌ `k8s.node.system_container.cpu.time` | Total cumulative CPU time (sum of all cores) spent by the system container since its creation | s | Counter | | | ❌ `k8s.node.system_container.cpu.usage` | Total CPU usage (sum of all cores per second) averaged over the sample window for the system container | \{cpu} | Gauge | | | ❌ `k8s.node.system_container.memory.usage` | System container memory usage | By | Gauge | | | ❌ `k8s.node.system_container.memory.working_set` | System container memory working\_set | By | Gauge | | | ❌ `k8s.node.uptime` | The time since the node started | s | Counter | | | ❌ `k8s.pod.cpu.node.utilization` | Pod cpu utilization as a ratio of the node's capacity | 1 | Gauge | | | ✅ `k8s.pod.cpu.time` | Total cumulative CPU time (sum of all cores) spent by the container/pod/node since its creation | s | Counter | | | ✅ `k8s.pod.cpu.usage` | Total CPU usage (sum of all cores per second) averaged over the sample window | \{cpu} | Gauge | | | ❌ `k8s.pod.cpu_limit_utilization` | Pod cpu utilization as a ratio of the pod's total container limits. If any container is missing a limit the metric is not emitted. | 1 | Gauge | | | ❌ `k8s.pod.cpu_request_utilization` | Pod cpu utilization as a ratio of the pod's total container requests. If any container is missing a request the metric is not emitted. | 1 | Gauge | | | ✅ `k8s.pod.filesystem.available` | Pod filesystem available | By | Gauge | | | ✅ `k8s.pod.filesystem.capacity` | Pod filesystem capacity | By | Gauge | | | ✅ `k8s.pod.filesystem.usage` | Pod filesystem usage | By | Gauge | | | ✅ `k8s.pod.memory.available` | Pod memory available | By | Gauge | | | ✅ `k8s.pod.memory.major_page_faults` | Pod memory major\_page\_faults | 1 | Gauge | | | ❌ `k8s.pod.memory.node.utilization` | Pod memory utilization as a ratio of the node's capacity | 1 | Gauge | | | ✅ `k8s.pod.memory.page_faults` | Pod memory page\_faults | 1 | Gauge | | | ✅ `k8s.pod.memory.rss` | Pod memory rss | By | Gauge | | | ✅ `k8s.pod.memory.usage` | Pod memory usage | By | Gauge | | | ✅ `k8s.pod.memory.working_set` | Pod memory working\_set | By | Gauge | | | ❌ `k8s.pod.memory_limit_utilization` | Pod memory utilization as a ratio of the pod's total container limits. If any container is missing a limit the metric is not emitted. | 1 | Gauge | | | ❌ `k8s.pod.memory_request_utilization` | Pod memory utilization as a ratio of the pod's total container requests. If any container is missing a request the metric is not emitted. | 1 | Gauge | | | ✅ `k8s.pod.network.errors` | Pod network errors | 1 | Counter | interface, direction | | ✅ `k8s.pod.network.io` | Pod network IO | By | Counter | interface, direction | | ❌ `k8s.pod.uptime` | The time since the pod started | s | Counter | | | ❌ `k8s.pod.volume.usage` | The number of used bytes in the pod volume. | By | UpDownCounter | | | ✅ `k8s.volume.available` | The number of available bytes in the volume. | By | Gauge | | | ✅ `k8s.volume.capacity` | The total capacity in bytes of the volume. | By | Gauge | | | ✅ `k8s.volume.inodes` | The total inodes in the filesystem. | 1 | Gauge | | | ✅ `k8s.volume.inodes.free` | The free inodes in the filesystem. | 1 | Gauge | | | ✅ `k8s.volume.inodes.used` | The inodes used by the filesystem. This may not equal inodes - free because filesystem may share inodes with other filesystems. | 1 | Gauge | | ## Attributes | Attribute Name | Description | Type | Values | | -------------- | ------------------------------------------------------------ | ------ | --------------------- | | `direction` | Direction of flow of bytes/operations (receive or transmit). | string | `receive`, `transmit` | | `fs.type` | The type of filesystem component. | string | `rootfs`, `logs` | | `interface` | Name of the network interface. | string | | ## Resource Attributes | Attribute Name | Description | Type | Enabled | | -------------------------------- | --------------------------------------------------------------- | ------ | ------- | | `aws.volume.id` | \[DEPRECATED] The id of the AWS Volume | string | ❌ | | `container.id` | Container id used to identify container | string | ✅ | | `fs.type` | \[DEPRECATED] The filesystem type of the Volume | string | ❌ | | `gce.pd.name` | \[DEPRECATED] The name of the persistent disk in GCE | string | ❌ | | `glusterfs.endpoints.name` | \[DEPRECATED] The endpoint name that details Glusterfs topology | string | ❌ | | `glusterfs.path` | \[DEPRECATED] Glusterfs volume path | string | ❌ | | `k8s.container.name` | Container name used by container runtime | string | ✅ | | `k8s.namespace.name` | The name of the namespace that the pod is running in | string | ✅ | | `k8s.node.name` | The name of the Node | string | ✅ | | `k8s.node.system_container.name` | The name of the system container | string | ✅ | | `k8s.persistentvolumeclaim.name` | The name of the Persistent Volume Claim | string | ✅ | | `k8s.pod.name` | The name of the Pod | string | ✅ | | `k8s.pod.uid` | The UID of the Pod | string | ✅ | | `k8s.volume.name` | The name of the Volume | string | ✅ | | `k8s.volume.type` | The type of the Volume | string | ✅ | | `partition` | \[DEPRECATED] The partition in the Volume | string | ❌ | ## Configuration ### Example Configuration ```yaml theme={null} kubelet_stats/default: kubelet_stats/tls: collection_interval: 10s auth_type: "tls" ca_file: "/path/to/ca.crt" key_file: "/path/to/apiserver.key" cert_file: "/path/to/apiserver.crt" endpoint: "1.2.3.4:5555" insecure_skip_verify: true kubelet_stats/sa: collection_interval: 10s auth_type: "serviceAccount" insecure_skip_verify: true kubelet_stats/metadata: collection_interval: 10s auth_type: "serviceAccount" extra_metadata_labels: - container.id - k8s.volume.type kubelet_stats/metadata_with_k8s_api: collection_interval: 10s auth_type: "serviceAccount" extra_metadata_labels: - k8s.volume.type k8s_api_config: auth_type: kubeConfig kubelet_stats/metric_groups: collection_interval: 20s auth_type: "serviceAccount" metric_groups: [ pod, node, volume ] kubelet_stats/container_cpu_node_utilization: collection_interval: 10s metric_groups: [ container, pod, node ] metrics: k8s.container.cpu.node.utilization: enabled: true kubelet_stats/pod_cpu_node_utilization: collection_interval: 10s metric_groups: [ container, pod, node ] metrics: k8s.pod.cpu.node.utilization: enabled: true kubelet_stats/container_memory_node_utilization: collection_interval: 10s metric_groups: [ container, pod, node ] metrics: k8s.container.memory.node.utilization: enabled: true kubelet_stats/pod_memory_node_utilization: collection_interval: 10s metric_groups: [ container, pod, node ] metrics: k8s.pod.memory.node.utilization: enabled: true kubelet_stats/all_network_interfaces_metrics: collection_interval: 10s metric_groups: [ container, pod, node ] collect_all_network_interfaces: pod: true node: true ``` *** *Last generated: 2026-07-06*