> ## Documentation Index
> Fetch the complete documentation index at: https://otel.fyi/llms.txt
> Use this file to discover all available pages before exploring further.

# Otelarrow

> OpenTelemetry receiver for Otelarrow

# Otelarrow Receiver

![Status](https://img.shields.io/badge/status-beta-yellow)

**Available in:** `contrib`, `k8s`

**Maintainers:** [@jmacd](https://github.com/jmacd), [@JakeDern](https://github.com/JakeDern)

**Source:** [opentelemetry-collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/otelarrowreceiver)

## Supported Telemetry

![Logs](https://img.shields.io/badge/logs-beta-blue) ![Metrics](https://img.shields.io/badge/metrics-beta-green) ![Traces](https://img.shields.io/badge/traces-beta-orange)

## Overview

## Getting Started

The OpenTelemetry Protocol with Apache Arrow receiver is an extension
of the core OpenTelemetry Collector [OTLP
receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver)
component with additional support for [OpenTelemetry Protocol with
Apache Arrow](https://github.com/open-telemetry/otel-arrow).

OpenTelemetry Protocol with Apache Arrow supports column-oriented data
transport using the Apache Arrow data format.  The [OpenTelemetry
Protocol with Apache Arrow
exporter](../../exporter/otelarrowexporter/README.md)
converts OTLP data into an optimized representation and then sends
batches of data using Apache Arrow to encode the stream.  This
component contains logic to reverse the process used in the
OpenTelemetry Protocol with Apache Arrow exporter.

The use of an OpenTelemetry Protocol with Apache Arrow
exporter-receiver pair is recommended when the network is expensive.
Typically, expect to see a 50% reduction in bandwidth compared with
the same data being sent using standard OTLP/gRPC and gzip
compression.

This component includes all the features and configuration of the core
OTLP receiver, making it possible to upgrade from the core component
simply by replacing "otlp" with "otelarrow" as the component name in
the collector configuration.

To enable the OpenTelemetry Protocol with Apache Arrow receiver,
include it in the list of receivers for a pipeline.  No further
configuration is needed.  This receiver listens on the standard
OTLP/gRPC port 4317 and serves standard OTLP over gRPC out of the box.

```yaml theme={null}
receivers:
  otelarrow:
```

## Advanced Configuration

Users may wish to configure gRPC settings, for example:

```
receivers:
  otelarrow:
    protocols:
      grpc:
        ...
```

* `endpoint` (default = 0.0.0.0:4317 for grpc protocol):
  host:port to which the receiver is going to receive data. The valid syntax is
  described at [https://github.com/grpc/grpc/blob/master/doc/naming.md](https://github.com/grpc/grpc/blob/master/doc/naming.md).

Several common configuration structures provide additional capabilities automatically:

* [gRPC settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configgrpc/README.md)
* [TLS and mTLS settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md)

### Admission Control Configuration

In the `admission` configuration block the following settings are available:

* `request_limit_mib` (default: 128): limits the number of requests that are received by the stream in terms of *uncompressed request size*. This should not be confused with `arrow.memory_limit_mib`, which limits allocations made by the consumer when translating arrow records into pdata objects. The `request_limit_mib` value is used to control how much traffic we admit, but does not control how much memory is used during request processing.

* `waiting_limit_mib` (default: 32): limits the number of requests waiting on admission after `request_limit_mib` is reached. This is another dimension of memory limiting that ensures waiters are not holding onto a significant amount of memory while waiting to be processed.

`request_limit_mib` and `waiting_limit_mib` are arguments supplied to [admission.BoundedQueue](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/internal/otelarrow/admission2). This custom semaphore is meant to be used within receivers to help limit memory within the collector pipeline.

### Arrow-specific Configuration

In the `arrow` configuration block, the following settings are available:

* `memory_limit_mib` (default: 128): limits the amount of concurrent memory used by Arrow data buffers.

When the limit is reached, the receiver will return RESOURCE\_EXHAUSTED
error codes to the receiver, which are [conditionally retryable, see
exporter retry configuration](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md).

### Compression Configuration

In the `arrow` configuration block, `zstd` sub-section applies to all
compression levels used by exporters:

* `memory_limit_mib` limits memory dedicated to Zstd decompression, per stream (default 128)
* `max_window_size_mib`: maximum size of the Zstd window in MiB, 0 indicates to determine based on level (default 32)
* `concurrency`: controls background CPU used for decompression, 0 indicates to let `zstd` library decide (default 1)

### Keepalive configuration

As a gRPC streaming service, the OTel Arrow receiver is able to limit
stream lifetime through configuration of the underlying http/2
connection via keepalive settings.

Keepalive settings are vital to the operation of OTel Arrow, because
longer-lived streams use more memory and streams are fixed to a single
host.  Since every stream of data is different, we recommend
experimenting to find a good balance between memory usage, stream
lifetime, and load balance.

gRPC libraries do not build-in a facility for long-lived RPCs to learn
about impending http/2 connection state changes, including the event
that initiates connection reset.  While the receiver knows its own
keepalive settings, a shorter maximum connection lifetime can be
imposed by intermediate http/2 proxies, and therefore the receiver and
exporter are expected to independently configure these limits.

```
receivers:
  otelarrow:
    protocols:
      grpc:
        keepalive:
          server_parameters:
            max_connection_age: 1m
            max_connection_age_grace: 10m
```

In the example configuration above, OpenTelemetry Protocol with Apache
Arrow streams will have reset initiated after 10 minutes.  Note that
`max_connection_age` is set to a small value and we recommend tuning
`max_connection_age_grace`.

OTel Arrow exporters are expected to configure their
`max_stream_lifetime` property to a value that is slightly smaller
than the receiver's `max_connection_age_grace` setting, which causes
the exporter to cleanly shut down streams, allowing requests to
complete before the http/2 connection is forcibly closed.  While the
exporter will retry data that was in-flight during an unexpected
stream shutdown, instrumentation about the telemetry pipeline will show
RPC errors when the exporter's `max_stream_lifetime` is not configured
correctly.

[See the exporter README for more
guidance](../../exporter/otelarrowexporter/README.md).  For the
example where `max_connection_age_grace` is set to 10 minutes, the
exporter's `max_stream_lifetime` should be set to the same number
minus a reasonable timeout to allow in-flight requests to complete.
For example, an exporter with `9m30s` stream lifetime:

```
exporters:
  otelarrow:
    timeout: 30s
    arrow:
      max_stream_lifetime: 9m30s
    endpoint: ...
    tls: ...
```

### Receiver metrics

In addition to the standard
[obsreport](https://pkg.go.dev/go.opentelemetry.io/collector/obsreport)
metrics, this component provides network-level measurement instruments
measuring bytes transported.  At the `normal` level of metrics detail:

* `otelcol_receiver_recv`: uncompressed bytes received, prior to compression
* `otelcol_receiver_recv_wire`: compressed bytes received, on the wire.

Arrow's compression performance can be derived by dividing the average
`otelcol_receiver_recv` value by the average `otelcol_receiver_recv_wire` value.

At the `detailed` metrics detail level, information about the stream
of data being returned from the receiver will be instrumented:

* `otelcol_receiver_sent`: uncompressed bytes sent, prior to compression
* `otelcol_receiver_sent_wire`: compressed bytes sent, on the wire.

Metric instruments expose the state of the admission controller at the
`normal` level of detail:

* `otelcol_otelarrow_admission_in_flight_bytes`: number of uncompressed bytes admitted into the pipeline
* `otelcol_otelarrow_admission_waiting_bytes`: number of uncompressed bytes waiting for admission

There several OpenTelemetry Protocol with Apache Arrow-consumer
related metrics available to help diagnose internal performance.
These are disabled at the basic level of detail.  At the normal level,
these metrics are introduced:

* `arrow_batch_records`: Counter of Arrow-IPC records processed
* `arrow_memory_inuse`: UpDownCounter of memory in use by current streams
* `arrow_schema_resets`: Counter of times the schema was adjusted, by data type.

```
service
  ...
  telemetry:
    ...
    metrics:
      ...
      level: detailed
```

## Configuration

### Example Configuration

```yaml theme={null}
protocols:
  grpc:
    # The following entry demonstrates how to specify TLS credentials for the server.
    # Note: These files do not exist. If the receiver is started with this configuration, it will fail.
    tls:
      cert_file: test.crt
      key_file: test.key

    # The following demonstrates how to set maximum limits on stream, message size and connection idle time.
    # Note: The test yaml has demonstrated configuration on a grouped by their structure; however, all of the settings can
    # be mix and matched like adding the maximum connection idle setting in this example.
    max_recv_msg_size_mib: 32
    max_concurrent_streams: 16
    read_buffer_size: 1024
    write_buffer_size: 1024

    # The following entry configures all of the keep alive settings. These settings are used to configure the receiver.
    keepalive:
      server_parameters:
        max_connection_idle: 11s
        max_connection_age: 12s
        max_connection_age_grace: 13s
        time: 30s
        timeout: 5s
      enforcement_policy:
        min_time: 10s
        permit_without_stream: true
  arrow:
    memory_limit_mib: 123
admission:
  request_limit_mib: 80
  waiting_limit_mib: 100
```

***

*Last generated: 2026-07-06*