ClickHouse Exporter

Available in: contrib Maintainers: @hanjm, @Frapschen, @SpencerTorres Source: opentelemetry-collector-contrib

Supported Telemetry

Overview

This exporter supports sending OpenTelemetry data to ClickHouse.

ClickHouse is an open-source, high performance columnar OLAP database management system for real-time analytics using SQL. Throughput can be measured in rows per second or megabytes per second. If the data is placed in the page cache, a query that is not too complex is processed on modern hardware at a speed of approximately 2-10 GB/s of uncompressed data on a single server. If 10 bytes of columns are extracted, the speed is expected to be around 100-200 million rows per second.

Note: Batching Recommendation For optimal performance, ClickHouse recommends inserting data in large batches:

We recommend inserting data in packets of at least 5000 rows, or no more than a single request per second. When inserting to a MergeTree table from a tab-separated dump, the insertion speed can be from 50 to 200 MB/s.

To achieve this natively, enable batching within the exporter’s sending_queue configuration. You do not need to add the external batch processor to your collector pipeline. Relying on the exporter’s internal batching is the recommended approach to avoid data-loss issues associated with the external processor. Enable it by adding a batch block inside sending_queue:

exporters:
  clickhouse:
    endpoint: tcp://127.0.0.1:9000
    sending_queue:
      # num_consumers controls how many batches are inserted into ClickHouse
      # concurrently.
      num_consumers: 10
      batch:
        min_size: 5000      # rows per INSERT (items sizer); tune to your workload
        flush_timeout: 5s   # flush a partial batch after this delay

If you are migrating from a pipeline that uses the standalone batch processor, remove batch from the pipeline’s processors list and configure sending_queue.batch instead. For durability across restarts, also set sending_queue.storage to a storage extension so queued batches survive a crash (at-least-once delivery).

Visualization Tools

Official ClickHouse Plugin for Grafana

The official ClickHouse Datasource for Grafana contains features that integrate directly with this exporter. You can view associated logs and traces, as well as visualize other queries such as tables and time series graphs. Learn how to configure the OpenTelemetry integration.

Altinity’s ClickHouse Plugin for Grafana

If the official plugin doesn’t meet your needs, you can try the Altinity plugin for ClickHouse, which also supports a wide range of features.

Logs

Get log severity count time series.

SELECT toDateTime(toStartOfInterval(Timestamp, INTERVAL 60 second)) as time, SeverityText, count() as count
FROM otel_logs
WHERE toStartOfFiveMinutes(Timestamp) >= toStartOfFiveMinutes(NOW() - INTERVAL 1 HOUR)
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
GROUP BY SeverityText, time
ORDER BY time;

The default logs table is ordered by (toStartOfFiveMinutes(Timestamp), ServiceName, Timestamp). For time range queries, filter on both toStartOfFiveMinutes(Timestamp) and Timestamp, and order by the tuple (toStartOfFiveMinutes(Timestamp), Timestamp) to use the primary key’s read-in-order optimization. Apply toStartOfFiveMinutes to the range bound as well (e.g. toStartOfFiveMinutes(NOW() - INTERVAL 1 HOUR)) so the time bucket bounds are not truncated off the scan.

Find any log.

SELECT Timestamp as log_time, Body
FROM otel_logs
WHERE toStartOfFiveMinutes(Timestamp) >= toStartOfFiveMinutes(NOW() - INTERVAL 1 HOUR)
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
ORDER BY (toStartOfFiveMinutes(Timestamp), Timestamp) DESC
LIMIT 100;

Find log with specific service.

SELECT Timestamp as log_time, Body
FROM otel_logs
WHERE ServiceName = 'clickhouse-exporter'
  AND toStartOfFiveMinutes(Timestamp) >= toStartOfFiveMinutes(NOW() - INTERVAL 1 HOUR)
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
ORDER BY (toStartOfFiveMinutes(Timestamp), Timestamp) DESC
LIMIT 100;

Find log with specific attribute.

SELECT Timestamp as log_time, Body
FROM otel_logs
WHERE LogAttributes['container_name'] = '/example_flog_1'
  AND toStartOfFiveMinutes(Timestamp) >= toStartOfFiveMinutes(NOW() - INTERVAL 1 HOUR)
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
ORDER BY (toStartOfFiveMinutes(Timestamp), Timestamp) DESC
LIMIT 100;

Find log with body contain string token.

SELECT Timestamp as log_time, Body
FROM otel_logs
WHERE hasToken(Body, 'http')
  AND toStartOfFiveMinutes(Timestamp) >= toStartOfFiveMinutes(NOW() - INTERVAL 1 HOUR)
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
ORDER BY (toStartOfFiveMinutes(Timestamp), Timestamp) DESC
LIMIT 100;

Find log with body contain string.

SELECT Timestamp as log_time, Body
FROM otel_logs
WHERE Body like '%http%'
  AND toStartOfFiveMinutes(Timestamp) >= toStartOfFiveMinutes(NOW() - INTERVAL 1 HOUR)
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
ORDER BY (toStartOfFiveMinutes(Timestamp), Timestamp) DESC
LIMIT 100;

Find log with body regexp match string.

SELECT Timestamp as log_time, Body
FROM otel_logs
WHERE match(Body, 'http')
  AND toStartOfFiveMinutes(Timestamp) >= toStartOfFiveMinutes(NOW() - INTERVAL 1 HOUR)
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
ORDER BY (toStartOfFiveMinutes(Timestamp), Timestamp) DESC
LIMIT 100;

Find log with body json extract.

SELECT Timestamp as log_time, Body
FROM otel_logs
WHERE JSONExtractFloat(Body, 'bytes') > 1000
  AND toStartOfFiveMinutes(Timestamp) >= toStartOfFiveMinutes(NOW() - INTERVAL 1 HOUR)
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
ORDER BY (toStartOfFiveMinutes(Timestamp), Timestamp) DESC
LIMIT 100;

Traces

Find spans with specific attribute.

SELECT Timestamp,
       TraceId,
       SpanId,
       ParentSpanId,
       SpanName,
       SpanKind,
       ServiceName,
       Duration,
       StatusCode,
       StatusMessage,
       toString(SpanAttributes),
       toString(ResourceAttributes),
       toString(Events.Name),
       toString(Links.TraceId)
FROM otel_traces
WHERE ServiceName = 'clickhouse-exporter'
  AND SpanAttributes['peer.service'] = 'telemetrygen-server'
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
Limit 100;

Find traces with traceID (using time primary index and TraceID skip index).

WITH
    '391dae938234560b16bb63f51501cb6f' as trace_id,
    (SELECT min(Start) FROM otel_traces_trace_id_ts WHERE TraceId = trace_id) as start,
    (SELECT max(End) + 1 FROM otel_traces_trace_id_ts WHERE TraceId = trace_id) as end
SELECT Timestamp,
       TraceId,
       SpanId,
       ParentSpanId,
       SpanName,
       SpanKind,
       ServiceName,
       Duration,
       StatusCode,
       StatusMessage,
       toString(SpanAttributes),
       toString(ResourceAttributes),
       toString(Events.Name),
       toString(Links.TraceId)
FROM otel_traces
WHERE TraceId = trace_id
  AND Timestamp >= start
  AND Timestamp <= end
Limit 100;

Find spans is error.

SELECT Timestamp,
       TraceId,
       SpanId,
       ParentSpanId,
       SpanName,
       SpanKind,
       ServiceName,
       Duration,
       StatusCode,
       StatusMessage,
       toString(SpanAttributes),
       toString(ResourceAttributes),
       toString(Events.Name),
       toString(Links.TraceId)
FROM otel_traces
WHERE ServiceName = 'clickhouse-exporter'
  AND StatusCode = 'Error'
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
Limit 100;

Find slow spans.

SELECT Timestamp,
       TraceId,
       SpanId,
       ParentSpanId,
       SpanName,
       SpanKind,
       ServiceName,
       Duration,
       StatusCode,
       StatusMessage,
       toString(SpanAttributes),
       toString(ResourceAttributes),
       toString(Events.Name),
       toString(Links.TraceId)
FROM otel_traces
WHERE ServiceName = 'clickhouse-exporter'
  AND Duration > 1 * 1e9
  AND Timestamp >= NOW() - INTERVAL 1 HOUR
Limit 100;

Metrics

Metrics data is stored in different clickhouse tables depending on their types. The tables will have a suffix to distinguish which type of metrics data is stored.

Metrics Type	Metrics Table
sum	_sum
gauge	_gauge
histogram	_histogram
exponential histogram	_exponential_histogram
summary	_summary

Before you make a metrics query, you need to know the type of metric you wish to use. If your metrics come from Prometheus(or someone else uses OpenMetrics protocol), you also need to know the compatibility between Prometheus(OpenMetrics) and OTLP Metrics.

Find a sum metrics with name

select TimeUnix,MetricName,Attributes,Value from otel_metrics_sum
where MetricName='calls' limit 100

Find a sum metrics with name, attribute.

select TimeUnix,MetricName,Attributes,Value from otel_metrics_sum
where MetricName='calls' and Attributes['service_name']='featureflagservice'
limit 100

The OTLP Metrics define two type value for one datapoint, clickhouse only use one value of float64 to store them.

Profiles

[!IMPORTANT] Profiles support is at development stability. The OpenTelemetry profiling signal itself is pre-GA (the OTLP profiles protocol is in v1development), so the schema and behavior may change in a backwards-incompatible way. To send profiles through a collector pipeline you must enable the service.profilesSupport feature gate (--feature-gates=+service.profilesSupport). The profiles table requires ClickHouse 26.2 or newer. It always uses text (full-text-search) indexes and does not fall back to bloom_filter on older server versions. If you manage the schema yourself (create_schema: false), you can adapt the DDL for an older version.

Profiles are stored as one denormalized row per OTLP Sample. The interned ProfilesDictionary (strings, functions, locations, mappings, links, attributes) is resolved at write time so each row is self-contained and can be queried without joins.

Performance Guide

A single ClickHouse instance with 32 CPU cores and 128 GB RAM can handle around 20 TB (20 Billion) logs per day, the data compression ratio is 7 ~ 11, the compressed data store in disk is 1.8 TB ~ 2.85 TB, add more clickhouse node to cluster can increase linearly. The otel-collector with otlp receiver/clickhouse tcp exporter (with sending_queue batching enabled) can process around 40k/s logs entry per CPU cores, add more collector node can increase linearly.

Reading a shared table by a resource attribute

The default schemas order primarily by time and ServiceName, and store resource-level labels (for example a tenant or namespace attribute) in the ResourceAttributes map rather than in the sort key. A skip index on the map values lets a query that filters on a single attribute value prune granules, but because every value shares the same time-partitioned parts, a wide time-range read filtered to one value still scans granules that also contain other values’ rows — the filter is applied after the granule is read. For a low-cardinality attribute this is negligible. If you query a shared table by a high-cardinality attribute (for example one tenant out of hundreds or thousands, each reading concurrently), that shared-parts scan can become the read bottleneck. In that case, set create_schema: false and manage the DDL yourself so the attribute participates in the primary key — for example by adding it early in ORDER BY or via a projection — accepting that this diverges from the default schema.

Configuration options

The following settings are required:

endpoint (no default): The ClickHouse server address, support multi host with port, for example:
- tcp protocol tcp://addr1:port,tcp://addr2:port or TLS tcp://addr1:port,addr2:port?secure=true
- http protocol http://addr1:port,addr2:port or https https://addr1:port,addr2:port
- clickhouse protocol clickhouse://addr1:port,addr2:port or TLS clickhouse://addr1:port,addr2:port?secure=true
When multiple endpoints are provided, the driver handles load balancing and automatic failover. By default, it uses in_order strategy (tries endpoints in the order specified). Alternatively, use connection_open_strategy=round_robin (distributes connections evenly) or connection_open_strategy=random (randomly selects endpoints) in connection_params. See connection_open_strategy documentation.

Many other ClickHouse specific options can be configured through query parameters e.g. addr?dial_timeout=5s&compress=lz4. For a full list of options see the ClickHouse driver documentation Connection options:

username (default = ): The authentication username.
password (default = ): The authentication password.
ttl (default = 0): The data time-to-live example 30m, 48h. Also, 0 means no ttl.
database (default = default): The database name. Overrides the database defined in endpoint when this setting is not equal to default.
connection_params (default = {}). Extra connection parameters with map format. Query parameters provided in endpoint will be individually overwritten if present in this map. Parameters can be either driver parameters (e.g., connection_open_strategy, max_open_conns) that control client-side behavior, or ClickHouse session settings (e.g., max_execution_time) that are passed to the server. See the driver parameters list for recognized driver options; all others are treated as session settings.
create_schema (default = true): When set to true, will run DDL to create the database and tables. (See schema management)
compress (default = lz4): Controls the compression algorithm. Valid options: none (disabled), zstd, lz4 (default), gzip, deflate, br, true (lz4). Ignored if compress is set in the endpoint or connection_params.
async_insert (default = true): Enables async inserts. Ignored if async inserts are configured in the endpoint or connection_params. Async inserts may still be overridden server-side.
tls Advanced TLS configuration (See TLS).

Additional DSN features: The underlying clickhouse-go module offers additional configuration. These can be set in the exporter’s endpoint or connection_params config values.

client_info_product Must be in productName/version format with comma separated entries. By default the exporter will append its binary build information. You can use this information to track the origin of INSERT statements in the system.query_log table.

ClickHouse tables:

logs_table_name (default = otel_logs): The table name for logs.
traces_table_name (default = otel_traces): The table name for traces.
profiles_table_name (default = otel_profiles): The table name for profiles.
metrics_tables
- gauge
  - name (default = “otel_metrics_gauge”)
- sum
  - name (default = “otel_metrics_sum”)
- summary
  - name (default = “otel_metrics_summary”)
- histogram
  - name (default = “otel_metrics_histogram”)
- exponential_histogram
  - name (default = “otel_metrics_exp_histogram”)

Cluster definition:

cluster_name (default = ): Optional. If present, will include ON CLUSTER cluster_name when creating tables.

Table engine:

table_engine
- name (default = MergeTree)
- params (default = )

Modifies ENGINE definition when table is created. If not set then ENGINE defaults to MergeTree(). Can be combined with cluster_name to enable replication for fault tolerance. Processing:

timeout (default = 5s): The timeout for every attempt to send data to the backend.
sending_queue
- enabled (default = true)
- num_consumers (default = 10): Number of concurrent consumers that dequeue and insert data into ClickHouse. Enabling batch does not reduce this parallelism, only the (cheap) queue reader becomes single-threaded, while inserts still run on up to num_consumers workers. Ignored if enabled is false.
- queue_size (default = 1000): Maximum size of the queue, measured in sizer units. Data is dropped when the queue is full unless block_on_overflow is enabled.
- sizer (default = requests): How queue_size is measured. One of requests, items, or bytes.
- block_on_overflow (default = false): If true, waits for space when the queue is full instead of dropping data (applies backpressure to the pipeline).
- storage (default = none): Name of a storage extension for a persistent, crash-safe queue. When unset, the queue is in-memory and is lost on restart.
- batch (disabled by default): Batches data inside the sending queue. Add an empty batch: {} to enable it with the defaults below. This replaces the standalone batch processor; see the batching recommendation near the top of this document.
  - flush_timeout (default = 200ms): Time after which a batch is sent regardless of size.
  - min_size (default = 8192): Minimum batch size before it is sent, in batch.sizer units.
  - max_size (default = 0): Maximum batch size; 0 means no limit. When set, larger batches are split, and it must be >= min_size.
  - sizer (default = items): How batch size is measured. One of items or bytes (not requests). If unset, inherits sending_queue.sizer.
retry_on_failure
- enabled (default = true)
- initial_interval (default = 5s): The Time to wait after the first failure before retrying; ignored if enabled is false
- max_interval (default = 30s): The upper bound on backoff; ignored if enabled is false
- max_elapsed_time (default = 300s): The maximum amount of time spent trying to send a batch; ignored if enabled is false

TLS

The exporter supports TLS. To enable TLS, you must specify the secure=true query parameter in the endpoint URL or use the https scheme. You may also use certificate authentication with the tls setting:

exporters:
  clickhouse:
    endpoint: . . .
    tls:
      insecure: false
      insecure_skip_verify: false
      ca_file: CAroot.crt
      cert_file: client.crt
      key_file: client.key

The available tls options are inherited from OpenTelemetry’s TLS config structure, more options are available than shown in this example.

Schema management

By default, the exporter will create the database and tables under the names defined in the config. This is fine for simple deployments, but for production workloads, it is recommended that you manage your own schema by setting create_schema to false in the config. This prevents each exporter process from racing to create the database and tables, and makes it easier to upgrade the exporter in the future. In this mode, the only SQL sent to your server will be for INSERT statements. The default DDL used by the exporter can be found in internal/sqltemplates. Be sure to customize the indexes, TTL, and partitioning to fit your deployment. Column names and types must be the same to preserve compatibility with the exporter’s INSERT statements. As long as the column names/types match the INSERT statement, you can create whatever kind of table you want. See ClickHouse’s LogHouse as an example of this flexibility.

Upgrading existing tables

Sometimes new columns are added to the exporter in a backwards compatible way. The exporter runs a DESC TABLE command on startup to determine which of these new columns are available on the table schema. If you already have tables created by a previous version of the exporter, you will need to add these new columns manually. Here is an example of a command you can use to update your existing table (adjust database and table names as needed):

ALTER TABLE otel.otel_logs ADD COLUMN IF NOT EXISTS EventName String CODEC(ZSTD(1));

To find the newest columns available check the internal/sqltemplates folder. The CREATE TABLE statements will always have the latest columns. In some cases the table changes will not be backwards compatible. Be sure to check the changelog for breaking changes before upgrading your collector.

Optional table upgrades

As mentioned in the previous section, the exporter is able to detect which columns are present on the schema for backwards compatibility. Here are some columns you can add to your table to update the schema:

-- EventName
ALTER TABLE otel.otel_logs ADD COLUMN IF NOT EXISTS EventName String CODEC(ZSTD(1));

-- JSON tables only. These 3 columns are part of one feature, you must add all 3 at once.
ALTER TABLE otel.otel_logs
  ADD COLUMN IF NOT EXISTS ResourceAttributesKeys Array(LowCardinality(String)) CODEC(ZSTD(1)),
  ADD COLUMN IF NOT EXISTS ScopeAttributesKeys Array(LowCardinality(String)) CODEC(ZSTD(1)),
  ADD COLUMN IF NOT EXISTS LogAttributesKeys Array(LowCardinality(String)) CODEC(ZSTD(1)),
  -- Optional indices
  ADD INDEX IF NOT EXISTS idx_res_attr_keys ResourceAttributesKeys TYPE bloom_filter(0.01) GRANULARITY 1,
  ADD INDEX IF NOT EXISTS idx_scope_attr_keys ScopeAttributesKeys TYPE bloom_filter(0.01) GRANULARITY 1,
  ADD INDEX IF NOT EXISTS idx_log_attr_keys LogAttributesKeys TYPE bloom_filter(0.01) GRANULARITY 1;

-- JSON tables only. These 2 columns are part of one feature, you must add all 2 at once.
ALTER TABLE otel.otel_traces
  ADD COLUMN IF NOT EXISTS ResourceAttributesKeys Array(LowCardinality(String)) CODEC(ZSTD(1)),
  ADD COLUMN IF NOT EXISTS SpanAttributesKeys  Array(LowCardinality(String)) CODEC(ZSTD(1)),
  -- Optional indices
  ADD INDEX IF NOT EXISTS idx_res_attr_keys ResourceAttributesKeys TYPE bloom_filter(0.01) GRANULARITY 1,
  ADD INDEX IF NOT EXISTS idx_span_attr_keys SpanAttributesKeys TYPE bloom_filter(0.01) GRANULARITY 1;

Example Config

This example shows how to configure the exporter to send data to a ClickHouse server. It uses the native protocol without TLS. The exporter will create the database and tables if they don’t exist. The data is stored for 72 hours (3 days).

receivers:
  examplereceiver:
exporters:
  clickhouse:
    endpoint: tcp://127.0.0.1:9000?dial_timeout=10s
    database: otel
    async_insert: true
    ttl: 72h
    compress: lz4
    create_schema: true
    logs_table_name: otel_logs
    traces_table_name: otel_traces
    timeout: 5s
    metrics_tables:
      gauge: 
        name: "otel_metrics_gauge"
      sum: 
        name: "otel_metrics_sum"
      summary: 
        name: "otel_metrics_summary"
      histogram: 
        name: "otel_metrics_histogram"
      exponential_histogram: 
        name: "otel_metrics_exp_histogram"
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s
    # cluster_name: my_cluster
    # table_engine:
    #   name: ReplicatedMergeTree
    #   params:
service:
  pipelines:
    logs:
      receivers: [ examplereceiver ]
      exporters: [ clickhouse ]

Experimental JSON support

JSON column types can be enabled per-exporter using the json config option:

exporters:
  clickhouse:
    endpoint: clickhouse://localhost:9000?enable_json_type=1
    json: true

Previously, the clickhouse.json feature gate was used to enable JSON for all ClickHouse exporter instances. This feature gate is now deprecated. Use the json config option instead, which allows per-pipeline control. You may also need to add enable_json_type=1 to your endpoint or connection_params. DDL has been updated, but feel free to tune the schema as needed. DDL can be found in the internal/sqltemplates package. All Map columns have been replaced with JSON. ClickHouse v25+ is recommended for reliable JSON support.

Contributing

Before contributing, review the contribution guidelines in CONTRIBUTING.md.

Integration tests

Integration tests can be run with the following command (includes unit tests):

go test -tags integration

Note: Make sure integration tests pass after making changes to SQL.

Configuration

Example Configuration

clickhouse:
  endpoint: clickhouse://127.0.0.1:9000
clickhouse/full:
  endpoint: clickhouse://127.0.0.1:9000
  username: foo
  password: bar
  database: otel
  ttl: 72h
  logs_table_name: otel_logs
  traces_table_name: otel_traces
  timeout: 5s
  tls:
    cert_file: client.crt
    key_file: client.key
  retry_on_failure:
    enabled: true
    initial_interval: 5s
    max_interval: 30s
    max_elapsed_time: 300s
  sending_queue:
    queue_size: 100
    storage: file_storage/clickhouse
  metrics_tables:
    gauge: 
      name: "otel_metrics_custom_gauge"
    sum: 
      name: "otel_metrics_custom_sum"
    summary: 
      name: "otel_metrics_custom_summary"
    histogram: 
      name: "otel_metrics_custom_histogram"
    exponential_histogram: 
      name: "otel_metrics_custom_exp_histogram"
clickhouse/batch:
  endpoint: clickhouse://127.0.0.1:9000
  sending_queue:
    batch:
      min_size: 5000
      max_size: 10000
      flush_timeout: 5s
clickhouse/json:
  endpoint: clickhouse://127.0.0.1:9000
  json: true
clickhouse/invalid-endpoint:
  endpoint: 127.0.0.1:9000

clickhouse/table-engine-empty:
  endpoint: clickhouse://127.0.0.1:9000
clickhouse/table-engine-name-only:
  endpoint: clickhouse://127.0.0.1:9000
  table_engine:
    name: ReplicatedReplacingMergeTree
clickhouse/table-engine-full:
  endpoint: clickhouse://127.0.0.1:9000
  table_engine:
    name: ReplicatedReplacingMergeTree
    params: "'/clickhouse/tables/{shard}/table_name', '{replica}', ver"
clickhouse/table-engine-params-only:
  endpoint: clickhouse://127.0.0.1:9000
  table_engine:
    params: "whatever"

Last generated: 2026-08-24

​ClickHouse Exporter

​Supported Telemetry

​Overview

​Visualization Tools

​Official ClickHouse Plugin for Grafana

​Altinity’s ClickHouse Plugin for Grafana

​Logs

​Traces

​Metrics

​Profiles

​Performance Guide

​Reading a shared table by a resource attribute

​Configuration options

​TLS

​Schema management

​Upgrading existing tables

​Optional table upgrades

​Example Config

​Experimental JSON support

​Contributing

​Integration tests

​Configuration

​Example Configuration