> ## Documentation Index
> Fetch the complete documentation index at: https://otel.fyi/llms.txt
> Use this file to discover all available pages before exploring further.

# Drain

> OpenTelemetry processor for Drain

# Drain Processor

![Status](https://img.shields.io/badge/status-alpha-red)

**Available in:** `contrib`, `k8s`

**Maintainers:** [@MikeGoldsmith](https://github.com/MikeGoldsmith), [@atoulme](https://github.com/atoulme), [@martinjt](https://github.com/martinjt)

**Source:** [opentelemetry-collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/drainprocessor)

## Supported Telemetry

![Logs](https://img.shields.io/badge/logs-alpha-blue)

## Overview

This processor **annotates**; it does not filter. Use the [filter processor](../filterprocessor/README.md) downstream to act on the `log.record.template` attribute — for example, to drop entire classes of noisy logs by pattern.

## How it works

Drain builds a parse tree from the token structure of log lines. Lines with similar structure are grouped into a **cluster**, and a **template** is derived by replacing variable tokens with `<*>` wildcards. As more logs arrive the templates become more accurate and stable.

Use the template **string** for filtering rules; it converges to the same value across instances given the same configuration and log patterns (see [Deployment considerations](#deployment-considerations)).

## Configuration

```yaml theme={null}
processors:
  drain:
    # Drain parse tree parameters
    tree_depth: 4              # default: 4 (minimum: 3; `depth` in the Drain paper)
    merge_threshold: 0.4       # default: 0.4, range [0.0, 1.0] (`st` in the Drain paper)
    max_node_children: 100     # default: 100 (`maxChild` in the Drain paper)
    max_clusters: 0            # default: 0 (unlimited, LRU eviction when > 0)
    extra_delimiters: []       # default: [] (extra token delimiters beyond whitespace)

    # Body extraction
    body_field: ""             # default: "" (use full body string)

    # Output attribute name
    template_attribute: "log.record.template"    # default

    # Parameter extraction (optional)
    extract_parameters: false                          # default: false
    params_attribute: "log.record.template.params"     # default

    # Seeding (optional)
    seed_templates: []
    seed_logs: []

    # Warmup suppression (optional)
    warmup_min_clusters: 0     # default: 0 (disabled; annotates from the first record)

    # Snapshot persistence (optional)
    storage: ""                # default: "" (disabled; ID of a storage extension)
    save_interval: 0s          # default: 0s (save on shutdown only; e.g. "5m" for periodic saves)
```

### Parameters

| Field                 | Type      | Default                        | Description                                                                                                                                                                                  |
| --------------------- | --------- | ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tree_depth`          | int       | `4`                            | Max depth of the Drain parse tree (`depth` in the Drain paper). Higher values produce more specific templates. Minimum: 3.                                                                   |
| `merge_threshold`     | float     | `0.4`                          | Minimum fraction of tokens that must match an existing cluster template for a log line to be merged into it rather than forming a new cluster (`st` in the Drain paper). Range: \[0.0, 1.0]. |
| `max_node_children`   | int       | `100`                          | Maximum children per internal parse tree node (`maxChild` in the Drain paper). Bounds memory on high-cardinality token positions.                                                            |
| `max_clusters`        | int       | `0`                            | Maximum clusters tracked. When exceeded, the least-recently-used cluster is evicted. `0` means unlimited.                                                                                    |
| `extra_delimiters`    | \[]string | `[]`                           | Additional token delimiters beyond whitespace (e.g. `[",", ":"]`).                                                                                                                           |
| `body_field`          | string    | `""`                           | If set, and the log body is a structured map, the value of this top-level key is used as the text to template instead of the full body.                                                      |
| `template_attribute`  | string    | `"log.record.template"`        | Attribute key written with the derived template string.                                                                                                                                      |
| `extract_parameters`  | bool      | `false`                        | When `true`, the body tokens occupying `<*>` positions of the matched template are written as a string slice attribute (see [Parameter extraction](#parameter-extraction)).                  |
| `params_attribute`    | string    | `"log.record.template.params"` | Attribute key for the extracted parameter slice. Only consulted when `extract_parameters` is `true`.                                                                                         |
| `seed_templates`      | \[]string | `[]`                           | Template strings to pre-load at startup (see [Seeding](#seeding)).                                                                                                                           |
| `seed_logs`           | \[]string | `[]`                           | Raw example log lines to train on at startup (see [Seeding](#seeding)).                                                                                                                      |
| `warmup_min_clusters` | int       | `0`                            | Number of distinct clusters that must be observed before annotation is enabled. `0` disables warmup suppression (see [Warmup suppression](#warmup-suppression)).                             |
| `storage`             | string    | `""`                           | ID of a [storage extension](../../extension/storage/) to use for persisting the Drain tree across restarts (see [Snapshot persistence](#snapshot-persistence)).                              |
| `save_interval`       | duration  | `0s`                           | Interval between periodic snapshot saves. `0s` saves on shutdown only. Requires `storage` to be set.                                                                                         |

## Seeding

Seeding pre-populates the Drain tree before any live logs arrive. This is the primary mechanism for stable templates across restarts.

### `seed_templates`

Provide known template strings directly. The processor trains on each entry at startup, establishing clusters for those patterns immediately.

```yaml theme={null}
processors:
  drain:
    seed_templates:
      - "user <*> logged in from <*>"
      - "connected to <*>"
      - "heartbeat ping <*>"
```

### `seed_logs`

Provide raw example log lines. The processor trains on them at startup, letting Drain derive the templates itself. Useful when exact template strings are not known in advance.

```yaml theme={null}
processors:
  drain:
    seed_logs:
      - "user alice logged in from 10.0.0.1"
      - "user bob logged in from 192.168.1.1"
      - "connected to 10.0.0.1"
```

Empty and whitespace-only entries in both lists are silently skipped.

## Deployment considerations

### Multiple collector instances

Each collector instance builds its Drain parse tree independently in memory. Two instances processing the same log patterns will converge on identical templates because the Drain algorithm is deterministic: given the same configuration and a representative sample of log forms, the same token structure produces the same template string.

The main caveat is the **early training phase**. Before an instance has seen enough lines to abstract a wildcard (e.g. before `"user alice logged in"` and `"user bob logged in"` have both been observed), different instances may temporarily produce different templates for the same logical pattern. This is most noticeable at startup with low-volume or highly variable log streams.

**Mitigations:**

* Use `seed_templates` or `seed_logs` to pre-load known patterns at startup. With a comprehensive seed set, instances start in an already-converged state and live training only fills in the gaps.
* Use `warmup_min_clusters` to suppress annotation until the tree has stabilised, avoiding unstable templates reaching downstream processors.

### Warmup suppression

When `warmup_min_clusters` is set to a value greater than zero, the processor trains on every record from the start but does not write `log.record.template` until that many distinct clusters have been observed. Records pass through immediately — there is no buffering or added latency — they simply arrive at the next processor unannotated during the warmup window.

The warmup window is observable via `otelcol_processor_incoming_items - otelcol_processor_drain_log_records_annotated` — the difference represents records that passed through without a template attribute.

```yaml theme={null}
processors:
  drain:
    warmup_min_clusters: 20
```

Once the threshold is reached, all subsequent records are annotated normally. Records that passed through during warmup are not re-annotated.

## Metrics

The processor emits the following internal telemetry metrics:

| Metric                                          | Type    | Description                                                                                                         |
| ----------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------- |
| `otelcol_processor_drain_clusters_active`       | gauge   | Current number of active clusters in the Drain parse tree. Useful for tracking tree growth and stability over time. |
| `otelcol_processor_drain_log_records_annotated` | counter | Number of log records successfully annotated with a template.                                                       |

## Output attributes

The processor sets the following attributes on each log record:

| Attribute                    | Type      | Example                         | Description                                                                                                                                                                |
| ---------------------------- | --------- | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `log.record.template`        | string    | `"user <*> logged in from <*>"` | The Drain-derived template string. Stable within an instance once the tree has warmed up. Use this for filtering rules.                                                    |
| `log.record.template.params` | \[]string | `["alice", "10.0.0.1"]`         | Optional. Body tokens at `<*>` positions in the template, in order. Only set when `extract_parameters` is `true` and the matched template has at least one `<*>` position. |

The attribute names are configurable via `template_attribute` and `params_attribute`.

> **Semantic conventions**: `log.record.template` aligns with the proposed OTel attribute in [open-telemetry/semantic-conventions#1283](https://github.com/open-telemetry/semantic-conventions/issues/1283) and [#2064](https://github.com/open-telemetry/semantic-conventions/issues/2064). These names may be updated if a convention is formally adopted.

## Parameter extraction

With `extract_parameters: true` the processor writes the body tokens at each `<*>` position of the matched template as a string slice attribute. The values appear in template order, one entry per `<*>`.

```yaml theme={null}
processors:
  drain:
    extract_parameters: true
```

Given the body `"user alice logged in from 10.0.0.1"` and the matched template `"user <*> logged in from <*>"`, the resulting attributes are:

```
log.record.template          = "user <*> logged in from <*>"
log.record.template.params   = ["alice", "10.0.0.1"]
```

Notes:

* Extraction is positional and unnamed. No regex configuration is required.
* Body tokenisation matches the parse tree's: whitespace plus any `extra_delimiters`.
* The parameter slice is only written when the template contains at least one `<*>`. Fully literal templates (single-record clusters before abstraction) emit no slice.
* During warmup suppression neither `log.record.template` nor `log.record.template.params` is written.

## Example pipeline

The following pipeline annotates logs with Drain templates and then drops known noisy patterns using the filter processor:

```yaml theme={null}
processors:
  drain:
    tree_depth: 4
    merge_threshold: 0.4
    max_clusters: 500
    seed_templates:
      - "user <*> logged in from <*>"
      - "connected to <*>"
      - "heartbeat ping <*>"
    warmup_min_clusters: 20

  filter/drop_noisy:
    error_mode: ignore
    logs:
      log_record:
        - attributes["log.record.template"] == "heartbeat ping <*>"
        - attributes["log.record.template"] == "connected to <*>"

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [drain, filter/drop_noisy]
      exporters: [otlp]
```

## `body_field`

`body_field` is a convenience for pipelines where the log body is a structured map and you do not have full control over how upstream processors shape it.

If you **do** control the pipeline, the preferred approach is a `move` operator in the filelog receiver (or equivalent) to promote the message field back to a plain string body before the drain processor sees the record:

```yaml theme={null}
operators:
  - type: json_parser
  - type: move
    from: body.message
    to: body
```

If you **cannot** do that — for example, logs arrive via OTLP already structured — set `body_field` to the map key whose value should be fed to Drain:

```yaml theme={null}
processors:
  drain:
    body_field: "message"
```

Given a log body `{"level": "info", "message": "user alice logged in from 10.0.0.1"}`, only the `message` value is fed to Drain. The full body is used unchanged if the field is absent or the body is not a map.

> **Note**: `body_field` only supports a single top-level key. Full OTTL path expressions (e.g. `body["event"]["message"]`) are not supported and are noted as a future extension.

## Snapshot persistence

When `storage` is set to the ID of a [storage extension](../../extension/storage/), the processor saves and restores the Drain tree state across restarts. This eliminates the need for seeding and warmup in most deployments.

```yaml theme={null}
extensions:
  file_storage:
    directory: /var/lib/otelcol/drain

processors:
  drain:
    storage: file_storage
    save_interval: 5m
```

### Lifecycle

* **Startup**: the processor attempts to load a snapshot from storage. If a valid snapshot is found, `seed_templates` and `seed_logs` are skipped (they are already incorporated in the snapshot). If the loaded tree has enough clusters to satisfy `warmup_min_clusters`, warmup is also skipped.
* **Runtime**: if `save_interval` is set, the tree is snapshotted periodically. A hash check avoids redundant writes when the tree has not changed.
* **Shutdown**: a final snapshot is always saved.

If the snapshot is corrupt or cannot be loaded, the processor logs a warning and falls back to seeding as if no snapshot existed.

### Shared storage for scaled deployments

With a shared storage backend (Redis, database), periodic saves allow new instances joining a scaled deployment to load a tree that was trained by already-running instances. This avoids the cold-start problem where newly added collectors must rebuild the tree from scratch.

Multiple instances writing to the same storage key is safe — all instances see similar log traffic and converge on similar trees, so any snapshot is useful as a starting point. The hash check reduces redundant writes across instances.

To clear the snapshot and force a fresh start (e.g. after changing Drain parameters), delete the storage data for the processor.

## Future extensions

* **OTTL body extraction**: support full OTTL path expressions for `body_field` instead of a single top-level key name.
* **Multi-instance tree merging**: gossip-based tree merging for consistent templates across horizontally scaled deployments.

***

*Last generated: 2026-07-20*