Filter Processor

Available in: core, contrib, k8s Maintainers: @TylerHelmuth, @evan-bradley, @edmocosta, @bogdandrutu Source: opentelemetry-collector-contrib

Supported Telemetry

Overview

[!NOTE] This documentation applies only to version 0.146.0 and later. Configuration from previous version is still supported, but no longer documented in this README. For information on earlier versions, please refer to the previous documentation.

Configuration

The Filter Processor utilizes the OpenTelemetry Transformation Language to create conditions that determine when telemetry should be dropped. If any condition is met, the telemetry is dropped (each condition is ORed together).

General Config

filter:
  error_mode: ignore
  <trace|metric|log|profile>_conditions: []

The Filter Processor’s primary configuration is broken down by signal (traces, metrics, logs, and profiles) and allows you to configure a list of conditions for the processor to evaluate. The list can be made of:

OTTL conditions. This option will meet most user’s needs. See Basic Config for more details.
Objects, which allows users to apply configuration options to a specific list of conditions. See Advanced Config for more details.

The OTTL allows the use of and, or, and () in conditions. See OTTL Boolean Expressions for more details.

Context

Within each <signal>_conditions list, only certain OTTL Contexts can be used. Each context provides access to different telemetry fields. Click the context name for detailed documentation.

Signal	Available Contexts
trace_conditions	resource, scope, span, and spanevent
metric_conditions	resource, scope, metric, and datapoint
log_conditions	resource, scope, and log
profile_conditions	resource, scope, and profile

Telemetry is evaluated hierarchically, from higher to lower levels. The hierarchy may vary by signal type; examples include: Logs: resource → scope → log Metrics: resource → scope → metric → datapoint Traces: resource → scope → span → spanevent For conditions that apply to the same signal, such as spans and span events, if the “higher” level telemetry matches a condition and is dropped, the “lower” level condition will not be checked. This means that if a span is dropped but a span event condition was defined, the span event condition will not be checked for that span. The same relationship applies to other signals. If all span events for a span are dropped, the span will be left intact. If all datapoints for a metric are dropped, the metric will also be dropped. Note that when a single condition contains paths from different contexts, resource and spanevent for example, the condition is evaluated in the lower context spanevent.

filter:
  trace_conditions:
    - resource.attributes["host.name"] == "localhost" or spanevent.name == "grpc.timeout"

This condition translates to: For each span event, check whether its parent span’s resource attribute host.name is localhost or whether the span event’s name is grpc.timeout. Drop the span event if either condition is true.

Error Modes

The filter processor also allows configuring an optional field, error_mode, which will determine how the processor reacts to errors that occur while processing an OTTL condition. ignore is the default mode.

error_mode	description
ignore	The processor ignores errors returned by conditions, logs them, and continues on to the next condition. This is the recommended mode.
silent	The processor ignores errors returned by conditions, does not log them, and continues on to the next condition.
propagate	The processor returns the error up the pipeline. This will result in the payload being dropped from the collector.

Feature Gate

`processor.filter.defaultErrorModeIgnore`

The processor.filter.defaultErrorModeIgnore feature gate changes the default error_mode of the filter processor from propagate to ignore. This gate is currently in stable (always enabled). The default error_mode is ignore and can no longer be reverted via this gate. The gate will be removed in v0.159.0.

Basic Config

The basic configuration style allows you to configure OTTL conditions as a flat, OR-ed list, without worrying about extra configurations. This is the simplest way to configure the Filter Processor. If you need explicit context specification or specific error modes, see Advanced Config. Format:

filter:
  error_mode: propagate
  <trace|metric|log|profile>_conditions:
    - string
    - string
    - string

Example:

processors:
  filter:
    error_mode: propagate
    trace_conditions:
      - span.attributes["container.name"] == "app_container_1"
      - resource.attributes["host.name"] == "localhost"
      - span.name == "app_3"
    metric_conditions:
      - metric.name == "my.metric" and resource.attributes["my_label"] == "abc123"
      - metric.type == METRIC_DATA_TYPE_HISTOGRAM
      - resource.attributes["service.name"] == "my_service_name"
    log_conditions:
      - IsMatch(log.body, ".*password.*")
      - log.severity_number < SEVERITY_NUMBER_WARN
    profile_conditions:
      - profile.duration_unix_nano > 3000

Advanced Config

For more complex use cases you may need to use the Filter Processor’s advanced configuration style to explicitly specify the context and apply additional settings to a group of conditions. Format:

filter:
  error_mode: ignore
  <trace|metric|log|profile>_conditions:
    - context: string
      error_mode: propagate
      conditions:
        - string
        - string
    - context: string
      conditions:
        - string

context: specifies the OTTL context for the conditions. See the Context table for valid values per signal type. Note: In most cases, you should not set this field manually. The processor’s context inferrer will automatically determine the correct context based on the paths used in your conditions. Only set this field if you have a specific need to set the context. error_mode: allows overriding the top-level error_mode for this specific group of conditions. Valid values are ignore, silent, and propagate. See the Error Modes table for more details. conditions: a list of OTTL conditions. If any condition is met, the telemetry is dropped (conditions are OR-ed together). Example:

processors:
  filter:
    error_mode: propagate
    trace_conditions:
      - conditions:
          - resource.attributes["host.name"] == "localhost"
      - error_mode: ignore
        conditions:
          - span.attributes["container.name"] == "container_1"
          - span.name == "app_3"
      - conditions:
          - spanevent.attributes["grpc"] == true
          - IsMatch(spanevent.name, ".*grpc.*")
    metric_conditions:
      - conditions:
          - metric.name == "my.metric" and resource.attributes["my_label"] == "abc123"
          - metric.type == METRIC_DATA_TYPE_HISTOGRAM
      - conditions:
          - metric.type == METRIC_DATA_TYPE_SUMMARY
    log_conditions:
      - conditions:
          - IsMatch(log.body, ".*password.*")
          - log.severity_number < SEVERITY_NUMBER_WARN

Context Inference

[!NOTE] This is an advanced topic and is not necessary to get started using the Filter Processor. Read on if you’re interested in how the Filter Processor parses your OTTL conditions.

An OTTL Context defines which Paths, functions, and enums are available when parsing the condition. The Filter Processor automatically infers the OTTL Context based on the paths defined in a condition. This inference is based on the Path names, functions, and enums present in the condition. The inference happens automatically because Path names are prefixed with the context name. For example:

trace_conditions:
  - span.attributes["container.name"] == "container_1" and spanevent.attributes["grpc"] == true

In this configuration, the inferred context value is spanevent, as it is the only context that supports parsing both spanevent and span Paths. In Filter processor, each condition in a list is parsed individually and grouped by its inferred context. Conditions are executed by context group in hierarchical order (resource → scope → signal-specific). For example:

trace_conditions:
  - scope.name == "my.scope"                          # inferred as scope context
  - span.name == "app_3"                              # inferred as span context
  - spanevent.name == "grpc.timeout"                  # inferred as spanevent context
  - resource.attributes["host.name"] == "localhost"   # inferred as resource context

Resource condition is executed first, followed by scope, span, and finally spanevent conditions. The primary benefit of context inference is that it enhances the efficiency of data filtering by linking them to the most suitable context. This optimization ensures that the processing are both accurate and performant, leveraging the hierarchical structure of contexts to avoid unnecessary iterations and improve overall processing efficiency. All of this happens automatically, leaving you to write OTTL conditions without worrying about context. In some situations a combination of Paths, functions, or enums is not allowed. For example:

trace_conditions:
  - IsRootSpan() or spanevent.name == "bar"

This condition fails because IsRootSpan() is only available in the span context, not span events, while the spanevent Path prefix requires the condition to evaluate in the span event context. The solution is to split into separate span and span event conditions:

trace_conditions:
  - context: span
    conditions:
      - IsRootSpan()
  - conditions:
      - spanevent.name == "bar"

The advanced configuration is required for IsRootSpan() because there is no Path prefix in the condition, so the context must be set explicitly to span.

Examples

Dropping data based on a resource attribute

processors:
  filter:
    error_mode: ignore
    trace_conditions:
      - resource.attributes["k8s.pod.name"] == "my-pod-name"

Dropping metrics with invalid type

processors:
  filter:
    error_mode: ignore
    metric_conditions:
      - metric.type == METRIC_DATA_TYPE_NONE

Dropping specific metric and value

processors:
  filter:
    error_mode: ignore
    metric_conditions:
      - metric.name == "k8s.pod.phase" and datapoint.value_int == 4

Dropping non-HTTP spans

processors:
  filter:
    error_mode: ignore
    trace_conditions:
      - span.attributes["http.request.method"] == nil

Dropping HTTP spans

processors:
  filter:
    error_mode: ignore
    trace_conditions:
      - span.attributes["http.request.method"] != nil

Dropping non-error spans with a duration of less than 1 second

processors:
  filter:
    error_mode: ignore
    trace_conditions:
      - (span.end_time - span.start_time) < Duration("1s") and span.status.code != STATUS_CODE_ERROR

OTTL Functions

The filter processor has access to all OTTL Converter functions In addition, the processor defines a few of its own functions: Metrics only functions

HasAttrKeyOnDatapoint
HasAttrOnDatapoint

HasAttrKeyOnDatapoint

HasAttrKeyOnDatapoint(key) Returns true if the given key appears in the attribute map of any datapoint on a metric. key must be a string. You must use the metrics.metric context. Examples:

HasAttrKeyOnDatapoint("http.method")

filter/keep_good_metrics:
  error_mode: ignore
  metrics:
    metric:
      - HasAttrKeyOnDatapoint("bad.metric")

HasAttrOnDatapoint

HasAttrOnDatapoint(key, value) Returns true if the given key and value appears in the attribute map of any datapoint on a metric. key and value must both be strings. If the value of the attribute on the datapoint is not a string, value will be compared to "". You must use the metrics.metric context. Examples:

HasAttrOnDatapoint("http.method", "GET")

# Drops metrics containing the 'bad.metric' attribute key and 'true' value
filter/keep_good_metrics:
  error_mode: ignore
  metrics:
    metric:
      - HasAttrOnDatapoint("bad.metric", "true")

Legacy Configuration

The following configuration options are deprecated and will be removed in a future release. Please migrate to the new *_conditions format.

Deprecated Config	Migrate To
`traces.resource`	`trace_conditions` with `resource.` prefix
`traces.span`	`trace_conditions` with `span.` prefix
`traces.spanevent`	`trace_conditions` with `spanevent.` prefix
`metrics.resource`	`metric_conditions` with `resource.` prefix
`metrics.metric`	`metric_conditions` with `metric.` prefix
`metrics.datapoint`	`metric_conditions` with `datapoint.` prefix
`logs.resource`	`log_conditions` with `resource.` prefix
`logs.log_record`	`log_conditions` with `log.` prefix
`profiles.resource`	`profile_conditions` with `resource.` prefix
`profiles.profile`	`profile_conditions` with `profile.` prefix

Troubleshooting

When using OTTL you can enable debug logging in the collector to print out useful information, such as if the condition matched and the TransformContext used in the condition, to help you troubleshoot why a condition is not behaving as you expect. This feature is very verbose, but provides you an accurate view into how OTTL views the underlying data.

receivers:
  file_log:
    start_at: beginning
    include: [ /Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.log ]


processors:
  filter:
    error_mode: ignore
    log_conditions:
      - log.body == "test"

exporters:
  debug:

service:
  telemetry:
    logs:
      level: debug
  pipelines:
    logs:
      receivers:
        - file_log
      processors:
        - filter
      exporters:
        - debug

2024-05-29T16:47:04.362-0600    debug   [email protected]/parser.go:338     condition evaluation result     {"kind": "processor", "name": "filter", "pipeline": "logs", "condition": "body == \"test\"", "match": true, "TransformContext": {"resource": {"attributes": {}, "dropped_attribute_count": 0}, "scope": {"attributes": {}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log"}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1717022824262063000, "severity_number": 0, "severity_text": "", "span_id": "", "time_unix_nano": 0, "trace_id": ""}, "cache": {}}}

Warnings

In general, understand your data before using the filter processor.

When using the Filter Processor make sure you understand the look of your incoming data and test the configuration thoroughly. In general, use as specific a configuration as possible to lower the risk of the wrong data being dropped.
Orphaned Telemetry: The processor allows dropping spans. Dropping a span may lead to orphaned spans if the dropped span is a parent. Dropping a span may lead to orphaned logs if the log references the dropped span.

Available Benchmarks

The filter processor is tested as part of the project’s load tests, with the results being publicly available on the benchmarks page. There you can find the CPU and memory usage of the processor evaluating a representative set of OTTL conditions against each signal at 10,000 items/second:

Traces: CPU and memory
Metrics: CPU and memory
Logs: CPU and memory

Refer to the test for more information about the setup.

Last generated: 2026-08-24

​Filter Processor

​Supported Telemetry

​Overview

​Configuration

​General Config

​Context

​Error Modes

​Feature Gate

processor.filter.defaultErrorModeIgnore

​Basic Config

​Advanced Config

​Context Inference

​Examples

​Dropping data based on a resource attribute

​Dropping metrics with invalid type

​Dropping specific metric and value

​Dropping non-HTTP spans

​Dropping HTTP spans

​Dropping non-error spans with a duration of less than 1 second

​OTTL Functions

​HasAttrKeyOnDatapoint

​HasAttrOnDatapoint

​Legacy Configuration

​Troubleshooting

​Warnings

​Available Benchmarks