Skip to main content

​
Filter Processor

Status Available in: core, contrib, k8s Maintainers: @TylerHelmuth, @evan-bradley, @edmocosta, @bogdandrutu Source: opentelemetry-collector-contrib

​
Supported Telemetry

Logs Metrics Traces

​
Overview

[!NOTE] This documentation applies only to version 0.146.0 and later. Configuration from previous version is still supported, but no longer documented in this README. For information on earlier versions, please refer to the previous documentation.

​
Configuration

The Filter Processor utilizes the OpenTelemetry Transformation Language to create conditions that determine when telemetry should be dropped. If any condition is met, the telemetry is dropped (each condition is ORed together).

​
General Config

filter:
  error_mode: propagate
  <trace|metric|log|profile>_conditions: []
The Filter Processor’s primary configuration is broken down by signal (traces, metrics, logs, and profiles) and allows you to configure a list of conditions for the processor to evaluate. The list can be made of:
  • OTTL conditions. This option will meet most user’s needs. See Basic Config for more details.
  • Objects, which allows users to apply configuration options to a specific list of conditions. See Advanced Config for more details.
The OTTL allows the use of and, or, and () in conditions. See OTTL Boolean Expressions for more details.

​
Context

Within each <signal>_conditions list, only certain OTTL Contexts can be used. Each context provides access to different telemetry fields. Click the context name for detailed documentation.
SignalAvailable Contexts
trace_conditionsresource, scope, span, and spanevent
metric_conditionsresource, scope, metric, and datapoint
log_conditionsresource, scope, and log
profile_conditionsresource, scope, and profile
Telemetry is evaluated hierarchically, from higher to lower levels. The hierarchy may vary by signal type; examples include: Logs: resource → scope → log Metrics: resource → scope → metric → datapoint Traces: resource → scope → span → spanevent For conditions that apply to the same signal, such as spans and span events, if the “higher” level telemetry matches a condition and is dropped, the “lower” level condition will not be checked. This means that if a span is dropped but a span event condition was defined, the span event condition will not be checked for that span. The same relationship applies to other signals. If all span events for a span are dropped, the span will be left intact. If all datapoints for a metric are dropped, the metric will also be dropped. Note that when a single condition contains paths from different contexts, resource and spanevent for example, the condition is evaluated in the lower context spanevent. 
filter:
  trace_conditions:
    - resource.attributes["host.name"] == "localhost" or spanevent.name == "grpc.timeout"
This condition translates to: For each span event, check whether its parent span’s resource attribute host.name is localhost or whether the span event’s name is grpc.timeout. Drop the span event if either condition is true.

​
Error Modes

The filter processor also allows configuring an optional field, error_mode, which will determine how the processor reacts to errors that occur while processing an OTTL condition. propagate is the default mode.
error_modedescription
ignoreThe processor ignores errors returned by conditions, logs them, and continues on to the next condition. This is the recommended mode.
silentThe processor ignores errors returned by conditions, does not log them, and continues on to the next condition.
propagateThe processor returns the error up the pipeline. This will result in the payload being dropped from the collector.

​
Basic Config

The basic configuration style allows you to configure OTTL conditions as a flat, OR-ed list, without worrying about extra configurations. This is the simplest way to configure the Filter Processor. If you need explicit context specification or specific error modes, see Advanced Config. Format: 
filter:
  error_mode: propagate
  <trace|metric|log|profile>_conditions:
    - string
    - string
    - string
Example: 
processors:
  filter:
    error_mode: propagate
    trace_conditions:
      - span.attributes["container.name"] == "app_container_1"
      - resource.attributes["host.name"] == "localhost"
      - span.name == "app_3"
    metric_conditions:
      - metric.name == "my.metric" and resource.attributes["my_label"] == "abc123"
      - metric.type == METRIC_DATA_TYPE_HISTOGRAM
      - resource.attributes["service.name"] == "my_service_name"
    log_conditions:
      - IsMatch(log.body, ".*password.*")
      - log.severity_number < SEVERITY_NUMBER_WARN
    profile_conditions:
      - profile.duration_unix_nano > 3000

​
Advanced Config

For more complex use cases you may need to use the Filter Processor’s advanced configuration style to explicitly specify the context and apply additional settings to a group of conditions. Format: 
filter:
  error_mode: ignore
  <trace|metric|log|profile>_conditions:
    - context: string
      error_mode: propagate
      conditions:
        - string
        - string
    - context: string
      conditions:
        - string
context: specifies the OTTL context for the conditions. See the Context table for valid values per signal type. Note: In most cases, you should not set this field manually. The processor’s context inferrer will automatically determine the correct context based on the paths used in your conditions. Only set this field if you have a specific need to set the context. error_mode: allows overriding the top-level error_mode for this specific group of conditions. Valid values are ignore, silent, and propagate. See the Error Modes table for more details. conditions: a list of OTTL conditions. If any condition is met, the telemetry is dropped (conditions are OR-ed together). Example: 
processors:
  filter:
    error_mode: propagate
    trace_conditions:
      - conditions:
          - resource.attributes["host.name"] == "localhost"
      - error_mode: ignore
        conditions:
          - span.attributes["container.name"] == "container_1"
          - span.name == "app_3"
      - conditions:
          - spanevent.attributes["grpc"] == true
          - IsMatch(spanevent.name, ".*grpc.*")
    metric_conditions:
      - conditions:
          - metric.name == "my.metric" and resource.attributes["my_label"] == "abc123"
          - metric.type == METRIC_DATA_TYPE_HISTOGRAM
      - conditions:
          - metric.type == METRIC_DATA_TYPE_SUMMARY
    log_conditions:
      - conditions:
          - IsMatch(log.body, ".*password.*")
          - log.severity_number < SEVERITY_NUMBER_WARN

​
Context Inference

[!NOTE] This is an advanced topic and is not necessary to get started using the Filter Processor. Read on if you’re interested in how the Filter Processor parses your OTTL conditions.
An OTTL Context defines which Paths, functions, and enums are available when parsing the condition. The Filter Processor automatically infers the OTTL Context based on the paths defined in a condition. This inference is based on the Path names, functions, and enums present in the condition. The inference happens automatically because Path names are prefixed with the context name. For example: 
trace_conditions:
  - span.attributes["container.name"] == "container_1" and spanevent.attributes["grpc"] == true
In this configuration, the inferred context value is spanevent, as it is the only context that supports parsing both spanevent and span Paths. In Filter processor, each condition in a list is parsed individually and grouped by its inferred context. Conditions are executed by context group in hierarchical order (resource → scope → signal-specific). For example: 
trace_conditions:
  - scope.name == "my.scope"                          # inferred as scope context
  - span.name == "app_3"                              # inferred as span context
  - spanevent.name == "grpc.timeout"                  # inferred as spanevent context
  - resource.attributes["host.name"] == "localhost"   # inferred as resource context
Resource condition is executed first, followed by scope, span, and finally spanevent conditions. The primary benefit of context inference is that it enhances the efficiency of data filtering by linking them to the most suitable context. This optimization ensures that the processing are both accurate and performant, leveraging the hierarchical structure of contexts to avoid unnecessary iterations and improve overall processing efficiency. All of this happens automatically, leaving you to write OTTL conditions without worrying about context. In some situations a combination of Paths, functions, or enums is not allowed. For example: 
trace_conditions:
  - IsRootSpan() or spanevent.name == "bar"
This condition fails because IsRootSpan() is only available in the span context, not span events, while the spanevent Path prefix requires the condition to evaluate in the span event context. The solution is to split into separate span and span event conditions: 
trace_conditions:
  - context: span
    conditions:
      - IsRootSpan()
  - conditions:
      - spanevent.name == "bar"
The advanced configuration is required for IsRootSpan() because there is no Path prefix in the condition, so the context must be set explicitly to span.

​
Examples

​
Dropping data based on a resource attribute

processors:
  filter:
    error_mode: ignore
    trace_conditions:
      - resource.attributes["k8s.pod.name"] == "my-pod-name"

​
Dropping metrics with invalid type

processors:
  filter:
    error_mode: ignore
    metric_conditions:
      - metric.type == METRIC_DATA_TYPE_NONE

​
Dropping specific metric and value

processors:
  filter:
    error_mode: ignore
    metric_conditions:
      - metric.name == "k8s.pod.phase" and datapoint.value_int == 4

​
Dropping non-HTTP spans

processors:
  filter:
    error_mode: ignore
    trace_conditions:
      - span.attributes["http.request.method"] == nil

​
Dropping HTTP spans

processors:
  filter:
    error_mode: ignore
    trace_conditions:
      - span.attributes["http.request.method"] != nil

​
Dropping non-error spans with a duration of less than 1 second

processors:
  filter:
    error_mode: ignore
    trace_conditions:
      - (span.end_time - span.start_time) < Duration("1s") and span.status.code != STATUS_CODE_ERROR

​
OTTL Functions

The filter processor has access to all OTTL Converter functions In addition, the processor defines a few of its own functions: Metrics only functions

​
HasAttrKeyOnDatapoint

HasAttrKeyOnDatapoint(key) Returns true if the given key appears in the attribute map of any datapoint on a metric. key must be a string. You must use the metrics.metric context. Examples:
  • HasAttrKeyOnDatapoint("http.method")

filter/keep_good_metrics:
  error_mode: ignore
  metrics:
    metric:
      - HasAttrKeyOnDatapoint("bad.metric")

​
HasAttrOnDatapoint

HasAttrOnDatapoint(key, value) Returns true if the given key and value appears in the attribute map of any datapoint on a metric. key and value must both be strings. If the value of the attribute on the datapoint is not a string, value will be compared to "". You must use the metrics.metric context. Examples:
  • HasAttrOnDatapoint("http.method", "GET")
# Drops metrics containing the 'bad.metric' attribute key and 'true' value
filter/keep_good_metrics:
  error_mode: ignore
  metrics:
    metric:
      - HasAttrOnDatapoint("bad.metric", "true")

​
Legacy Configuration

The following configuration options are deprecated and will be removed in a future release. Please migrate to the new *_conditions format.
Deprecated ConfigMigrate To
traces.resourcetrace_conditions with resource. prefix
traces.spantrace_conditions with span. prefix
traces.spaneventtrace_conditions with spanevent. prefix
metrics.resourcemetric_conditions with resource. prefix
metrics.metricmetric_conditions with metric. prefix
metrics.datapointmetric_conditions with datapoint. prefix
logs.resourcelog_conditions with resource. prefix
logs.log_recordlog_conditions with log. prefix
profiles.resourceprofile_conditions with resource. prefix
profiles.profileprofile_conditions with profile. prefix

​
Troubleshooting

When using OTTL you can enable debug logging in the collector to print out useful information, such as if the condition matched and the TransformContext used in the condition, to help you troubleshoot why a condition is not behaving as you expect. This feature is very verbose, but provides you an accurate view into how OTTL views the underlying data. 
receivers:
  file_log:
    start_at: beginning
    include: [ /Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.log ]


processors:
  filter:
    error_mode: ignore
    log_conditions:
      - log.body == "test"

exporters:
  debug:

service:
  telemetry:
    logs:
      level: debug
  pipelines:
    logs:
      receivers:
        - file_log
      processors:
        - filter
      exporters:
        - debug

2024-05-29T16:47:04.362-0600    debug   [email protected]/parser.go:338     condition evaluation result     {"kind": "processor", "name": "filter", "pipeline": "logs", "condition": "body == \"test\"", "match": true, "TransformContext": {"resource": {"attributes": {}, "dropped_attribute_count": 0}, "scope": {"attributes": {}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log"}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1717022824262063000, "severity_number": 0, "severity_text": "", "span_id": "", "time_unix_nano": 0, "trace_id": ""}, "cache": {}}}

​
Warnings

In general, understand your data before using the filter processor.
  • When using the Filter Processor make sure you understand the look of your incoming data and test the configuration thoroughly. In general, use as specific a configuration as possible to lower the risk of the wrong data being dropped.
  • Orphaned Telemetry: The processor allows dropping spans. Dropping a span may lead to orphaned spans if the dropped span is a parent. Dropping a span may lead to orphaned logs if the log references the dropped span.
Last generated: 2026-04-13