Transform Processor

Available in: contrib, k8s Maintainers: @TylerHelmuth, @evan-bradley, @edmocosta, @bogdandrutu Source: opentelemetry-collector-contrib

Supported Telemetry

Overview

The Transform Processor modifies telemetry based on configuration using the OpenTelemetry Transformation Language (OTTL). For each signal type, the processor takes a list of statements and executes them against the incoming telemetry, following the order specified in the configuration. Each statement can access and transform telemetry using functions, and allows the use of a condition to help decide whether the function should be executed.

Config
Grammar
Supported functions
Examples
Troubleshooting
Contributing
Feature Gate
Compatibility
Performance and scaling
Self-observability

Config

General Config

[!NOTE] If you don’t know how to write OTTL statements yet, first see OTTL’s Getting Started docs.

transform:
  error_mode: ignore
  <trace|metric|log|profile>_statements: []

The Transform Processor’s primary configuration section is broken down by signal (traces, metrics, logs, and profiles) and allows you to configure a list of statements for the processor to execute. The list can be made of:

OTTL statements. This option will meet most user’s needs. See Basic Config for more details.
Objects, which allows users to apply configuration options to a specific list of statements. See Advanced Config for more details.

[!NOTE] Support for the profile signal (the profile_statements configuration and the profile context) is experimental and subject to change or removal in the future.

Within each <signal_statements> list, only certain OTTL Path prefixes can be used:

Signal	Path Prefix Values
trace_statements	`resource`, `scope`, `span`, and `spanevent`
metric_statements	`resource`, `scope`, `metric`, `datapoint`, and `exemplar`
log_statements	`resource`, `scope`, and `log`
profile_statements	`resource`, `scope`, and `profile`

This means, for example, that you cannot use the Path span.attributes within the log_statements configuration section. error_mode: determines how the processor treats errors that occur while processing a statement. If the top-level error_mode is not specified, ignore will be used. The top-level error_mode can be overridden at statement group level, offering more granular control over error handling. If the statement group error_mode is not specified, the top-level error_mode is applied.

error_mode	description
ignore	The processor ignores errors returned by statements, logs the error, and continues on to the next statement. This is the recommended mode.
silent	The processor ignores errors returned by statements, does not log the error, and continues on to the next statement.
propagate	The processor returns the error up the pipeline. This will result in the payload being dropped from the collector.

Basic Config

[!NOTE] If you don’t know how to write OTTL statements yet, first see OTTL’s Getting Started docs.

The basic configuration style allows you to configure OTTL statements as a list, without worrying about extra configurations. This is the simplest way to configure the Transform Processor. If you need global conditions or specific error modes see Advanced Config. Format:

transform:
  error_mode: ignore
  <trace|metric|log|profile>_statements:
    - string
    - string
    - string

Example:

transform:
  error_mode: ignore
  trace_statements:
    - keep_keys(span.attributes, ["service.name", "service.namespace", "cloud.region", "process.command_line"])
    - replace_pattern(span.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
    - limit(span.attributes, 100, [])
    - truncate_all(span.attributes, 4096)
  metric_statements:
    - keep_keys(resource.attributes, ["host.name"])
    - truncate_all(resource.attributes, 4096)
    - set(metric.description, "Sum") where metric.type == "Sum"
    - convert_sum_to_gauge() where metric.name == "system.processes.count"
    - convert_gauge_to_sum("cumulative", false) where metric.name == "prometheus_metric"
  log_statements:
    - set(log.severity_text, "FAIL") where log.body == "request failed"
    - replace_all_matches(log.attributes, "/user/*/list/*", "/user/{userId}/list/{listId}")
    - replace_all_patterns(log.attributes, "value", "/account/\\d{4}", "/account/{accountId}")
    - set(log.body, log.attributes["http.route"])
  profile_statements:
    - keep_keys(resource.attributes, ["host.name"])
    - set(profile.attributes["tag"], "profile#23")
    - set(profile.original_payload_format, "json")

In some situations a combination of Paths, functions, or enums is not allowed, and the solution might require multiple Advanced Config configuration groups. See Context Inference for more details.

Advanced Config

[!NOTE] If you don’t know how to write OTTL statements yet, first see OTTL’s Getting Started docs.

For more complex use cases you may need to use the Transform Processor’s advanced configuration style to group related OTTL statements. Format:

transform:
  error_mode: ignore
  <trace|metric|log|profile>_statements:
    - context: string
      error_mode: propagate
      shared_cache: true
      conditions: 
        - string
        - string
      statements:
        - string
        - string
        - string
    - context: string
      error_mode: silent
      statements:
        - string
        - string
        - string

error_mode: allows overriding the top-level error_mode. See General Config for details on how to configure error_mode. shared_cache (experimental): enables sharing per-context caches between all advanced statement groups with the option enabled in this Transform Processor instance. Use this when you want to hold state across multiple iterations over a set of data. conditions: a list comprised of multiple where clauses, which will be processed as global conditions for the accompanying set of statements. The conditions are ORed together, which means only one condition needs to evaluate to true in order for the statements (including their individual Where clauses) to be executed. statements: a list of OTTL statements. Example:

transform:
  error_mode: ignore
  metric_statements:
    - error_mode: propagate
      conditions:
        - metric.type == METRIC_DATA_TYPE_SUM
      statements:
        - set(metric.description, "Sum")

  log_statements:
    - conditions:
        - IsMap(log.body) and log.body["object"] != nil
      statements:
        - set(log.body, log.attributes["http.route"])

The Transform Processor will enforce that all the Paths, functions, and enums used in a group’s statements are parsable. In some situations a combination of Paths, functions, or enums is not allowed, and it might require multiple configuration groups. See Context Inference for more details.

Context inference

[!NOTE] This is an advanced topic and is not necessary to get started using the Transform Processor. Read on if you’re interested in how the Transform Processor parses your OTTL statements.

An OTTL Context defines which Paths, functions, and enums are available when parsing the statement. The Transform Processor automatically infers the OTTL Context based on the paths defined in a statement. This inference is based on the Path names, functions, and enums present in the statements. The inference happens automatically because Path names are prefixed with the Context name. For example:

metric_statements:
  - set(metric.description, "test passed") where datapoint.attributes["test"] == "pass"

In this configuration, the inferred Context value is datapoint, as it is the only Context that supports parsing both datapoint and metric Paths. In the following example, the inferred Context is metric, as metric is the context capable of parsing both metric and resource data.

metric_statements:
  - set(resource.attributes["test"], "passed")
  - set(metric.description, "test passed")

The primary benefit of context inference is that it enhances the efficiency of statement processing by linking them to the most suitable context. This optimization ensures that data transformations are both accurate and performant, leveraging the hierarchical structure of contexts to avoid unnecessary iterations and improve overall processing efficiency. All of this happens automatically, leaving you to write OTTL statements without worrying about Context. In some situations a combination of Paths, functions, or enums is not allowed. For example:

metric_statements:
  - convert_sum_to_gauge() where metric.name == "system.processes.count"
  - limit(datapoint.attributes, 100, ["host.name"])

In this configuration, the datapoint Path prefixed is used in the same group of statements as the convert_sum_to_gauge function. Since convert_sum_to_gauge can only be used with the metrics, not datapoints, but the list statements contains a reference to the datapoints via the datapoint Path prefix, the group of statements cannot be parsed. The solution is to separate the statements into separate Advanced Config groups:

metric_statements:
  - statements:
    - convert_sum_to_gauge() where metric.name == "system.processes.count"
  - statements:
      - limit(datapoint.attributes, 100, ["host.name"])

Grammar

You can learn more in-depth details on the capabilities and limitations of the OpenTelemetry Transformation Language used by the Transform Processor by reading about its grammar.

Supported functions:

These common functions can be used for any Signal.

OTTL Functions

In addition to the common OTTL functions, the processor defines its own functions to help with transformations specific to this processor: Metrics only functions

convert_sum_to_gauge
convert_gauge_to_sum
extract_avg_metric
extract_count_metric
extract_percentile_metric
extract_sum_metric
convert_summary_count_val_to_sum
convert_summary_quantile_val_to_gauge
convert_summary_sum_val_to_sum
copy_metric
scale_metric
aggregate_on_attributes
convert_exponential_histogram_to_histogram
aggregate_on_attribute_value
merge_histogram_buckets

Logs only functions

ParseCEF
ParseCLF
ParseELF
ParseLEEF

Traces only functions

set_semconv_span_name

convert_sum_to_gauge

convert_sum_to_gauge() Converts incoming metrics of type “Sum” to type “Gauge”, retaining the metric’s datapoints. Noop for metrics that are not of type “Sum”. NOTE: This function may cause a metric to break semantics for Gauge metrics. Use at your own risk. Examples:

convert_sum_to_gauge()

convert_gauge_to_sum

convert_gauge_to_sum(aggregation_temporality, is_monotonic) Converts incoming metrics of type “Gauge” to type “Sum”, retaining the metric’s datapoints and setting its aggregation temporality and monotonicity accordingly. Noop for metrics that are not of type “Gauge”. aggregation_temporality is a string ("cumulative" or "delta") that specifies the resultant metric’s aggregation temporality. is_monotonic is a boolean that specifies the resultant metric’s monotonicity. NOTE: This function may cause a metric to break semantics for Sum metrics. Use at your own risk. Examples:

convert_gauge_to_sum("cumulative", false)
convert_gauge_to_sum("delta", true)

extract_avg_metric

[!NOTE]
This function supports Histograms, ExponentialHistograms and Summaries.

extract_avg_metric(Optional[suffix]) The extract_avg_metric function creates a new Gauge metric from a Histogram, ExponentialHistogram or Summary by dividing the sum value by the count value for each data point. Data points where the sum value is missing or the count value is 0 are skipped, to avoid division by zero. A metric will only be created if there is at least one data point. suffix is an optional string that defines the suffix for the metric name. By default, it is set to _avg. The name for the new metric will be <original metric name><suffix>. The fields that are copied are: timestamp, starttimestamp, attributes, description, and unit. The new metric that is created will be passed to all subsequent statements in the metrics statements list. Examples:

extract_avg_metric()
extract_avg_metric(".avg")

extract_count_metric

[!NOTE]
This function supports Histograms, ExponentialHistograms and Summaries.

extract_count_metric(is_monotonic, Optional[suffix]) The extract_count_metric function creates a new Sum metric from a Histogram, ExponentialHistogram or Summary’s count value. A metric will only be created if there is at least one data point. is_monotonic is a boolean representing the monotonicity of the new metric. suffix is an optional string that defines the suffix for the metric name. By default, it is set to _count. The name for the new metric will be <original metric name><suffix>. The fields that are copied are: timestamp, starttimestamp, attributes, description, and aggregation_temporality. As metrics of type Summary don’t have an aggregation_temporality field, this field will be set to AGGREGATION_TEMPORALITY_CUMULATIVE for those metrics. The new metric that is created will be passed to all subsequent statements in the metrics statements list.

[!WARNING]
This function may cause a metric to break semantics for Sum metrics. Use only if you’re confident you know what the resulting monotonicity should be.

Examples:

extract_count_metric(true, ".count")
extract_count_metric(false, ".count")

extract_percentile_metric

[!NOTE]
This function supports Histograms and ExponentialHistograms.

extract_percentile_metric(percentile, Optional[suffix]) The extract_percentile_metric function creates a new Gauge metric from a Histogram or ExponentialHistogram by calculating the specified percentile value from the bucket counts. A metric will only be created if there is at least one data point. percentile is a float64 value greater than 0 and less than 100 representing the desired percentile to extract (e.g., 50 for median, 95 for p95, 99 for p99). suffix is an optional string that defines the suffix for the metric name. By default, it is set to _p{percentile} (e.g., _p50, _p95, _p99). The name for the new metric will be <original metric name><suffix>. The fields that are copied are: timestamp, starttimestamp, attributes, description, and unit. For Histograms, the function uses linear interpolation within buckets to estimate the percentile value. Since the lowest bucket has no finite lower bound (-Inf), the function uses the data point’s Min value when available. If Min is not set and the upper bound of that bucket is non-negative, the lower bound is assumed to be 0. Similarly, if the percentile falls in the last bucket (+Inf upper bound), the data point’s Max value is used for interpolation when available. For ExponentialHistograms, it uses logarithmic interpolation appropriate for the exponential bucket structure. The new metric that is created will be passed to all subsequent statements in the metrics statements list. Examples:

extract_percentile_metric(50.0) - Extract median (p50) with default suffix _p50
extract_percentile_metric(95.0, "_p95_custom") - Extract p95 with custom suffix _p95_custom
extract_percentile_metric(99.9) where metric.name == "http.server.duration" - Extract p99.9 only for specific metrics

extract_sum_metric

[!NOTE]
This function supports Histograms, ExponentialHistograms and Summaries.

extract_sum_metric(is_monotonic, Optional[suffix]) The extract_sum_metric function creates a new Sum metric from a Histogram, ExponentialHistogram or Summary’s sum value. If the sum value of a Histogram or ExponentialHistogram data point is missing, no data point is added to the output metric. A metric will only be created if there is at least one data point. is_monotonic is a boolean representing the monotonicity of the new metric. suffix is an optional string that defines the suffix for the metric name. By default, it is set to _sum. The name for the new metric will be <original metric name><suffix>. The fields that are copied are: timestamp, starttimestamp, attributes, description, and aggregation_temporality. As metrics of type Summary don’t have an aggregation_temporality field, this field will be set to AGGREGATION_TEMPORALITY_CUMULATIVE for those metrics. The new metric that is created will be passed to all subsequent statements in the metrics statements list.

[!WARNING]
This function may cause a metric to break semantics for Sum metrics. Use only if you’re confident you know what the resulting monotonicity should be.

Examples:

extract_sum_metric(true, ".sum")
extract_sum_metric(false, ".sum")

convert_summary_count_val_to_sum

convert_summary_count_val_to_sum(aggregation_temporality, is_monotonic, Optional[suffix]) The convert_summary_count_val_to_sum function creates a new Sum metric from a Summary’s count value. aggregation_temporality is a string ("cumulative" or "delta") representing the desired aggregation temporality of the new metric. is_monotonic is a boolean representing the monotonicity of the new metric. suffix is an optional string that defines the suffix for the metric name. By default, it is set to _count. The name for the new metric will be <summary metric name><suffix>. The fields that are copied are: timestamp, starttimestamp, attributes, and description. The new metric that is created will be passed to all functions in the metrics statements list. Function conditions will apply. NOTE: This function may cause a metric to break semantics for Sum metrics. Use at your own risk. Examples:

convert_summary_count_val_to_sum("delta", true, ".count")
convert_summary_count_val_to_sum("cumulative", false, ".count")

convert_summary_quantile_val_to_gauge

convert_summary_quantile_val_to_gauge(Optional[attributeKey], Optional[suffix]) The convert_summary_quantile_val_to_gauge function creates a new Gauge metric and injects each of the Summary’s quantiles into a single Gauge datapoint. attributeKey is an optional string that specifies the attribute key holding the quantile value for each corresponding output data point. The default key is quantile. suffix is an optional string representing the suffix of the metric name. The default value is .quantiles. The name for the new metric will be <summary metric name>.quantiles. The fields that are copied are: timestamp, starttimestamp, attributes, unit and description. The new metric that is created will be passed to all functions in the metrics statements list. Function conditions will apply. Examples:

convert_summary_quantile_val_to_gauge("custom_quantile", "custom_suffix")
convert_summary_quantile_val_to_gauge("custom_quantile")
convert_summary_quantile_val_to_gauge()

convert_summary_sum_val_to_sum

convert_summary_sum_val_to_sum(aggregation_temporality, is_monotonic, Optional[suffix]) The convert_summary_sum_val_to_sum function creates a new Sum metric from a Summary’s sum value. aggregation_temporality is a string ("cumulative" or "delta") representing the desired aggregation temporality of the new metric. is_monotonic is a boolean representing the monotonicity of the new metric. suffix is an optional string that defines the suffix for the metric name. By default, it is set to _sum. The name for the new metric will be <summary metric name><suffix>. The fields that are copied are: timestamp, starttimestamp, attributes, and description. The new metric that is created will be passed to all functions in the metrics statements list. Function conditions will apply. NOTE: This function may cause a metric to break semantics for Sum metrics. Use at your own risk. Examples:

convert_summary_sum_val_to_sum("delta", true, ".sum")
convert_summary_sum_val_to_sum("cumulative", false, ".sum")

copy_metric

copy_metric(Optional[name], Optional[description], Optional[unit]) The copy_metric function copies the current metric, adding it to the end of the metric slice. name is an optional string. description is an optional string. unit is an optional string. The new metric will be exactly the same as the current metric. You can use the optional parameters to set the new metric’s name, description, and unit. NOTE: The new metric is appended to the end of the metric slice and therefore will be included in all the metric statements. It is a best practice to ALWAYS include a Where clause when copying a metric that WILL NOT match the new metric. Examples:

copy_metric(name="http.request.status_code", unit="s") where metric.name == "http.status_code
copy_metric(desc="new desc") where metric.description == "old desc"

convert_exponential_histogram_to_histogram

Warning: This conversion is not defined by the OpenTelemetry Specification and may lose precision. convert_exponential_histogram_to_histogram(distribution, [ExplicitBounds]) Converts an exponential histogram to an explicit histogram. The function requires two arguments:

distribution controls how each exponential bucket is allocated:
- upper places the bucket count at its upper bound.
- midpoint places the bucket count at its arithmetic midpoint.
- uniform assumes a uniform distribution within the bucket and allocates counts according to the overlap with each explicit bucket. Rounding is deterministic.
- random uses the same overlap calculation as uniform, with randomized rounding.
ExplicitBounds is a non-empty, strictly increasing list of explicit histogram bounds.

The result:

Converts positive, negative, and zero buckets.
Preserves the total count.
Contains len(ExplicitBounds) + 1 bucket counts. For bounds [b0, b1], the buckets are (-Inf, b0], (b0, b1], and (b1, +Inf).

For example:

convert_exponential_histogram_to_histogram("random", [0.0, 10.0, 100.0, 1000.0, 10000.0])

Use this function only when downstream consumers cannot accept exponential histograms.

scale_metric

scale_metric(factor, Optional[unit]) The scale_metric function multiplies the values in the data points in the metric by the float value factor. If the optional string unit is provided, the metric’s unit will be set to this value. The supported data types are: Supported metric types are Gauge, Sum, Histogram, and Summary. Examples:

scale_metric(0.1): Scale the metric by a factor of 0.1. The unit of the metric will not be modified.
scale_metric(10.0, "kWh"): Scale the metric by a factor of 10.0 and sets the unit to kWh.

aggregate_on_attributes

aggregate_on_attributes(function, Optional[attributes]) The aggregate_on_attributes function aggregates all data points in the metric. function is a case-sensitive string that represents the aggregation function to apply. attributes is an optional list of datapoint attribute keys (strings) to aggregate upon. The optional attributes argument controls which datapoint attributes are kept before aggregation:

omitted (aggregate_on_attributes(function)): all existing data point attributes are kept. Aggregation only combines data points that already share the same full attribute set.
[] (empty list): all datapoint attributes are removed. Aggregation is performed across all data points in the metric.
[...] (list of attribute keys): only the specified data point attributes are kept. All other attributes are removed, and aggregation is performed by grouping on the remaining attributes.

NOTE: This function is supported only in metric context. The following metric types can be aggregated:

sum
gauge
histogram
exponential histogram

Supported aggregation functions are:

sum
max
min
mean
median
count

NOTE: Only the sum aggregation function is supported for histogram and exponential histogram datatypes. Examples:

aggregate_on_attributes("sum", ["attr1", "attr2"]) where metric.name == "system.memory.usage"
aggregate_on_attributes("max") where metric.name == "system.memory.usage"
aggregate_on_attributes("max", []) where metric.name == "system.memory.usage"

The aggregate_on_attributes function can also be used in conjunction with keep_matching_keys or delete_matching_keys. For example, to remove attribute keys matching a regex and aggregate the metrics on the remaining attributes, you can perform the following statement sequence:

statements:
   - delete_matching_keys(resource.attributes, "(?i).*myRegex.*") where metric.name == "system.memory.usage"
   - aggregate_on_attributes("sum") where metric.name == "system.memory.usage"

To aggregate only using a specified set of attributes, you can use keep_matching_keys.

aggregate_on_attribute_value

aggregate_on_attribute_value(function, attribute, values, newValue) The aggregate_on_attribute_value function aggregates all datapoints in the metric containing the attribute attribute (type string) with one of the values present in the values parameter (list of strings) into a single datapoint where the attribute has the value newValue (type string). function is a case-sensitive string that represents the aggregation function. NOTE: This function is supported only in metric context. The following metric types can be aggregated:

sum
gauge
histogram
exponential histogram

Supported aggregation functions are:

sum
max
min
mean
median
count

NOTE: Only the sum aggregation function is supported for histogram and exponential histogram datatypes. Examples:

aggregate_on_attribute_value("sum", "attr1", ["val1", "val2"], "new_val") where metric.name == "system.memory.usage"

The aggregate_on_attribute_value function can also be used in conjunction with keep_matching_keys or delete_matching_keys. For example, to remove attribute keys matching a regex and aggregate the metrics on the remaining attributes, you can perform the following statement sequence:

statements:
   - delete_matching_keys(resource.attributes, "(?i).*myRegex.*") where metric.name == "system.memory.usage"
   - aggregate_on_attribute_value("sum", "attr1", ["val1", "val2"], "new_val") where metric.name == "system.memory.usage"

To aggregate only using a specified set of attributes, you can use keep_matching_keys.

merge_histogram_buckets

merge_histogram_buckets(target_value, method) The merge_histogram_buckets function merges explicit histogram buckets. The method argument is optional and defaults to remove_explicit_bound. target_value is interpreted according to method:

remove_explicit_bound: target_value is the explicit boundary to remove. The function merges the bucket ending at this boundary with the next bucket. This method uses floating-point tolerance (epsilon = 1e-12) when matching the boundary.
limit_buckets: target_value is the maximum number of buckets to keep. It must be a positive integer. The function reduces resolution with a single uniform compaction pass. It chooses the smallest divisor that keeps the resulting bucket count at or below target_value, merges adjacent buckets in groups of that size from lower to higher bucket order, combines their counts, and keeps any partial final group. Bucket count values and boundary widths do not affect which buckets are merged. The resulting histogram may have fewer than target_value buckets when no smaller uniform divisor can stay within the limit.

The function:

Preserves the total count and sum of the histogram.
Only works on histogram metrics (no-op for other metric types).
Makes no changes if:
- The explicit boundary is not found when using remove_explicit_bound.
- The histogram is already at or below the requested bucket limit when using limit_buckets.
- The histogram is empty.
- The histogram structure is invalid (mismatched bounds and counts, or unordered bounds when using limit_buckets).

NOTE: The limit_buckets method reduces histogram resolution and may affect percentile or quantile accuracy. Use only when you are confident that reducing bucket detail is acceptable for downstream consumers. Examples:

- merge_histogram_buckets(0.5) where metric.name == "http_request_duration"

# Given a histogram with:
# bounds: [0.1, 0.5, 1.0]
# counts: [5, 8, 3, 1]
#
# After merging at 0.5:
# bounds: [0.1, 1.0]
# counts: [5, 11, 1]

# Limit histograms to at most 5 buckets
- merge_histogram_buckets(5, method="limit_buckets") where metric.name == "http_request_duration"

# Given a histogram with:
# bounds: [0.1, 0.2, 0.5, 1.0, 2.5, 5.0, 10.0, 30.0]
# counts: [80, 4, 6, 120, 3, 2, 40, 10, 1]
#
# After limiting to 5 buckets, one compaction pass merges bucket pairs:
# bounds: [0.2, 1.0, 5.0, 30.0]
# counts: [84, 126, 5, 50, 1]

ParseCEF

ParseCEF(target) The ParseCEF function returns a pcommon.Map that is the result of parsing the target string as a Common Event Format (CEF) message. target is a Getter that returns a string. If the returned string is empty, or cannot be parsed as CEF, an error will be returned. ParseCEF is tolerant of an optional syslog header preceding the CEF: token; parsing begins at the first occurrence of CEF: in the input. The returned map has the following top-level fields:

cef.version — the CEF version (the integer following CEF:).
cef.device_vendor, cef.device_product, cef.device_version, cef.device_event_class_id, cef.name, cef.severity — the six CEF header fields.
cef.extensions — a map of the parsed key/value extension pairs.

Within the header fields, the escape sequences \| (pipe) and \\ (backslash) are unescaped. Within extension values, the escape sequences \\ (backslash), \= (equals), \n (newline), and \r (carriage return) are unescaped. Extension parsing uses the position of the next key= token as the end of the current value, so values may contain spaces. All extension values are returned as strings. Examples:

ParseCEF(body)
ParseCEF("CEF:0|Security|threatmanager|1.0|100|worm successfully stopped|10|src=10.0.0.1 dst=2.1.2.2 spt=1232")

ParseCLF

ParseCLF(target, Optional[format]) The ParseCLF function returns a pcommon.Map that is the result of parsing the target string as a Common Log Format (CLF) HTTP access log entry. target is a Getter that returns a string. If the returned string is empty, or cannot be parsed in the selected format, an error will be returned. format is an optional string that selects the log format to parse. Valid values are:

"clf" (default) — the strict Common Log Format:

remotehost rfc931 auth_user [date] "request" status bytes

"combined" — the NCSA Combined Log Format used by default in many Apache and nginx configurations, which is CLF with the quoted referer and user-agent appended:
remotehost rfc931 auth_user [date] "request" status bytes "referer" "user-agent"

Quoted fields (request, referer, user-agent) may contain backslash escape sequences as produced by Apache (\", \\, \xhh, and C-style control escapes such as \n and \t — see the mod_log_config format notes) and nginx (\xhh). These sequences are unescaped in the returned values. The returned map has the following fields:

clf.remote_host — the client’s DNS name or IP address.
clf.rfc931 — the remote logname of the user (CLF uses - when unknown).
clf.auth_user — the authenticated user (CLF uses - when unknown).
clf.timestamp — the contents of the bracketed date field, preserved as a string.
clf.request — the raw request line as sent by the client.
clf.method, clf.request_uri, clf.protocol — the parsed components of the request line, only set when the request line is well-formed.
clf.status — the HTTP status code as an integer.
clf.bytes — the content-length of the response as an integer. Omitted when CLF reports - (e.g. on a 304 response).
clf.referer, clf.user_agent — the referer and user-agent strings, only set when format is "combined".

Examples:

ParseCLF(body)
ParseCLF("127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb.gif HTTP/1.0\" 200 2326")
ParseCLF(body, "combined")

ParseELF

ParseELF(target) The ParseELF function returns a pcommon.Map that is the result of parsing the target string as a W3C Extended Log Format (ELF) log block. target is a Getter that returns a string containing a complete ELF log block (one or more directive lines followed by data lines). If the string is empty or does not contain a valid #Version directive, an error is returned. Intended usage: ParseELF is designed for pipelines where the full ELF block — header directives (#Version, #Fields, etc.) and data lines — is available as a single string. This is the case when using the filelog receiver with a multiline configuration that groups an entire ELF file (or rotated segment) into one log record body, or when the entire log content is read from a single field. In a line-by-line streaming pipeline where #Fields arrives only once at file open, each individual data line does not carry its own header; for that pattern you would need to prepend the known header directives to each data line before passing the string to ParseELF. The returned map contains the following keys:

elf.version — value of the #Version directive (required).
elf.software — value of #Software (omitted if not present).
elf.date — value of #Date (omitted if not present).
elf.start_date — value of #Start-Date (omitted if not present).
elf.end_date — value of #End-Date (omitted if not present).
elf.remark — value of #Remark (omitted if not present).
elf.fields — string slice of field names from the last #Fields directive.
elf.entries — slice of maps, one per data line, keyed by field name. Missing values are represented as "-".

Multiple #Fields directives within a single block are supported; each directive applies to subsequent data lines until the next #Fields directive is encountered. Double-quoted field values (as produced by Microsoft IIS) are handled correctly. Examples:

ParseELF(body)
ParseELF("#Version: 1.0\n#Fields: time cs-method cs-uri\n00:34:23 GET /foo/bar.html")

ParseLEEF

ParseLEEF(target) The ParseLEEF function returns a pcommon.Map that is the result of parsing the target string as a Log Event Extended Format (LEEF) message. target is a Getter that returns a string. If the returned string is empty, or cannot be parsed as LEEF, an error will be returned. ParseLEEF can parse both LEEF 1.0 and LEEF 2.0 messages. The function is tolerant of an optional syslog header preceding the LEEF: token; parsing begins at the first occurrence of LEEF: in the input, so a literal LEEF: appearing in a syslog header ahead of the real header would be misinterpreted. The returned map has the following top-level fields:

leef.version — the LEEF version ("1.0" or "2.0").
leef.vendor, leef.product.name, leef.product.version, leef.event.id — the LEEF header fields.
leef.attributes — a map of the parsed key/value attribute pairs.

For LEEF 1.0 the attribute delimiter is always a tab. For LEEF 2.0 the delimiter is taken from the header and must be either a single character or a 0x-prefixed hex value decoding to a single byte (e.g. 0x09 for tab). An empty delimiter field defaults to tab. The delimiter field is also optional: if the position normally occupied by the delimiter looks like the start of an attribute (i.e. contains =), it is treated as the first attribute and the delimiter defaults to tab. Attribute parsing is lenient: pairs without an = separator or with an empty key are silently skipped, and when the same key appears more than once the last occurrence wins. Whitespace within keys and values is preserved verbatim, since the LEEF spec defines a value as everything up to the delimiter. All attribute values are returned as strings. LEEF defines a set of predefined event attributes (e.g. src, dst, srcPort, usrName, devTime) with expected types such as Integer, IPv4/IPv6, and Time. Examples:

ParseLEEF(body)
ParseLEEF("LEEF:1.0|Microsoft|MSExchange|4.0 SP1|15345|src=10.50.1.1\tdst=2.10.20.20\tsev=5")
ParseLEEF("LEEF:2.0|Lancope|StealthWatch|1.0|41|^|src=10.0.1.8^dst=10.0.0.5^sev=5")

set_semconv_span_name

set_semconv_span_name(semconvVersion, Optional[originalSpanNameAttribute]) The set_semconv_span_name() function overwrites a span name using the OpenTelemetry semantic conventions for HTTP, RPC, messaging, and database spans. In other cases, the original span.name remains unchanged. The primary use case of the set_semconv_span_name() function is to address high-cardinality issues in span metrics when span.name doesn’t comply with the OpenTelemetry requirement that span names be low cardinality such as GET /product/12345, GET/product?id=12345, or SELECT * FROM product WHERE id=12345. Parameters:

semconvVersion is the version of the Semantic Conventions used to generate the span.name. A set of older/deprecated semconv attributes are also supported for backward compatibility (see the table below). Versions 1.37.0 to 1.43.0 are supported.
originalSpanNameAttribute is the optional name of the attribute used to copy the original span.name if different from the name derived from semantic conventions.

Sanitization examples:

Span with high-cardinality name but recommended semantic convention attributes
- Incoming span:
  span.name: GET /api/v1/users/123 # /!\ high cardinality span.kind: server span.attributes http.request.method: GET http.route: /api/v1/users/{id} url.path: /api/v1/users/123
- Span name after applying set_semconv_span_name("1.43.0"): GET /api/v1/users/{id}
- No loss of information on span.name occurs because the recommended attribute http.route is present.
Span with high-cardinality name lacking recommended semantic convention attribute http.route
- Incoming span:
  span.name: GET /api/v1/users/123 # /!\ high cardinality span.kind: server span.attributes http.request.method: GET url.path: /api/v1/users/123
- Span name after applying set_semconv_span_name("1.43.0"): GET
- Loss of information on span.name occurs because the recommended attribute http.route is missing. Note that this loss of information is mitigated if the instrumentation produced attributes that contain the URL path like url.path or url.full.

Compliant span name is unchanged

Incoming span:

span.name: GET /api/v1/users/{id}
span.kind: server
span.attributes
   http.request.method: GET
   http.route: /api/v1/users/{id}
   url.path: /api/v1/users/123

Span name after applying set_semconv_span_name("1.43.0"): GET /api/v1/users/{id}

Backward compatibility: set_semconv_span_name will also read the following deprecated attributes, mapping each to its current equivalent:

v1.43.0 Attribute	Older attribute
`http.request.method`	`http.method`
`rpc.method`	`rpc.grpc.method`
`rpc.service`	`rpc.grpc.service`
`rpc.system.name`	`rpc.system`
`db.system.name`	`db.system`
`db.operation.name`	`db.operation`
`db.namespace`	`db.name`
`messaging.operation.name`	`messaging.operation`
`messaging.destination.name`	`messaging.destination`

Examples:

set_semconv_span_name("1.43.0")
set_semconv_span_name("1.43.0", "original_span_name")

Examples

Perform transformation if field does not exist

Set attribute test to "pass" if the attribute test does not exist:

transform:
  error_mode: ignore
  trace_statements:
    # accessing a map with a key that does not exist will return nil. 
    - set(span.attributes["test"], "pass") where span.attributes["test"] == nil

Rename attribute

There are 2 ways to rename an attribute key: You can either set a new attribute and delete the old:

transform:
  error_mode: ignore
  trace_statements:
    - set(resource.attributes["namespace"], resource.attributes["k8s.namespace.name"])
    - delete_key(resource.attributes, "k8s.namespace.name") 

Or you can update the key using regex:

transform:
  error_mode: ignore
  trace_statements:
    - replace_all_patterns(resource.attributes, "key", "k8s\\.namespace\\.name", "namespace")

Move field to attribute

Set attribute body to the value of the log body:

transform:
  error_mode: ignore
  log_statements:
    - set(log.attributes["body"], log.body)

Combine two attributes

Set attribute test to the value of attributes "foo" and "bar" combined.

transform:
  error_mode: ignore
  trace_statements:
    # Use Concat function to combine any number of string, separated by a delimiter.
    - set(resource.attributes["test"], Concat([resource.attributes["foo"], resource.attributes["bar"]], " "))

Parsing JSON logs

Given the following json body

{
  "name": "log",
  "attr1": "foo",
  "attr2": "bar",
  "nested": {
    "attr3": "example"
  }
}

add specific fields as attributes on the log:

transform:
  log_statements:
    - statements:
        # Parse body as JSON and merge the resulting map with the cache map, ignoring non-json bodies.
        # cache is a field exposed by OTTL that is a temporary storage place for complex operations.
        - merge_maps(log.cache, ParseJSON(log.body), "upsert") where IsMatch(log.body, "^\\{") 
          
        # Set attributes using the values merged into cache.
        # If the attribute doesn't exist in cache then nothing happens.
        - set(log.attributes["attr1"], log.cache["attr1"])
        - set(log.attributes["attr2"], log.cache["attr2"])
        
        # To access nested maps you can chain index ([]) operations.
        # If nested or attr3 do not exist in cache then nothing happens.
        - set(log.attributes["nested.attr3"], log.cache["nested"]["attr3"])

Override context statements error mode

transform:
  # default error mode applied to all context statements
  error_mode: propagate
  log_statements:
    # overrides the default error mode for these statements
    - error_mode: ignore
      statements:
        - merge_maps(log.cache, ParseJSON(log.body), "upsert") where IsMatch(log.body, "^\\{")
        - set(log.attributes["attr1"], log.cache["attr1"])

    # uses the default error mode
    - statements:
        - set(log.attributes["namespace"], log.attributes["k8s.namespace.name"])

Get Severity of an Unstructured Log Body

Given the following unstructured log body

[2023-09-22 07:38:22,570] INFO [Something]: some interesting log

You can find the severity using IsMatch:

transform:
  error_mode: ignore
  log_statements:
    - set(log.severity_number, SEVERITY_NUMBER_INFO) where IsString(log.body) and IsMatch(log.body, "\\sINFO\\s")
    - set(log.severity_number, SEVERITY_NUMBER_WARN) where IsString(log.body) and IsMatch(log.body, "\\sWARN\\s")
    - set(log.severity_number, SEVERITY_NUMBER_ERROR) where IsString(log.body) and IsMatch(log.body, "\\sERROR\\s")

Copy attributes matching regular expression to a separate location

If you want to move resource attributes, which keys are matching the regular expression pod_labels_.* to a new attribute location kubernetes.labels, use the following configuration:

transform:
  error_mode: ignore
  trace_statements:
    - statements:
        - set(resource.cache["attrs"], resource.attributes)
        - keep_matching_keys(resource.cache["attrs"], "pod_labels_.*")
        - set(resource.attributes["kubernetes.labels"], resource.cache["attrs"])

The configuration can be used also with delete_matching_keys() to copy the attributes that do not match the regular expression.

Troubleshooting

When using OTTL you can enable debug logging in the collector to print out useful information, such as the current Statement and the current TransformContext, to help you troubleshoot why a statement is not behaving as you expect. This feature is very verbose, but provides you an accurate view into how OTTL views the underlying data.

receivers:
  file_log:
    start_at: beginning
    include: [ test.log ]

processors:
  transform:
    error_mode: ignore
    log_statements:
      - set(resource.attributes["test"], "pass")
      - set(scope.attributes["test"], ["pass"])
      - set(log.attributes["test"], true)
          

exporters:
  debug:

service:
  telemetry:
    logs:
      level: debug
  pipelines:
    logs:
      receivers:
        - file_log
      processors:
        - transform
      exporters:
        - debug

2025-02-13T13:01:07.590-0700    debug   [email protected]/parser.go:356     initial TransformContext before executing StatementSequence     {"otelcol.component.id": "transform", "otelcol.component.kind": "Processor", "otelcol.pipeline.id": "logs", "otelcol.signal": "logs", "TransformContext": {"resource": {"attributes": {}, "dropped_attribute_count": 0}, "cache": {}}}
2025-02-13T13:01:07.591-0700    debug   [email protected]/parser.go:35      TransformContext after statement execution      {"otelcol.component.id": "transform", "otelcol.component.kind": "Processor", "otelcol.pipeline.id": "logs", "otelcol.signal": "logs", "statement": "set(resource.attributes[\"test\"], \"pass\")", "condition matched": true, "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "cache": {}}}
2025-02-13T13:01:07.593-0700    debug   [email protected]/parser.go:356     initial TransformContext before executing StatementSequence     {"otelcol.component.id": "transform", "otelcol.component.kind": "Processor", "otelcol.pipeline.id": "logs", "otelcol.signal": "logs", "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "scope": {"attributes": {}, "dropped_attribute_count": 0, "name": "", "version": ""}, "cache": {}}}
2025-02-13T13:01:07.594-0700    debug   [email protected]/parser.go:35      TransformContext after statement execution      {"otelcol.component.id": "transform", "otelcol.component.kind": "Processor", "otelcol.pipeline.id": "logs", "otelcol.signal": "logs", "statement": "set(scope.attributes[\"test\"], [\"pass\"])", "condition matched": true, "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "scope": {"attributes": {"test": ["pass"]}, "dropped_attribute_count": 0, "name": "", "version": ""}, "cache": {}}}
2025-02-13T13:01:07.594-0700    debug   [email protected]/parser.go:356     initial TransformContext before executing StatementSequence     {"otelcol.component.id": "transform", "otelcol.component.kind": "Processor", "otelcol.pipeline.id": "logs", "otelcol.signal": "logs", "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "scope": {"attributes": {"test": ["pass"]}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log"}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1739476867483160000, "severity_number": 0, "severity_text": "", "span_id": "0000000000000000", "time_unix_nano": 0, "trace_id": "00000000000000000000000000000000"}, "cache": {}}}
2025-02-13T13:01:07.594-0700    debug   [email protected]/parser.go:35      TransformContext after statement execution      {"otelcol.component.id": "transform", "otelcol.component.kind": "Processor", "otelcol.pipeline.id": "logs", "otelcol.signal": "logs", "statement": "set(log.attributes[\"test\"], true)", "condition matched": true, "TransformContext": {"resource": {"attributes": {"test": "pass"}, "dropped_attribute_count": 0}, "scope": {"attributes": {"test": ["pass"]}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log", "test": true}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1739476867483160000, "severity_number": 0, "severity_text": "", "span_id": "0000000000000000", "time_unix_nano": 0, "trace_id": "00000000000000000000000000000000"}, "cache": {}}}
2025-02-13T13:01:07.594-0700    info    Logs    {"otelcol.component.id": "debug", "otelcol.component.kind": "Exporter", "otelcol.signal": "logs", "resource logs": 1, "log records": 1}

Contributing

See CONTRIBUTING.md.

Warnings

The Transform Processor uses the OpenTelemetry Transformation Language (OTTL) which allows users to modify all aspects of their telemetry. Some specific risks are listed below, but this is not an exhaustive list. In general, understand your data before using the Transform Processor.

Unsound Transformations: Several Metric-only functions allow you to transform one metric data type to another or create new metrics from an existing metrics. Transformations between metric data types are not defined in the metrics data model. These functions have the expectation that you understand the incoming data and know that it can be meaningfully converted to a new metric data type or can meaningfully be used to create new metrics.
- Although the OTTL allows the set function to be used with metric.data_type, its implementation in the Transform Processor is NOOP. To modify a data type you must use a function specific to that purpose.
Identity Conflict: Transformation of metrics have the potential to affect the identity of a metric leading to an Identity Crisis. Be especially cautious when transforming metric name and when reducing/changing existing attributes. Adding new attributes is safe.
Orphaned Telemetry: The processor allows you to modify span_id, trace_id, and parent_span_id for traces and span_id, and trace_id logs. Modifying these fields could lead to orphaned spans or logs.
Other: Statements that modify a higher context (resource or scope) using values from a lower context (for example, set(resource.attributes["to"], log.attributes["from"])) may produce unexpected results. The statement runs once per span, metric, or log record, and every record that shares the same resource or scope writes to the same shared value, so the last record processed determines the final value. For logs, the flatten_data option can be used to avoid this. See #32080 for more details.

Feature Gate

`transform.flatten.logs`

The transform.flatten.logs feature gate enables the flatten_data configuration option (default false). With flatten_data: true, the processor provides each log record with a distinct copy of its resource and scope. Then, after applying all transformations, the log records are regrouped by resource and scope. This option is useful when applying transformations which alter the resource or scope. e.g. set(resource.attributes["to"], log.attributes["from"]), which may otherwise result in unexpected behavior. Using this option typically incurs a performance penalty as the processor must compute many hashes and create copies of resource and scope information for every log record. The feature is currently only available for log processing.

Example Usage

config.yaml:

transform:
  flatten_data: true
  log_statements:
    - set(resource.attributes["to"], log.attributes["from"])

Run collector: ./otelcol --config config.yaml --feature-gates=transform.flatten.logs

`ottl.set.allowNil`

The ottl.set.allowNil feature gate changes the behavior of the OTTL set function when a nil value is evaluated. When enabled, set will pass the nil value directly to the target. Depending on the target, this may result in an error or an empty value. See the OTTL Documentation for full details and migration instructions.

Compatibility

The Transform Processor operates only on the telemetry flowing through the pipeline and has no external service or system dependencies. Its behavior is defined entirely by the OTTL functions, paths, and enums it exposes, which are versioned together with the processor in each OpenTelemetry Collector release. There are no external dependency versions to track for compatibility. Configuration syntax and OTTL semantics follow the OpenTelemetry Collector’s versioning and stability policies. Breaking changes to configuration or OTTL behavior are gated behind feature gates and announced in the changelog.

Performance and scaling

The Transform Processor is stateless across batches: each incoming batch is processed independently, so the processor requires no coordination and can be scaled horizontally by running additional Collector replicas. The shared_cache option and OTTL’s cache only hold state for the duration of a single execution and are never shared across batches or replicas. CPU and memory cost scale with the number and complexity of the statements you configure and with the volume of telemetry flowing through the pipeline. To keep the processor efficient:

Use conditions (statement-level where clauses or advanced global conditions) to skip statements for telemetry they don’t apply to.
Prefer fewer, more targeted statements over broad transformations applied to all telemetry.
The flatten_data option incurs a performance penalty because the processor must compute hashes and copy resource and scope information for every log record.

The transform processor is tested as part of the project’s load tests, with the results publicly available on the benchmarks page. There you can find the CPU and memory usage of the processor executing a representative set of OTTL statements against each signal at 10,000 items/second:

Traces: CPU and memory
Metrics: CPU and memory
Logs: CPU and memory

Refer to the test for more information about the setup. Each function that is unique to the transform processor also has a Go benchmark covering its execution. Run them with:

make benchmark-transform

Self-observability

The Transform Processor does not emit any component-specific metrics or spans. It relies on the standard telemetry that the Collector produces for every processor, which is described in the Collector’s internal telemetry documentation.

Metrics

The following metrics are emitted for the processor:

otelcol_processor_incoming_items: the number of items (spans, metric data points, or log records) passed to the processor.
otelcol_processor_outgoing_items: the number of items emitted by the processor.
otelcol_processor_internal_duration: a histogram of the time taken to process a batch of telemetry through the processor.

These metrics carry the otel.signal and processor attributes, so you can observe throughput per signal and per configured transform processor instance.

Logs

The amount of detail the processor logs is controlled by the Collector’s service::telemetry::logs::level setting:

At info (the default), the processor logs no per-statement information. Errors are logged according to the configured error_mode: ignore logs each failed statement or condition at warn level, silent suppresses them entirely, and propagate returns the error up the pipeline (where the payload is dropped and the failure is surfaced by the pipeline rather than by the processor).
At debug, OTTL additionally logs the details it uses to evaluate your statements: the OTTL context it infers for each statement group, the TransformContext before a statement sequence runs and after every individual statement executes, and the result of each condition evaluation. This is very verbose but is the most useful level for debugging why a statement is or isn’t behaving as expected, since it shows exactly how OTTL sees the underlying data. See Troubleshooting for an example of this output.

Configuration

Example Configuration

transform:
  trace_statements:
    - context: span
      statements:
        - set(name, "bear") where attributes["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])
    - context: resource
      statements:
        - set(attributes["name"], "bear")
  metric_statements:
    - context: datapoint
      statements:
        - set(metric.name, "bear") where attributes["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])
    - context: resource
      statements:
        - set(attributes["name"], "bear")
  log_statements:
    - context: log
      statements:
        - set(body, "bear") where attributes["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])
    - context: resource
      statements:
        - set(attributes["name"], "bear")
  profile_statements:
    - context: profile
      statements:
        - set(original_payload_format, "bear") where original_payload_format == "/animal"
    - context: resource
      statements:
        - set(attributes["name"], "bear")

transform/with_conditions:
  trace_statements:
    - context: span
      conditions:
        - attributes["http.path"] == "/animal"
      statements:
        - set(name, "bear")
  metric_statements:
    - context: datapoint
      conditions:
        - attributes["http.path"] == "/animal"
      statements:
        - set(metric.name, "bear")
  log_statements:
    - context: log
      conditions:
        - attributes["http.path"] == "/animal"
      statements:
        - set(body, "bear")     
  profile_statements:
    - context: profile
      conditions:
        - original_payload_format == "/animal"
      statements:
        - set(original_payload_format, "bear")

transform/ignore_errors:
  error_mode: ignore
  trace_statements:
    - context: resource
      statements:
        - set(attributes["name"], "bear")

transform/bad_syntax_log:
  log_statements:
    - context: log
      statements:
        - set(body, "bear" where attributes["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])

transform/bad_syntax_metric:
  metric_statements:
    - context: datapoint
      statements:
        - set(name, "bear" where attributes["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])

transform/bad_syntax_trace:
  trace_statements:
    - context: span
      statements:
        - set(name, "bear" where attributes["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])

transform/bad_syntax_profile:
  log_statements:
    - context: profile
      statements:
        - set(original_payload_format, "bear" where attributes["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])

transform/bad_syntax_multi_signal:
  trace_statements:
    - context: span
      statements:
        - set(name, "bear" where attributes["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])
  metric_statements:
    - context: datapoint
      statements:
        - set(name, "bear" attributes["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])
  log_statements:
    - context: log
      statements:
        - set(body, "bear" none["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])
  profile_statements:
    - context: profile
      statements:
        - set(original_payload_format, "bear" none["http.path"] == "/animal"
        - keep_keys(attributes, ["http.method", "http.path"])

transform/unknown_function_log:
  log_statements:
    - context: log
      statements:
        - set(body, "bear") where attributes["http.path"] == "/animal"
        - not_a_function(attributes, ["http.method", "http.path"])

transform/unknown_function_metric:
  metric_statements:
    - context: datapoint
      statements:
        - set(metric.name, "bear") where attributes["http.path"] == "/animal"
        - not_a_function(attributes, ["http.method", "http.path"])

transform/unknown_function_trace:
  trace_statements:
    - context: span
      statements:
        - set(name, "bear") where attributes["http.path"] == "/animal"
        - not_a_function(attributes, ["http.method", "http.path"])

transform/unknown_function_profile:
  log_statements:
    - context: profile
      statements:
        - set(original_payload_format, "bear") where attributes["http.path"] == "/animal"
        - not_a_function(attributes, ["http.method", "http.path"])

transform/unknown_context:
  trace_statements:
    - context: test
      statements:
        - set(name, "bear") where attributes["http.path"] == "/animal"

transform/unknown_error_mode:
  error_mode: test

transform/structured_configuration_with_path_context:
  trace_statements:
    - context: span
      statements:
        - set(span.name, "bear") where span.attributes["http.path"] == "/animal"
  metric_statements:
    - context: metric
      statements:
        - set(metric.name, "bear") where resource.attributes["http.path"] == "/animal"
  log_statements:
    - context: log
      statements:
        - set(log.body, "bear") where log.attributes["http.path"] == "/animal"
  profile_statements:
    - context: profile
      statements:
        - set(profile.original_payload_format, "bear") where profile.original_payload_format == "/animal"

transform/structured_configuration_with_inferred_context:
  trace_statements:
    - statements:
      - set(span.name, "bear") where span.attributes["http.path"] == "/animal"
      - set(resource.attributes["name"], "bear")
  metric_statements:
    - statements:
      - set(metric.name, "bear") where resource.attributes["http.path"] == "/animal"
      - set(resource.attributes["name"], "bear")
  log_statements:
    - statements:
      - set(log.body, "bear") where log.attributes["http.path"] == "/animal"
      - set(resource.attributes["name"], "bear")
  profile_statements:
    - statements:
        - set(profile.original_payload_format, "bear") where profile.original_payload_format == "/animal"
        - set(resource.attributes["name"], "bear")

transform/flat_configuration:
  trace_statements:
    - set(span.name, "bear") where span.attributes["http.path"] == "/animal"
    - set(resource.attributes["name"], "bear")
  metric_statements:
    - set(metric.name, "bear") where resource.attributes["http.path"] == "/animal"
    - set(resource.attributes["name"], "bear")
  log_statements:
    - set(log.body, "bear") where log.attributes["http.path"] == "/animal"
    - set(resource.attributes["name"], "bear")
  profile_statements:
    - set(profile.original_payload_format, "bear") where profile.original_payload_format == "/animal"
    - set(resource.attributes["name"], "bear")

transform/mixed_configuration_styles:
  trace_statements:
    - set(span.name, "bear") where span.attributes["http.path"] == "/animal"
    - context: span
      statements:
        - set(attributes["name"], "bear")
        - keep_keys(attributes, ["http.method", "http.path"])

transform/with_shared_cache_key:
  trace_statements:
    - statements:
        - set(resource.attributes["name"], "propagate")
  metric_statements:
    - statements:
        - set(resource.attributes["name"], "silent")
      shared_cache: true

transform/context_statements_error_mode:
  error_mode: ignore
  trace_statements:
    - error_mode: propagate
      statements:
        - set(resource.attributes["name"], "propagate")
    - statements:
        - set(resource.attributes["name"], "ignore")
  metric_statements:
    - error_mode: silent
      statements:
        - set(resource.attributes["name"], "silent")
    - statements:
        - set(resource.attributes["name"], "ignore")
  log_statements:
    - error_mode: propagate
      statements:
        - set(resource.attributes["name"], "propagate")
    - statements:
        - set(resource.attributes["name"], "ignore")
  profile_statements:
    - error_mode: propagate
      statements:
        - set(resource.attributes["name"], "propagate")
    - statements:
        - set(resource.attributes["name"], "ignore")

transform/shared_cache:
  trace_statements:
    - shared_cache: true
      statements:
        - set(resource.attributes["name"], "bear")
    - shared_cache: true
      statements:
        - set(resource.attributes["copy"], resource.attributes["name"])
  metric_statements:
    - shared_cache: true
      statements:
        - set(resource.attributes["name"], "bear")
  log_statements:
    - shared_cache: true
      statements:
        - set(resource.attributes["name"], "bear")

Last generated: 2026-10-05

​Transform Processor

​Supported Telemetry

​Overview

​Config

​General Config

​Basic Config

​Advanced Config

​Context inference

​Grammar

​Supported functions:

​convert_sum_to_gauge

​convert_gauge_to_sum

​extract_avg_metric

​extract_count_metric

​extract_percentile_metric

​extract_sum_metric

​convert_summary_count_val_to_sum

​convert_summary_quantile_val_to_gauge

​convert_summary_sum_val_to_sum

​copy_metric

​convert_exponential_histogram_to_histogram

​scale_metric

​aggregate_on_attributes

​aggregate_on_attribute_value

​merge_histogram_buckets

​ParseCEF

​ParseCLF

​ParseELF

​ParseLEEF

​set_semconv_span_name

​Examples

​Perform transformation if field does not exist

​Rename attribute

​Move field to attribute

​Combine two attributes

​Parsing JSON logs

​Override context statements error mode

​Get Severity of an Unstructured Log Body

​Copy attributes matching regular expression to a separate location

​Troubleshooting

​Contributing

​Warnings

​Feature Gate

​transform.flatten.logs

​Example Usage

​ottl.set.allowNil

​Compatibility

​Performance and scaling

​Self-observability

​Metrics

​Logs

​Configuration

​Example Configuration

Transform Processor

Supported Telemetry

Overview

Config

General Config

Basic Config

Advanced Config

Context inference

Grammar

Supported functions:

convert_sum_to_gauge

convert_gauge_to_sum

extract_avg_metric

extract_count_metric

extract_percentile_metric

extract_sum_metric

convert_summary_count_val_to_sum

convert_summary_quantile_val_to_gauge

convert_summary_sum_val_to_sum

copy_metric

convert_exponential_histogram_to_histogram

scale_metric

aggregate_on_attributes

aggregate_on_attribute_value

merge_histogram_buckets

ParseCEF

ParseCLF

ParseELF

ParseLEEF

set_semconv_span_name

Examples

Perform transformation if field does not exist

Rename attribute

Move field to attribute

Combine two attributes

Parsing JSON logs

Override context statements error mode

Get Severity of an Unstructured Log Body

Copy attributes matching regular expression to a separate location

Troubleshooting

Contributing

Warnings

Feature Gate

`transform.flatten.logs`

Example Usage

`ottl.set.allowNil`

Compatibility

Performance and scaling

Self-observability

Metrics

Logs

Configuration

Example Configuration