Awslambda Receiver

Available in: contrib Maintainers: @MichaelKatsoulis, @Kavindu-Dodan, @axw, @pjanotti Source: opentelemetry-collector-contrib

Supported Telemetry

Overview

A receiver for collecting logs & metrics from AWS services via Lambda invocations. AWS Lambda is a popular serverless service used extensively for event-driven architectures. Many AWS services (S3, CloudWatch, SNS, SQS) can trigger Lambda functions, making it an ideal entry point for collecting data from AWS services. This receiver is designed to run as part of an OpenTelemetry Collector deployed as an AWS Lambda function. The awslambdareceiver enables users to:

Collect logs & metrics stored at various AWS services as OTel native log records & metric data points
Collect custom logs via AWS services that can trigger Lambda functions
Decode/Unmarshal AWS-specific log formats using OpenTelemetry encoding extensions
Leverage OpenTelemetry’s processors to further enrich, filter, or transform collected data before exporting

How It Works

The awslambdareceiver operates as follows:

Accepts Lambda Invocations
Identifies the event source (S3, CloudWatch, etc.)
Uses configured encoding extensions to parse the data
Creates OpenTelemetry records matching the data type (logs, metrics)
Forward derived records to the next component in the pipeline (processors, exporters)

Event handling

The receiver automatically detects the event source based on the Lambda invocation message format. Table below summarizes supported signals and their sources:

Signal	Sources
Logs	S3, CloudWatch Logs subscription
Metrics	S3

Sections below summarize how each event source is handled.

S3 Event handling

S3 events are handled in the following manner:

Receive S3 event notification (for example, using Lambda trigger on s3:ObjectCreated:*)
Download S3 object payload
Decode payload using the configured encoding extension
- Default encoding: Preserve S3 object content as-is
- Custom encoding: Use specified encoding extension (for example, aws_logs_encoding for AWS log formats)
- Metrics use awscloudwatchmetricstreams_encoding extension by default

The following metadata is available through client.Info for both logs and metrics S3 events. This metadata is added to the request context and can be accessed by any downstream component in the pipeline (for example, processors):

Metadata Key	Description
cloud.region	The S3 bucket region
aws.s3.bucket.name	The S3 bucket name
aws.s3.bucket.arn	The S3 bucket ARN
aws.s3.key	The S3 object key

[!NOTE] Static metadata such as cloud.provider are not available through client.Info. These can be added using processors (for example, the resource processor).

CloudWatch Logs subscription

CloudWatch Logs events are handled in the following manner:

Receive CloudWatch Logs subscription filter event
Parse the CloudWatch Logs message (note - unlike S3 events, the payload is included in the event)
Decode payload using the configured encoding extension
- Default encoding: Parse CloudWatch Logs messages to OpenTelemetry log records
- Custom encoding: Use specified encoding extension (for example, aws_logs_encoding for AWS log formats)

Deployment

The following example shows how to deploy the collector as an AWS Lambda function that receives logs from a CloudWatch Logs subscription filter, using the AWS CLI.

Build the collector binary

Use the OpenTelemetry Collector Builder (OCB) to build a minimal binary containing only the required components. This keeps the binary small enough for Lambda’s deployment package size limit. Create a builder manifest:

cat > builder-config.yaml << 'EOF'
dist:
  name: collector
  output_path: .
  otelcol_version: 0.149.0

receivers:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awslambdareceiver v0.149.0

exporters:
  - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.149.0
EOF

Build the binary for Lambda:

CGO_ENABLED=0 GOOS=linux GOARCH=arm64 builder --config=builder-config.yaml

Package and deploy

The Lambda provided.al2023 runtime executes a binary named bootstrap. Create a wrapper script that passes the --config flag to the collector:

cat > bootstrap << 'EOF'
#!/bin/sh
exec /var/task/collector --config /var/task/collector-config.yaml "$@"
EOF
chmod +x bootstrap

Write a collector configuration:

cat > collector-config.yaml << 'EOF'
receivers:
  aws_lambda:

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    logs:
      receivers: [aws_lambda]
      exporters: [debug]
EOF

zip function.zip bootstrap collector collector-config.yaml

Create an IAM role for the Lambda function:

aws iam create-role \
  --role-name otel-lambda-role \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {"Service": "lambda.amazonaws.com"},
      "Action": "sts:AssumeRole"
    }]
  }'

aws iam attach-role-policy \
  --role-name otel-lambda-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

Create the Lambda function:

aws lambda create-function \
  --function-name otel-lambda-cloudwatch-logs \
  --runtime provided.al2023 \
  --handler bootstrap \
  --architectures arm64 \
  --zip-file fileb://function.zip \
  --role arn:aws:iam::<ACCOUNT_ID>:role/otel-lambda-role \
  --timeout 30

Set up the CloudWatch Logs trigger

Grant CloudWatch Logs permission to invoke the Lambda function:

aws lambda add-permission \
  --function-name otel-lambda-cloudwatch-logs \
  --statement-id cw-logs-trigger \
  --action lambda:InvokeFunction \
  --principal logs.amazonaws.com \
  --source-arn "arn:aws:logs:<REGION>:<ACCOUNT_ID>:log-group:<LOG_GROUP_NAME>:*"

Create a subscription filter to forward log events to the Lambda:

aws logs put-subscription-filter \
  --log-group-name <LOG_GROUP_NAME> \
  --filter-name otel-lambda-filter \
  --filter-pattern "" \
  --destination-arn "arn:aws:lambda:<REGION>:<ACCOUNT_ID>:function:otel-lambda-cloudwatch-logs"

An empty --filter-pattern forwards all log events. See Filter pattern syntax for filtering options.

Configurations

The following receiver configuration parameters are supported.

Name	Description
`s3::encoding`	Optional encoder to use for S3 event processing
`cloudwatch::encoding`	Optional encoder to use for CloudWatch event processing

Consider following notes on default behaviors:

When s3::encoding is not specified, the receiver defaults to preserving the S3 object content as-is for logs.
- The log record’s Body field will be a string type where the S3 object content is valid UTF-8, and otherwise will be a byte array.
When cloudwatch::encoding is not specified, the receiver defaults to parsing CloudWatch Logs messages to OpenTelemetry log records.
For metrics, the default behavior is to decode using awscloudwatchmetricstreams_encoding extension.

[!NOTE] The receiver supports end to end streaming utilizing encoding extension streaming capabilities. For extensions that does not support streaming, xstreamencoding wrapper will be used where full payload get processed at once.

Given below are example configurations for various use cases.

Example 1: VPC Flow Logs from S3

receivers:
  aws_lambda:
    s3:
      encoding: aws_logs_encoding

extensions:
  aws_logs_encoding:
    format: vpcflow
    vpcflow:
      file_format: plain-text

exporters:
  otlp_http:
    endpoint: "https://my-backend:443"

service:
  extensions:
    - aws_logs_encoding
  pipelines:
    logs:
      receivers: [aws_lambda]
      exporters: [otlp_http]

In this example, the awslambdareceiver is expected to be triggered when a VPC flow log is created at S3 bucket. The receiver retrieves the log file from S3 and decodes it using the aws_logs_encoding extension with the vpcflow format. Parsed logs are forwarded to an OTLP listener via the otlp_http exporter.

Example 2: ELB Access Logs from S3

receivers:
  aws_lambda:
    s3:
      encoding: aws_logs_encoding

extensions:
  aws_logs_encoding:
    format: elbaccess
    elbaccess:
      file_format: plain-text

exporters:
  otlp_http:
    endpoint: "https://my-backend:443"

service:
  extensions:
    - aws_logs_encoding
  pipelines:
    logs:
      receivers: [aws_lambda]
      exporters: [otlp_http]

Similar to the first example, this configuration is for collecting ELB access logs stored in S3.

Multi-Format S3 Configuration (encodings)

The encodings field enables routing different S3 object key patterns to different decoders within a single Lambda deployment. This is useful when a Lambda function receives events from:

A single S3 bucket that stores multiple log types — for example, VPC Flow Logs and CloudTrail logs written to the same bucket under different key prefixes.
Multiple S3 buckets with different log types — for example, one bucket for VPC Flow Logs and another for WAF logs, both configured to trigger the same Lambda function.

encoding (single, top-level) and encodings (list) are mutually exclusive — use one or the other. Each entry in encodings supports three fields:

Field	Required	Description
`name`	yes	Unique identifier for this entry. For known names (`vpcflow`, `cloudtrail`, etc.) the default `path_pattern` is applied automatically.
`encoding`	no	Extension ID of the decoder (e.g. `awslogs_encoding/vpcflow`). Omit to pass content through as raw bytes using the built-in default decoder.
`path_pattern`	no*	Prefix pattern matched against the S3 object key. `` matches one path segment. Omit to use the built-in default for known names. Use `""` as a catch-all.

* May be omitted only for built-in known names. For any other name, path_pattern must be set explicitly (use "*" for a catch-all). Each entry in encodings is evaluated in order of pattern specificity (more-specific patterns are matched first; "*" catch-all is matched last). Users may list entries in any order.

Combining encodings with extensions

The encoding field references a collector extension by its component ID. Each referenced extension must be declared in the extensions: block and listed under service.extensions. The following example decodes VPC Flow Logs and CloudTrail events into structured log records, and forwards anything else as raw bytes via the catch-all entry:

extensions:
  awslogs_encoding/vpcflow:
    format: vpcflow
    vpcflow:
      file_format: plain-text
  awslogs_encoding/cloudtrail:
    format: cloudtrail

receivers:
  aws_lambda:
    s3:
      encodings:
        - name: vpcflow
          encoding: awslogs_encoding/vpcflow     # decode VPC Flow Log fields into structured records
        - name: cloudtrail
          encoding: awslogs_encoding/cloudtrail  # decode CloudTrail JSON events into structured records
          path_pattern: "myorg/*/CloudTrail"     # optional: override default (AWSLogs/*/CloudTrail); omit to use the default
        - name: catchall
          path_pattern: "*"                      # forward anything else as raw bytes

Built-in default path patterns

The following well-known names have built-in default path patterns. When path_pattern is omitted for these names, the receiver uses the corresponding default.

Name	Default path pattern
`vpcflow`	`AWSLogs/*/vpcflowlogs`
`cloudtrail`	`AWSLogs/*/CloudTrail`
`elbaccess`	`AWSLogs/*/elasticloadbalancing`
`waf`	`AWSLogs/*/WAFLogs`
`networkfirewall`	`AWSLogs/*/network-firewall`

In the default patterns * matches exactly one path segment (the AWS account ID in standard AWS log paths). For any name not listed above, path_pattern must be specified explicitly.

Example 3: CloudWatch Logs using CloudWatch Subscription Filters

receivers:
  aws_lambda:

exporters:
  otlp_http:
    endpoint: "https://my-backend:443"

service:
  pipelines:
    logs:
      receivers: [aws_lambda]
      exporters: [otlp_http]

For this deployment configuration, when receiver is triggered by a CloudWatch Logs subscription filter, the CloudWatch messages will be extracted and converted to an OpenTelemetry log record. These logs then get forwarded to an OTLP listener via the otlp_http exporter.

Example 4: Arbitrary S3 content (logs or metrics)

receivers:
  aws_lambda:

exporters:
  otlp_http:
    endpoint: "https://my-backend:443"

service:
  pipelines:
    logs:
      receivers: [aws_lambda]
      exporters: [otlp_http]

For this deployment configuration, when receiver is triggered by an S3 event,

Logs: Content of the S3 object will be added to an OpenTelemetry log record. If content is string, then it will be added as-is.
Metrics: Metrics will be decoded using awscloudwatchmetricstreams_encoding extension.

AWS Permissions

The Lambda function requires the following IAM permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::your-log-bucket/*"
    }
  ]
}

Error Handling

Detailed error information is logged for troubleshooting These logs can be views via the configured CloudWatch Logs group for the Lambda function.
Error retrying Error retrying can be configured through the Lambda deployment setting. Read more about at AWS error handling for asynchronous invocations.
Retaining failed records This component supports replaying retained failure records stored at S3. Read more about retaining records at Capture records of Lambda Async Invocations.

Error replaying from S3

When an S3 bucket is configured as the destination for retaining failed Lambda records, the receiver supports replaying those failed events for reprocessing. To enable this feature, set the failure_bucket_arn configuration to the ARN of your S3 bucket used as the Lambda failure destination.

receivers:
  aws_lambda:
    s3:
      encoding: aws_logs_encoding
    failure_bucket_arn: "arn:aws:s3:::example"

With required configuration present, receiver accepts a custom event to trigger replaying failed events. Consider the event structure below,

{
  "replayFailedEvents": {
    "dryrun": false,
    "removeOnSuccess": true
  }
}

JSON key replayFailedEvents defines the custom event type for replaying failed events. The table below explains supported options,

Option	Description	Default
dryrun	Run the command without processing. Useful to understand details about replaying error files	false
removeOnSuccess	Configure whether to remove error event from S3 error destination, if processing is successful	true

[!NOTE]
It is recommended to use “dryrun” mode to validate the number of replayable errors in the error destination bucket. If there are many errors, the Lambda invocation may time out before processing all error entries. If a timeout occur, you will need to run the custom event multiple times to fully process all error events from the bucket.

Running with AWS CLI

First, obtain the name of the deployed Lambda function from your deployment. Then, invoke the Lambda with the following command:

aws lambda invoke \
  --function-name <LAMBDA_DEPLOYMENT_NAME> \
  --payload '{ "replayFailedEvents": {}}' \
  --cli-binary-format raw-in-base64-out /dev/null

If successful, you should see "StatusCode": 200 in the output. Check the CloudWatch logs for detailed information.

[!NOTE] Using AWS CLI, you can use --timeout option to increase currently configured Lambda timeout for custom invocations. Also note that errors resulting from this manual trigger are not retained back to S3 failure destination. This is because Lambda only retains errors for asynchronous invocations.

To perform a dry run, use the following command with dryrun set to true:

aws lambda invoke \
  --function-name <LAMBDA_DEPLOYMENT_NAME> \
  --payload '{ "replayFailedEvents": { "dryrun": true }}' \
  --cli-binary-format raw-in-base64-out /dev/null

This allows you to see how many error events are available for replay without actually processing them.

Configuration

Example Configuration

aws_lambda/aws_logs_encoding:
  s3:
    encoding: aws_logs_encoding
  cloudwatch:
    encoding: aws_logs_encoding

aws_lambda/json_log_encoding:
  s3:
    encoding: json_log_encoding

aws_lambda/empty_encoding:

aws_lambda/with_failure_arn:
  failure_bucket_arn: arn:aws:s3:::example

aws_lambda/s3_multi_encoding:
  s3:
    encodings:
      - name: vpcflow
        encoding: awslogs_encoding/vpcflow
      - name: cloudtrail
        encoding: awslogs_encoding/cloudtrail
      - name: catchall
        path_pattern: "*"

Last generated: 2026-06-01

​Awslambda Receiver

​Supported Telemetry

​Overview

​Overview

​How It Works

​Event handling

​S3 Event handling

​CloudWatch Logs subscription

​Deployment

​Build the collector binary

​Package and deploy

​Set up the CloudWatch Logs trigger

​Configurations

​Example 1: VPC Flow Logs from S3

​Example 2: ELB Access Logs from S3

​Multi-Format S3 Configuration (encodings)

​Combining encodings with extensions

​Built-in default path patterns

​Example 3: CloudWatch Logs using CloudWatch Subscription Filters

​Example 4: Arbitrary S3 content (logs or metrics)

​AWS Permissions

​Error Handling

​Error replaying from S3

​Running with AWS CLI

​Configuration

​Example Configuration

Awslambda Receiver

Supported Telemetry

Overview

Overview

How It Works

Event handling

S3 Event handling

CloudWatch Logs subscription

Deployment

Build the collector binary

Package and deploy

Set up the CloudWatch Logs trigger

Configurations

Example 1: VPC Flow Logs from S3

Example 2: ELB Access Logs from S3

Multi-Format S3 Configuration (encodings)

Combining encodings with extensions

Built-in default path patterns

Example 3: CloudWatch Logs using CloudWatch Subscription Filters

Example 4: Arbitrary S3 content (logs or metrics)

AWS Permissions

Error Handling

Error replaying from S3

Running with AWS CLI

Configuration

Example Configuration