> ## Documentation Index > Fetch the complete documentation index at: https://otel.fyi/llms.txt > Use this file to discover all available pages before exploring further. # Awslambda > OpenTelemetry receiver for Awslambda # Awslambda Receiver ![Status](https://img.shields.io/badge/status-alpha-red) **Available in:** `contrib` **Maintainers:** [@MichaelKatsoulis](https://github.com/MichaelKatsoulis), [@Kavindu-Dodan](https://github.com/Kavindu-Dodan), [@axw](https://github.com/axw), [@pjanotti](https://github.com/pjanotti) **Source:** [opentelemetry-collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awslambdareceiver) ## Supported Telemetry ![Logs](https://img.shields.io/badge/logs-alpha-blue) ![Metrics](https://img.shields.io/badge/metrics-alpha-green) ## Overview ## Overview A receiver for collecting logs & metrics from AWS services via Lambda invocations. AWS Lambda is a popular serverless service used extensively for event-driven architectures. Many AWS services (S3, CloudWatch, SNS, SQS) can trigger Lambda functions, making it an ideal entry point for collecting data from AWS services. This receiver is designed to run as part of an OpenTelemetry Collector deployed as an AWS Lambda function. The `awslambdareceiver` enables users to: * Collect logs & metrics stored at various AWS services as OTel native log records & metric data points * Collect custom logs via AWS services that can trigger Lambda functions * Decode/Unmarshal AWS-specific log formats using OpenTelemetry encoding extensions * Leverage OpenTelemetry's processors to further enrich, filter, or transform collected data before exporting ## How It Works The `awslambdareceiver` operates as follows: 1. Accepts Lambda Invocations 2. Identifies the event source (S3, CloudWatch, etc.) 3. Uses configured encoding extensions to parse the data 4. Creates OpenTelemetry records matching the data type (logs, metrics) 5. Forward derived records to the next component in the pipeline (processors, exporters) ## Event handling The receiver automatically detects the event source based on the Lambda invocation message format. Table below summarizes supported signals and their sources: | Signal | Sources | | ------- | ---------------------------------------- | | Logs | S3, CloudWatch Logs subscription, Custom | | Metrics | S3 | Sections below summarize how each event source is handled. ### S3 Event handling S3 events are handled in the following manner: * Receive S3 event notification (for example, using Lambda trigger on `s3:ObjectCreated:*`) * Download S3 object payload * Decode payload using the configured encoding extension * Default encoding: Preserve S3 object content as-is * Custom encoding: Use specified encoding extension (for example, `aws_logs_encoding` for AWS log formats) * Metrics use `awscloudwatchmetricstreams_encoding` extension by default The following metadata is available through `client.Info` for both logs and metrics S3 events. This metadata is added to the request context and can be accessed by any downstream component in the pipeline (for example, processors): | Metadata Key | Description | | ------------------ | -------------------- | | cloud.region | The S3 bucket region | | aws.s3.bucket.name | The S3 bucket name | | aws.s3.bucket.arn | The S3 bucket ARN | | aws.s3.key | The S3 object key | > \[!NOTE] > Static metadata such as `cloud.provider` are not available through `client.Info`. > These can be added using processors (for example, the resource processor). ### CloudWatch Logs subscription handling CloudWatch Logs events are handled in the following manner: * Receive CloudWatch Logs subscription filter event * Parse the CloudWatch Logs message (note - unlike S3 events, the payload is included in the event) * Decode payload using the configured encoding extension * Default encoding: Parse CloudWatch Logs messages to OpenTelemetry log records * Custom encoding: Use specified encoding extension (for example, `aws_logs_encoding` for AWS log formats) The following metadata is available through `client.Info` for CloudWatch Logs events. This metadata is added to the request context and can be accessed by any downstream component in the pipeline (for example, processors): | Metadata Key | Description | | -------------------- | -------------------------- | | cloud.account.id | AWS account ID | | aws.log.group.names | CloudWatch log group name | | aws.log.stream.names | CloudWatch log stream name | > \[!NOTE] > Static metadata such as `cloud.provider` are not available through `client.Info`. > These can be added using processors (for example, the resource processor). ### Custom event handling Events that are not recognized as S3 or CloudWatch Logs events are handled as custom events. For custom events, only the logs signal is supported. Custom events are handled in the following manner: * Receive the Lambda invocation payload * Decode the payload using the configured encoding extension If an encoding extension is not configured, then the receiver will error out and return the error. The encoding used for custom events is configured through the `custom::encoding` option. See the [Configurations](#configurations) section for details. ## Deployment The following example shows how to deploy the collector as an AWS Lambda function that receives logs from a CloudWatch Logs subscription filter, using the AWS CLI. ### Build the collector binary Use the [OpenTelemetry Collector Builder (OCB)](https://opentelemetry.io/docs/collector/custom-collector/) to build a minimal binary containing only the required components. This keeps the binary small enough for Lambda's deployment package size limit. Create a builder manifest: ```shell theme={null} cat > builder-config.yaml << 'EOF' dist: name: collector output_path: . otelcol_version: 0.149.0 receivers: - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awslambdareceiver v0.149.0 exporters: - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.149.0 EOF ``` Build the binary for Lambda: ```shell theme={null} CGO_ENABLED=0 GOOS=linux GOARCH=arm64 builder --config=builder-config.yaml ``` ### Package and deploy The Lambda `provided.al2023` runtime executes a binary named `bootstrap`. Create a wrapper script that passes the `--config` flag to the collector: ```shell theme={null} cat > bootstrap << 'EOF' #!/bin/sh exec /var/task/collector --config /var/task/collector-config.yaml "$@" EOF chmod +x bootstrap ``` Write a collector configuration: ```shell theme={null} cat > collector-config.yaml << 'EOF' receivers: aws_lambda: exporters: debug: verbosity: detailed service: pipelines: logs: receivers: [aws_lambda] exporters: [debug] EOF zip function.zip bootstrap collector collector-config.yaml ``` Create an IAM role for the Lambda function: ```shell theme={null} aws iam create-role \ --role-name otel-lambda-role \ --assume-role-policy-document '{ "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole" }] }' aws iam attach-role-policy \ --role-name otel-lambda-role \ --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole ``` Create the Lambda function: ```shell theme={null} aws lambda create-function \ --function-name otel-lambda-cloudwatch-logs \ --runtime provided.al2023 \ --handler bootstrap \ --architectures arm64 \ --zip-file fileb://function.zip \ --role arn:aws:iam:::role/otel-lambda-role \ --timeout 30 ``` ### Set up the CloudWatch Logs trigger Grant CloudWatch Logs permission to invoke the Lambda function: ```shell theme={null} aws lambda add-permission \ --function-name otel-lambda-cloudwatch-logs \ --statement-id cw-logs-trigger \ --action lambda:InvokeFunction \ --principal logs.amazonaws.com \ --source-arn "arn:aws:logs:::log-group::*" ``` Create a subscription filter to forward log events to the Lambda: ```shell theme={null} aws logs put-subscription-filter \ --log-group-name \ --filter-name otel-lambda-filter \ --filter-pattern "" \ --destination-arn "arn:aws:lambda:::function:otel-lambda-cloudwatch-logs" ``` An empty `--filter-pattern` forwards all log events. See [Filter pattern syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html) for filtering options. ### Configurations The following receiver configuration parameters are supported. | Name | Description | | :---------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------- | | `s3::encoding` | Optional encoder to use for S3 event processing | | `s3::encodings` | Optional list of path-based encoders for multi-format S3 routing (see [Multi-Format S3 Configuration](#multi-format-s3-configuration-encodings)) | | `s3::access_key_id` | Optional static AWS access key ID for S3 access | | `s3::secret_access_key` | Optional static AWS secret access key for S3 access | | `s3::session_token` | Optional static AWS session token for S3 access | | `cloudwatch::encoding` | Optional encoder to use for CloudWatch event processing | | `custom::encoding` | Optional encoder to use for custom event processing | | `failure_bucket_arn` | Optional S3 bucket ARN holding failed Lambda records for replay | Consider following notes on default behaviors: * When `s3::encoding` is not specified, the receiver defaults to preserving the S3 object content as-is for logs. * The log record's `Body` field will be a string type where the S3 object content is valid UTF-8, and otherwise will be a byte array. * When `cloudwatch::encoding` is not specified, the receiver defaults to parsing CloudWatch Logs messages to OpenTelemetry log records. * For metrics, the default behavior is to decode using `awscloudwatchmetricstreams_encoding` extension. > \[!NOTE] > The receiver supports end to end streaming utilizing encoding extension streaming capabilities. > For extensions that does not support streaming, xstreamencoding wrapper will be used where full payload get processed at once. Given below are example configurations for various use cases. ### Example 1: VPC Flow Logs from S3 ```yaml theme={null} receivers: aws_lambda: s3: encoding: aws_logs_encoding extensions: aws_logs_encoding: format: vpcflow vpcflow: file_format: plain-text exporters: otlp_http: endpoint: "https://my-backend:443" service: extensions: - aws_logs_encoding pipelines: logs: receivers: [aws_lambda] exporters: [otlp_http] ``` In this example, the `awslambdareceiver` is expected to be triggered when a VPC flow log is created at S3 bucket. The receiver retrieves the log file from S3 and decodes it using the `aws_logs_encoding` extension with the `vpcflow` format. Parsed logs are forwarded to an OTLP listener via the `otlp_http` exporter. ### Example 2: ELB Access Logs from S3 ```yaml theme={null} receivers: aws_lambda: s3: encoding: aws_logs_encoding extensions: aws_logs_encoding: format: elbaccess elbaccess: file_format: plain-text exporters: otlp_http: endpoint: "https://my-backend:443" service: extensions: - aws_logs_encoding pipelines: logs: receivers: [aws_lambda] exporters: [otlp_http] ``` Similar to the first example, this configuration is for collecting ELB access logs stored in S3. ### Multi-Format S3 Configuration (encodings) The `encodings` field enables routing different S3 object key patterns to different decoders within a single Lambda deployment. This is useful when a Lambda function receives events from: * **A single S3 bucket that stores multiple log types** — for example, VPC Flow Logs and CloudTrail logs written to the same bucket under different key prefixes. * **Multiple S3 buckets with different log types** — for example, one bucket for VPC Flow Logs and another for WAF logs, both configured to trigger the same Lambda function. `encoding` (single, top-level) and `encodings` (list) are mutually exclusive — use one or the other. Each entry in `encodings` supports three fields: | Field | Required | Description | | -------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | yes | Unique identifier for this entry. For known names (`vpcflow`, `cloudtrail`, etc.) the default `path_pattern` is applied automatically. | | `encoding` | no | Extension ID of the decoder (e.g. `awslogs_encoding/vpcflow`). Omit to pass content through as raw bytes using the built-in default decoder. | | `path_pattern` | no\* | Prefix pattern matched against the S3 object key. `*` matches one path segment. Omit to use the built-in default for known names. Use `"*"` as a catch-all. | \* May be omitted only for built-in known names. For any other name, `path_pattern` must be set explicitly (use `"*"` for a catch-all). Each entry in `encodings` is evaluated in order of pattern specificity (more-specific patterns are matched first; `"*"` catch-all is matched last). Users may list entries in any order. #### Combining encodings with extensions The `encoding` field references a collector extension by its component ID. Each referenced extension must be declared in the `extensions:` block and listed under `service.extensions`. The following example decodes VPC Flow Logs and CloudTrail events into structured log records, and forwards anything else as raw bytes via the catch-all entry: ```yaml theme={null} extensions: awslogs_encoding/vpcflow: format: vpcflow vpcflow: file_format: plain-text awslogs_encoding/cloudtrail: format: cloudtrail receivers: aws_lambda: s3: encodings: - name: vpcflow encoding: awslogs_encoding/vpcflow # decode VPC Flow Log fields into structured records - name: cloudtrail encoding: awslogs_encoding/cloudtrail # decode CloudTrail JSON events into structured records path_pattern: "myorg/*/CloudTrail" # optional: override default (AWSLogs/*/CloudTrail); omit to use the default - name: catchall path_pattern: "*" # forward anything else as raw bytes ``` #### Built-in default path patterns The following well-known names have built-in default path patterns. When `path_pattern` is omitted for these names, the receiver uses the corresponding default. | Name | Default path pattern | | ----------------- | -------------------------------- | | `vpcflow` | `AWSLogs/*/vpcflowlogs` | | `cloudtrail` | `AWSLogs/*/CloudTrail` | | `elbaccess` | `AWSLogs/*/elasticloadbalancing` | | `waf` | `AWSLogs/*/WAFLogs` | | `networkfirewall` | `AWSLogs/*/network-firewall` | In the default patterns `*` matches exactly one path segment (the AWS account ID in standard AWS log paths). For any name not listed above, `path_pattern` must be specified explicitly. ### Example 3: CloudWatch Logs using CloudWatch Subscription Filters ```yaml theme={null} receivers: aws_lambda: exporters: otlp_http: endpoint: "https://my-backend:443" service: pipelines: logs: receivers: [aws_lambda] exporters: [otlp_http] ``` For this deployment configuration, when receiver is triggered by a CloudWatch Logs subscription filter, the CloudWatch messages will be extracted and converted to an OpenTelemetry log record. These logs then get forwarded to an OTLP listener via the `otlp_http` exporter. ### Example 4: Arbitrary S3 content (logs or metrics) ```yaml theme={null} receivers: aws_lambda: exporters: otlp_http: endpoint: "https://my-backend:443" service: pipelines: logs: receivers: [aws_lambda] exporters: [otlp_http] ``` For this deployment configuration, when receiver is triggered by an S3 event, * Logs: Content of the S3 object will be added to an OpenTelemetry log record. If content is string, then it will be added as-is. * Metrics: Metrics will be decoded using `awscloudwatchmetricstreams_encoding` extension. ## AWS Permissions The Lambda function requires the following IAM permissions: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": "arn:aws:s3:::your-log-bucket/*" } ] } ``` ## Error Handling * Detailed error information is logged for troubleshooting These logs can be views via the configured CloudWatch Logs group for the Lambda function. * Error retrying Error retrying can be configured through the Lambda deployment setting. Read more about at [AWS error handling for asynchronous invocations](https://docs.aws.amazon.com/lambda/latest/dg/invocation-async-configuring.html). * Retaining failed records This component supports replaying retained failure records stored at S3. Read more about retaining records at [Capture records of Lambda Async Invocations](https://docs.aws.amazon.com/lambda/latest/dg/invocation-async-retain-records.html). ### Error replaying from S3 When an S3 bucket is configured as the destination for retaining failed Lambda records, the receiver supports replaying those failed events for reprocessing. To enable this feature, set the `failure_bucket_arn` configuration to the ARN of your S3 bucket used as the Lambda failure destination. ```yaml theme={null} receivers: aws_lambda: s3: encoding: aws_logs_encoding failure_bucket_arn: "arn:aws:s3:::example" ``` With required configuration present, receiver accepts a custom event to trigger replaying failed events. Consider the event structure below, ```json theme={null} { "replayFailedEvents": { "dryrun": false, "removeOnSuccess": true } } ``` JSON key `replayFailedEvents` defines the custom event type for replaying failed events. The table below explains supported options, | Option | Description | Default | | --------------- | ---------------------------------------------------------------------------------------------- | ------- | | dryrun | Run the command without processing. Useful to understand details about replaying error files | false | | removeOnSuccess | Configure whether to remove error event from S3 error destination, if processing is successful | true | > \[!NOTE]\ > It is recommended to use "dryrun" mode to validate the number of replayable errors in the error destination bucket. > If there are many errors, the Lambda invocation may time out before processing all error entries. > If a timeout occur, you will need to run the custom event multiple times to fully process all error events from the bucket. ### Running with AWS CLI First, obtain the name of the deployed Lambda function from your deployment. Then, invoke the Lambda with the following command: ```shell theme={null} aws lambda invoke \ --function-name \ --payload '{ "replayFailedEvents": {}}' \ --cli-binary-format raw-in-base64-out /dev/null ``` If successful, you should see `"StatusCode": 200` in the output. Check the CloudWatch logs for detailed information. > \[!NOTE] > Using AWS CLI, you can use `--timeout` option to increase currently configured Lambda timeout for custom invocations. > Also note that errors resulting from this manual trigger are not retained back to S3 failure destination. > This is because Lambda only retains errors for asynchronous invocations. To perform a dry run, use the following command with `dryrun` set to `true`: ```shell theme={null} aws lambda invoke \ --function-name \ --payload '{ "replayFailedEvents": { "dryrun": true }}' \ --cli-binary-format raw-in-base64-out /dev/null ``` This allows you to see how many error events are available for replay without actually processing them. ## Configuration ### Example Configuration ```yaml theme={null} aws_lambda/aws_logs_encoding: s3: encoding: aws_logs_encoding cloudwatch: encoding: aws_logs_encoding aws_lambda/json_log_encoding: s3: encoding: json_log_encoding aws_lambda/empty_encoding: aws_lambda/with_failure_arn: failure_bucket_arn: arn:aws:s3:::example aws_lambda/s3_multi_encoding: s3: encodings: - name: vpcflow encoding: awslogs_encoding/vpcflow - name: cloudtrail encoding: awslogs_encoding/cloudtrail - name: catchall path_pattern: "*" aws_lambda/custom_event: custom: encoding: custom_encoding aws_lambda/s3_with_aws_options: s3: encoding: aws_logs_encoding access_key_id: "key" secret_access_key: "accessKey" session_token: "session" ``` *** *Last generated: 2026-07-06*