Skip to main content

​
Googlecloudstorage Exporter

Status Available in: contrib Maintainers: @constanca-m, @braydonk Source: opentelemetry-collector-contrib

​
Supported Telemetry

Logs Traces

​
Overview

This exporter writes received OpenTelemetry data to a cloud storage bucket.

​
Configuration

NameDescriptionRequiredDefault
encodingThe encoding extension ID to use for marshaling logs and traces. If left empty, plog.JSONMarshaler will be used for logs and ptrace.JSONMarshaler will be used for traces.No
bucket.project_idThe project where the bucket will be created or where it exists. If left empty, it will query the metadata endpoint. It requires the collector to be running in a Google Cloud environment.No
bucket.nameName for the bucket storage.Yes
bucket.file_prefixPrefix for the created filename. This prefix is applied after the partition path (if any).Nologs
bucket.partitionConfiguration for time-based partitioning. See below for details.No
bucket.reuse_if_existsControls bucket creation behavior. If true, checks if bucket exists and uses it (requires storage.buckets.get permission on the bucket); fails if bucket doesn’t exist. If false, attempts to create bucket; fails if bucket already exists (requires storage.buckets.create permission at project level). Set to true when the service account lacks project-level bucket creation permissions but has bucket-level permissions.Nofalse
bucket.regionRegion where the bucket will be created or where it exists. If left empty, it will query the metadata endpoint. It requires the collector to be running in a Google Cloud environment.Yes
bucket.compressionCompression algorithm used to compress data before uploading. Valid values are gzip, zstd, or no value set for no compression.No
timeoutTime to wait per individual attempt to send data to the backend. See exporterhelper docs for details.No5s
retry_on_failureConfiguration for how to retry failed requests. See exporterhelper docs for details.No
sending_queueConfiguration for the internal sending queue. See exporterhelper docs for details.No

​
Partition Configuration

The bucket.partition configuration allows you to organize files into time-based folders.
NameDescriptionRequiredDefault
formatTime format string for time-based partitions. If set, formatted UTC time is prepended to the filename. Supports strftime format (e.g., year=%Y/month=%m).No
prefixPrefix for the partition folder structure. If set, this value is prepended to the partition path.No

​
Example

Here is an example configuration for this exporter: 
exporters:
  google_cloud_storage:
    encoding: text_encoding
    bucket:
      name: bucket-test
      project_id: project-test
      reuse_if_exists: true
      region: europe-west1
      file_prefix: app-logs
      partition:
        prefix: archive
        format: "year=%Y/month=%m/day=%d/hour=%H"

extensions:
  # text encoding to ensure only the body is placed in the bucket
  text_encoding:

​
Resiliency Settings

This exporter supports standard OpenTelemetry Collector resiliency settings. You can configure timeouts, retry behaviors, and a sending queue to ensure reliable data delivery to Google Cloud Storage. 
exporters:
  google_cloud_storage:
    bucket:
      name: bucket-test
      region: europe-west1
    timeout: 5s
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 1000
For more details on how to configure these specific sections, refer to the exporterhelper configuration documentation.

​
Retry Behavior

This exporter utilizes two independent layers of retries: Google Cloud SDK Retries (always on): The underlying Google Cloud Go SDK automatically retries transient errors (like network blips) in the background. By default, the SDK handles its own exponential backoff and idempotency checks. Exporter Helper Retries (optional via retry_on_failure): This is the OpenTelemetry-level retry mechanism. It integrates with the sending queue to ensure data is not dropped during prolonged outages. Important Notes on Retries: If the Google Cloud SDK determines an error is permanent (e.g., unauthorized access) or not idempotent, the exporter will immediately wrap it as a permanent error. This halts the retry_on_failure loop and drops the payload to prevent infinite queue blocking. You can find more information on what the SDK considers retryable in the Google Cloud Storage Retry Strategy documentation. Because file names are generated dynamically using a UUID and timestamp, each retry attempt by the OpenTelemetry retry_on_failure mechanism generates a new filename. In rare cases (such as when GCS commits an object but returns a transient error before the client receives acknowledgment) this can result in duplicate blobs in GCS.

​
Compression Example

exporters:
  google_cloud_storage:
    bucket:
      name: compressed-logs-bucket
      project_id: my-project
      region: us-central1
      compression: gzip
      file_prefix: logs
      partition:
        format: "year=%Y/month=%m/day=%d"

​
Using with Bucket-Level Permissions Only

When the service account lacks project-level bucket creation permissions but has bucket-level permissions: 
exporters:
  google_cloud_storage:
    bucket:
      name: existing-bucket
      project_id: my-project
      region: us-central1
      reuse_if_exists: true
      file_prefix: collector-logs
With reuse_if_exists: true, the exporter checks if the bucket exists using storage.buckets.get permission on that specific bucket (not project-level). If the bucket doesn’t exist, the exporter will fail with an error. This allows service accounts with only bucket-level permissions to work without requiring project-level bucket creation permissions.

​
Configuration

​
Example Configuration

google_cloud_storage:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id

google_cloud_storage/with_partition:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    partition:
      format: year=%Y
      prefix: my-logs

google_cloud_storage/with_resiliency:
  encoding: test
  timeout: 3s
  sending_queue:
    num_consumers: 7
    queue_size: 123
  retry_on_failure:
    enabled: true
    initial_interval: 1s
    max_interval: 5s
    max_elapsed_time: 20s
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id

google_cloud_storage/empty_bucket_name:
  encoding: test
  bucket:
    region: test-region
    project_id: test-project-id

google_cloud_storage/invalid_partition_format:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    partition:
      format: year=%invalid

google_cloud_storage/with_gzip_compression:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    compression: gzip

google_cloud_storage/with_zstd_compression:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    compression: zstd

google_cloud_storage/unsupported_compression:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    compression: snappy
Last generated: 2026-06-01