Skip to main content

​
Googlecloudstorage Exporter

Status Available in: contrib Maintainers: @constanca-m, @braydonk Source: opentelemetry-collector-contrib

​
Supported Telemetry

Logs Traces

​
Overview

This exporter writes received OpenTelemetry data to a cloud storage bucket.

​
Configuration

NameDescriptionRequiredDefault
encodingThe encoding extension ID to use for marshaling logs and traces. If left empty, plog.JSONMarshaler will be used for logs and ptrace.JSONMarshaler will be used for traces.No
bucket.project_idThe project where the bucket will be created or where it exists. If left empty, it will query the metadata endpoint. It requires the collector to be running in a Google Cloud environment.No
bucket.nameName for the bucket storage.Yes
bucket.file_prefixPrefix for the created filename. This prefix is applied after the partition path (if any).Nologs
bucket.partitionConfiguration for time-based partitioning. See below for details.No
bucket.reuse_if_existsControls bucket creation behavior. If true, checks if bucket exists and uses it (requires storage.buckets.get permission on the bucket); fails if bucket doesn’t exist. If false, attempts to create bucket; fails if bucket already exists (requires storage.buckets.create permission at project level). Set to true when the service account lacks project-level bucket creation permissions but has bucket-level permissions.Nofalse
bucket.regionRegion where the bucket will be created or where it exists. If left empty, it will query the metadata endpoint. It requires the collector to be running in a Google Cloud environment.Yes
bucket.compressionCompression algorithm used to compress data before uploading. Valid values are gzip, zstd, or no value set for no compression.No
resource_attrs_to_gcsMapping of GCS upload configuration values to resource attribute values. See resource_attrs_to_gcs below.No
timeoutTime to wait per individual attempt to send data to the backend. See exporterhelper docs for details.No5s
retry_on_failureConfiguration for how to retry failed requests. See exporterhelper docs for details.No
sending_queueConfiguration for the internal sending queue. See exporterhelper docs for details.No

​
Partition Configuration

The bucket.partition configuration allows you to organize files into time-based folders.
NameDescriptionRequiredDefault
formatTime format string for time-based partitions. If set, formatted UTC time is prepended to the filename. Supports strftime format (e.g., year=%Y/month=%m).No
prefixPrefix for the partition folder structure. If set, this value is prepended to the partition path.No

​
resource_attrs_to_gcs

NameDescriptionRequiredDefault
prefixResource attribute whose value is inserted as a partition path segment between bucket.partition.prefix and the time-based bucket.partition.format. If the attribute is present on the data, its value is used; otherwise the segment is omitted.No
When resource_attrs_to_gcs.prefix is configured, a partition segment is derived from the specified resource attribute, allowing data to be routed into per-attribute folders (for example, one folder per host or service). The resulting layout is <partition.prefix>/<attribute value>/<partition.format>/..., so the existing bucket.partition.prefix is preserved and the attribute segment is appended after it. The attribute value is stringified, so non-string attributes are converted to their string form (for example, the int 42 becomes 42 and a bool becomes true); complex types such as maps or slices render as their textual representation and are not recommended. Because resource attribute values are arbitrary and may be untrusted, the value is normalized with the standard library path.Clean before being used as an object-name segment: duplicate slashes are collapsed and empty, . and .. segments are removed (so the value cannot traverse above its segment root). Leading and trailing / are trimmed, while internal / are preserved so a value can span multiple folder levels (e.g. service-a/logs). If the value normalizes to an empty string, no extra segment is added. Note that high-cardinality attribute values produce a correspondingly large number of object key prefixes.
Note: the attribute value is read from the first resource of each exported batch. If a batch can contain multiple distinct attribute values and you need strict per-value routing, separate the data upstream first using the routingconnector or groupbyattrsprocessor.

​
Example

Here is an example configuration for this exporter: 
exporters:
  google_cloud_storage:
    encoding: text_encoding
    bucket:
      name: bucket-test
      project_id: project-test
      reuse_if_exists: true
      region: europe-west1
      file_prefix: app-logs
      partition:
        prefix: archive
        format: "year=%Y/month=%m/day=%d/hour=%H"

extensions:
  # text encoding to ensure only the body is placed in the bucket
  text_encoding:

​
Routing by Resource Attribute

This configuration writes each service’s data into its own folder by deriving a partition segment from the service.name resource attribute, while keeping a static storage prefix: 
exporters:
  google_cloud_storage:
    bucket:
      name: archive-bucket
      region: europe-west1
      file_prefix: logs
      partition:
        prefix: storage
        format: "%Y-%m-%d/%H"
    resource_attrs_to_gcs:
      prefix: service.name
Given a resource with service.name = serviceA, an object is written to: 
storage/serviceA/2026-06-17/14/logs_<uuid>
If the service.name attribute is absent, the attribute segment is omitted and the object is written to storage/2026-06-17/14/logs_<uuid>.

​
Resiliency Settings

This exporter supports standard OpenTelemetry Collector resiliency settings. You can configure timeouts, retry behaviors, and a sending queue to ensure reliable data delivery to Google Cloud Storage. 
exporters:
  google_cloud_storage:
    bucket:
      name: bucket-test
      region: europe-west1
    timeout: 5s
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 1000
For more details on how to configure these specific sections, refer to the exporterhelper configuration documentation.

​
Retry Behavior

This exporter utilizes two independent layers of retries: Google Cloud SDK Retries (always on): The underlying Google Cloud Go SDK automatically retries transient errors (like network blips) in the background. By default, the SDK handles its own exponential backoff and idempotency checks. Exporter Helper Retries (optional via retry_on_failure): This is the OpenTelemetry-level retry mechanism. It integrates with the sending queue to ensure data is not dropped during prolonged outages. Important Notes on Retries: If the Google Cloud SDK determines an error is permanent (e.g., unauthorized access) or not idempotent, the exporter will immediately wrap it as a permanent error. This halts the retry_on_failure loop and drops the payload to prevent infinite queue blocking. You can find more information on what the SDK considers retryable in the Google Cloud Storage Retry Strategy documentation. Because file names are generated dynamically using a UUID and timestamp, each retry attempt by the OpenTelemetry retry_on_failure mechanism generates a new filename. In rare cases (such as when GCS commits an object but returns a transient error before the client receives acknowledgment) this can result in duplicate blobs in GCS.

​
Compression Example

exporters:
  google_cloud_storage:
    bucket:
      name: compressed-logs-bucket
      project_id: my-project
      region: us-central1
      compression: gzip
      file_prefix: logs
      partition:
        format: "year=%Y/month=%m/day=%d"

​
Using with Bucket-Level Permissions Only

When the service account lacks project-level bucket creation permissions but has bucket-level permissions: 
exporters:
  google_cloud_storage:
    bucket:
      name: existing-bucket
      project_id: my-project
      region: us-central1
      reuse_if_exists: true
      file_prefix: collector-logs
With reuse_if_exists: true, the exporter checks if the bucket exists using storage.buckets.get permission on that specific bucket (not project-level). If the bucket doesn’t exist, the exporter will fail with an error. This allows service accounts with only bucket-level permissions to work without requiring project-level bucket creation permissions.

​
Configuration

​
Example Configuration

google_cloud_storage:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id

google_cloud_storage/with_partition:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    partition:
      format: year=%Y
      prefix: my-logs

google_cloud_storage/with_resource_attrs_to_gcs:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    partition:
      format: "%Y-%m-%d/%H"
      prefix: storage
  resource_attrs_to_gcs:
    prefix: service.name

google_cloud_storage/with_resiliency:
  encoding: test
  timeout: 3s
  sending_queue:
    num_consumers: 7
    queue_size: 123
  retry_on_failure:
    enabled: true
    initial_interval: 1s
    max_interval: 5s
    max_elapsed_time: 20s
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id

google_cloud_storage/empty_bucket_name:
  encoding: test
  bucket:
    region: test-region
    project_id: test-project-id

google_cloud_storage/invalid_partition_format:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    partition:
      format: year=%invalid

google_cloud_storage/with_gzip_compression:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    compression: gzip

google_cloud_storage/with_zstd_compression:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    compression: zstd

google_cloud_storage/unsupported_compression:
  encoding: test
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
    compression: snappy

google_cloud_storage/with_universe_domain:
  encoding: test
  universe_domain: apis.example.com
  bucket:
    region: test-region
    name: test-bucket
    project_id: test-project-id
Last generated: 2026-07-06