​

Googlecloudpubsub Receiver

​

Supported Telemetry

​

Overview

⚠️ This is a community-provided module. It has been developed and extensively tested at Collibra, but it is not officially supported by GCP.

• project (Optional): The Google Cloud Project of the client connects to.
• subscription (Required): The subscription name to receive OTLP data from. The subscription name should be a fully qualified resource name (eg: projects/otel-project/subscriptions/otlp).
• encoding (Optional): The encoding that will be used to received data from the subscription. This can either be otlp_proto_trace, otlp_proto_metric, otlp_proto_log or and encoding extension (see encoding). This will only be used as a fallback, when no content-type attribute is present.
• compression (Optional): The compression that will be used on received data from the subscription. When set it can only be gzip. This will only be used as a fallback, when no content-encoding attribute is present.
• endpoint (Optional): Override the default Pubsub Endpoint, useful when connecting to the PubSub emulator instance or switching between global and regional service endpoints.
• insecure (Optional): allows performing “insecure” SSL connections and transfers, useful when connecting to a local emulator instance. Only has effect if Endpoint is not ""
• ignore_encoding_error (Optional): Ignore errors when the configured encoder fails to decoding a PubSub messages. It’s advised to set this to true when using a custom encoder, and use receiver.googlecloudpubsub.encoding_error metric to monitor the number of errors. Ignoring the error will cause the receiver to drop the message.
• flow_control (Optional): Allows to fine tune the flow control settings for the subscription stream. See Flow control for more details.

receivers:
  googlecloudpubsub:
    project: otel-project
    subscription: projects/otel-project/subscriptions/otlp-logs

​

Encoding

extensions:
  text_encoding:
    encoding: utf8
    unmarshaling_separator: "\r?\n"

service:
  extensions: [text_encoding]
  pipelines:
    logs:
      receivers: [googlecloudpubsub]
      processors: []
      exporters: [debug]

​

Pubsub subscription

​

Flow control

• trigger_ack_batch_duration (Optional): Part of the control loop, the time between each message acknowledge batch.
• stream_ack_deadline (Optional): The ack deadline to use for the stream. The minimum deadline you can specify is 10 seconds. The maximum deadline you can specify is 600 seconds (10 minutes).
• max_outstanding_messages (Optional): Flow control settings for the maximum number of outstanding messages. When there are max_outstanding_messages currently sent to the streaming pull client that have not yet been acked or nacked, the server stops sending more messages. The sending of messages resumes once the number of outstanding messages is less than this value. If the value is <= 0, there is no limit to the number of outstanding messages.
• max_outstanding_bytes (Optional): Flow control settings for the maximum number of outstanding bytes. When there are max_outstanding_bytes or more worth of messages currently sent to the streaming pull client that have not yet been acked or nacked, the server will stop sending more messages. The sending of messages resumes once the number of outstanding bytes is less than this value. If the value is <= 0, there is no limit to the number of outstanding bytes.

flow_control:
  trigger_ack_batch_duration: 1s
  stream_ack_deadline: 60s
  max_outstanding_messages: 1000000
  max_outstanding_bytes: 1000000

​

Filtering

attributes.ce-type = "org.opentelemetry.otlp.traces.v1"
AND
attributes.content-type = "application/protobuf"

​

Configuration

​

Example Configuration

googlecloudpubsub:
googlecloudpubsub/customname:
  project: my-project
  user_agent: opentelemetry-collector-contrib {{version}}
  timeout: 20s
  subscription: projects/my-project/subscriptions/otlp-subscription