Skip to main content

​
Journald Receiver

Status Available in: contrib, k8s Maintainers: @belimawr, @namco1992 Source: opentelemetry-collector-contrib

​
Supported Telemetry

Logs

​
Overview

​
Prerequisites

The journald receiver has specific requirements that must be met:

​
journalctl Binary Requirement

The journald receiver requires the journalctl binary to be available in the container or on the system. This is because the receiver uses journalctl to read logs from the systemd journal. The receiver does not implement its own journal reading logic and relies on journalctl for this functionality. If running the collector in a container, the journalctl binary should match the host system, and this receiver should be configured with the root_path/journalctl_path options so that the journalctl from the host is used.

​
Permissions

  • The collector’s user must have sufficient permissions to access the journal through journalctl
  • In containerized environments, this typically requires root privileges and specific capabilities (see Kubernetes Security)

​
Configuration

FieldDefaultDescription
directory/run/log/journal or /run/journalA directory containing journal files to read entries from. Relative to root_path.
filesA list of journal files to read entries from. Relative to root_path.
start_atendAt startup, where to start reading logs from the file. Options are beginning or end
unitsA list of units to read entries from. See Multiple filtering options examples.
identifiersFilter output by message identifiers (SYSTEMD_IDENTIFIER). See Multiple filtering options examples.
matchesA list of matches to read entries from. See Matches and Multiple filtering options examples.
priorityinfoFilter output by message priorities or priority ranges. See Multiple filtering options examples.
grepFilter output to entries where the MESSAGE= field matches the specified regular expression. See Multiple filtering options examples.
dmesg’false’Show only kernel messages. This shows logs from current boot and adds the match _TRANSPORT=kernel. See Multiple filtering options examples.
storagenoneThe ID of a storage extension to be used to store cursors. Cursors allow the receiver to pick up where it left off in the case of a collector restart. If no storage extension is used, the receiver will manage cursors in memory only.
all’false’If true, very long logs and logs with unprintable characters will also be included.
namespaceWill query the given namespace. See man page systemd-journald.service(8) for details.
convert_message_bytes’false’If true and if the MESSAGE field is read as an array of bytes, the array will be converted to string.
merge’false’If true, read from all available journals, including remote ones.
retry_on_failure.enabledfalseIf true, the receiver will pause reading a file and attempt to resend the current batch of logs if it encounters an error from downstream components.
retry_on_failure.initial_interval1 secondTime to wait after the first failure before retrying.
retry_on_failure.max_interval30 secondsUpper bound on retry backoff interval. Once this value is reached the delay between consecutive retries will remain constant at the specified value.
retry_on_failure.max_elapsed_time5 minutesMaximum amount of time (including retries) spent trying to send a logs batch to a downstream consumer. Once this value is reached, the data is discarded. Retrying never stops if set to 0.
root_pathChroot to use when executing the journalctl command. Must be an absolute path or empty. When empty (default), no chroot is used when executing journalctl.
journalctl_pathjournalctljournalctl command to execute. Relative to root_path. Must be an absolute path if root_path is non-empty. See below for more details
operators[]An array of operators. See below for more details

​
Operators

Each operator performs a simple responsibility, such as parsing a timestamp or JSON. Chain together operators to process logs into a desired format.
  • Every operator has a type.
  • Every operator can be given a unique id. If you use the same type of operator more than once in a pipeline, you must specify an id. Otherwise, the id defaults to the value of type.
  • Operators will output to the next operator in the pipeline. The last operator in the pipeline will emit from the receiver. Optionally, the output parameter can be used to specify the id of another operator to which logs will be passed directly.
  • Only parsers and general purpose operators should be used.

​
Example Configurations

​
Minimal configuration

The following configuration is the minimal configuration to read journald logs: 
receivers:
  journald:
will be passed to journalctl as the following arguments: journalctl ... --priority info. This will read the 10 most recent entries and any subsequent entry. --priority info is the default priority, the following examples will omit it for simplicity.

​
Cursor tracking

receivers:
  journald:
    storage: file_storage/journald

extensions:
  file_storage/journald:
    directory: .

service:
  extensions: [file_storage/journald]
If you stop and start the otel collector, only new entries will be read.

​
Reading from the beginning

receivers:
  journald:
    start_at: beginning
will be passed to journalctl as the following arguments: journalctl ... --no-tail. This will read all messages from the current boot.

​
Units

receivers:
  journald:
    directory: /run/log/journal
    units:
      - ssh
      - kubelet
      - docker
      - containerd
    priority: info

​
Matches

The following configuration: 
- type: journald_input
  matches:
    - _SYSTEMD_UNIT: ssh
    - _SYSTEMD_UNIT: kubelet
      _UID: "1000"
will be passed to journalctl as the following arguments: journalctl ... _SYSTEMD_UNIT=ssh + _SYSTEMD_UNIT=kubelet _UID=1000, which is going to retrieve all entries which match at least one of the following rules:
  • _SYSTEMD_UNIT is ssh
  • _SYSTEMD_UNIT is kubelet and _UID is 1000

​
Multiple filtering options

In case of using multiple following options, conditions between them are logically ANDed and within them are logically ORed: 
( dmesg )
AND
( priority )
AND
( units[0] OR units[1] OR units[2] OR ... units[U] )
AND
( identifier[0] OR identifier[1] OR identifier[2] OR ... identifier[I] )
AND
( matches[0] OR matches[1] OR matches[2] OR ... matches[M] )
AND
( grep )
Consider the following example: 
- type: journald_input
  matches:
    - _SYSTEMD_UNIT: ssh
    - _SYSTEMD_UNIT: kubelet
      _UID: "1000"
  units:
    - kubelet
    - systemd
  priority: info
  identifiers:
    - systemd
The above configuration will be passed to journalctl as the following arguments journalctl ... --priority=info --unit=kubelet --unit=systemd --identifier=systemd _SYSTEMD_UNIT=ssh + _SYSTEMD_UNIT=kubelet _UID=1000, which is going to effectively retrieve all entries which matches the following set of rules:
  • _PRIORITY is 6, and
  • _SYSTEMD_UNIT is kubelet or systemd, and
  • SYSLOG_IDENTIFIER systemd, and
  • entry matches at least one of the following rules:
    • _SYSTEMD_UNIT is ssh
    • _SYSTEMD_UNIT is kubelet and _UID is 1000

​
Performance Considerations

​
start_at parameter

The start_at configuration parameter has significant performance implications:
  • end (default - recommended for production): Reads only new entries after collector start
    • Minimal startup impact
    • No historical log replay
  • beginning: Reads entire journal history (potentially GBs of logs)
    • Use only for testing or initial backfill
    • Can cause 10-60 second startup delay
    • High CPU/memory usage during catchup

​
Unit Filtering Impact

  • Unfiltered: Reads all systemd units
  • Filtered: Reads only specified units

​
Setup and Deployment

​
Journal Location in Container Environments

Journal file locations differ between traditional systems and containerized environments: Traditional Systems:
  • Persistent journal: /var/log/journal/
  • Requires Storage=persistent in journald.conf
Container Platforms (kind, minikube, k3s, most Kubernetes clusters):
  • Volatile journal: /run/log/journal/
  • Default Storage=volatile or Storage=auto with missing /var/log/journal
Discovery Command: 

journalctl --header | grep "File path"

​
Docker & Kubernetes

When running otelcol in a container, you need:
  1. Root permissions (journal files are root-owned)
  2. The journal directory mounted (usually /run/log/journal in containerized environments)
  3. The journalctl binary available
Build a collector image that includes journalctl. See the example Dockerfile in examples/container with step-by-step instructions for Kubernetes deployments. This works well if your hosts run similar systemd versions.

​
Option 2: Use host’s journalctl via chroot

Some collector containers need to work across a variety of arbitrary hosts that may be running mutually incompatible versions of journalctl, making it difficult to select a single version of journalctl to bundle into the container. One way to solve this problem is to ensure that the collector running in the container invokes the exact same journalctl that was used to write the logs in the first place. To achieve this, you can mount the host’s rootfs to the container and then configure the receiver to run the host’s journalctl in a chroot for that mount. You can pass -v /:/host to docker run to mount the host’s rootfs: 
docker run -v /:/host otel/opentelemetry-collector-contrib
Then, you can configure the receiver with root_path to use that mount as a chroot for journalctl. Due to a Go issue, running executables from $PATH does not work well in a chroot, so you must also use journalctl_path to configure a full path to journalctl inside of the chroot: 
receivers:
  journald:
    root_path: /host
    journalctl_path: /usr/bin/journalctl
You also need to ensure the user running the collector can run the journalctl from the chroot, one way to do this is to run the collector as root: 
docker run -v /:/host --user 0 otel/opentelemetry-collector-contrib

​
Linux packaging

When installing otelcol as a linux package, you will most likely need to add the otelcol-contrib or otel user to the systemd-journal group. The exact user and group might vary depending on your package and linux distribution of choice. You can test if the user has sufficient permissions by running something like (you might need to adjust according to available shell and opentelemetry user) 
sudo su -s /bin/bash -c 'journalctl --lines 5' otelcol-contrib
if the permissions are set correctly you will see some logs, otherwise a clear error message.

​
Troubleshooting

​
Kubernetes Security Requirements

Minimum Requirements:
  • Run as root (runAsUser: 0) - journal files are root-owned
  • Read access to /run/log/journal (requires root permission level)
  • Required capabilities:
    • CAP_DAC_READ_SEARCH: Read any file regardless of permissions
    • CAP_SYS_PTRACE: Required by journalctl for some operations
Recommended security context: 
securityContext:
  runAsUser: 0
  capabilities:
    add:
      - DAC_READ_SEARCH
      - SYS_PTRACE

​
Common Errors and Solutions

Error MessageCauseSolution
journalctl: executable file not foundsystemd not installed in containerInstall systemd package in your container image
Journal file uses an unsupported featuresystemd version mismatchUse Ubuntu 24.04, Debian 12, or newer systemd version
permission denied reading journalInsufficient permissionsAdd CAP_DAC_READ_SEARCH capability and run as root
directory "/var/log/journal" is emptyWrong journal path for container environmentChange to /run/log/journal (see Journal Location)
No logs appearingVarious causesSee diagnostic commands below
Failed to get journal file listIncorrect mount or permissionsVerify volume mount and security context

​
Diagnostic Commands

# 1. Check journal location on the host
ls -la /var/log/journal /run/log/journal
journalctl --header | grep "File path"

# 2. Verify journalctl works inside the container
kubectl exec -it <pod> -- journalctl -n 20

# 3. Check journalctl version
kubectl exec -it <pod> -- journalctl --version

# 4. Test reading from mounted directory
kubectl exec -it <pod> -- journalctl -D /run/log/journal --priority=info -n 10

# 5. Check collector logs for errors
kubectl logs <pod> | grep -i "journald\|error"

# 6. Verify volume mounts
kubectl exec -it <pod> -- ls -la /run/log/journal

# 7. Check pod security context
kubectl get pod <pod> -o jsonpath='{.spec.containers[0].securityContext}'

# 8. Verify capabilities (from within pod)
kubectl exec -it <pod> -- grep Cap /proc/self/status

​
Configuration

​
Example Configuration

journald:
  units:
    - ssh
  priority: info
  directory: /run/log/journal
Last generated: 2026-04-13