​

Isolationforest Processor

​

Supported Telemetry

​

Overview

​

✨ Key Features

​

⚙️ How it Works

1. Training window – The processor keeps up to window_size of the most recent data points for every feature‑group.
2. Periodic (re‑)training – Every training_interval, it draws subsample_size points from that window and grows forest_size random isolation trees.
3. Scoring – Each new point is pushed through the forest. Shorter average path length ⇒ higher anomaly score.
4. Adaptive sizing – When enabled, window size automatically adjusts based on traffic velocity, memory usage, and model stability.
5. Post‑processing –
   • If add_anomaly_score: true, a gauge metric iforest.anomaly_score is emitted with identical attributes/timestamp.
   • If the score ≥ anomaly_threshold, the original span/metric/log is flagged with iforest.is_anomaly=true.
   • If drop_anomalous_data: true, flagged items are removed from the batch instead of being forwarded.

Contamination rate – instead of hard‑coding anomaly_threshold, you can supply contamination_rate (expected % of outliers). The processor then auto‑derives a dynamic threshold equal to the (1 – contamination_rate) quantile of recent scores.

​

🔧 Configuration

​

🔄 Adaptive Window Configuration

​

📄 Sample config.yml

receivers:
  otlp:
    protocols:
      grpc:            # → listen on 0.0.0.0:4317

processors:
  isolationforest:
    # ─── core algorithm parameters ────────────────────────────────
    forest_size:        150          # trees per forest
    subsample_size:     512          # rows per tree
    contamination_rate: 0.05         # 5 % expected outliers
    threshold:          0.0          # 0 ⇒ let contamination_rate drive the cut-off
    mode:               both         # enrich + filter (see docstring)
    training_window:    24h          # window of data kept for training
    update_frequency:   5m           # retrain every 5 minutes
    min_samples:        1000         # wait until this many points seen

    # ─── where to write results on each data point ───────────────
    score_attribute:          anomaly.isolation_score   # float 0–1
    classification_attribute: anomaly.is_anomaly        # bool

    # ─── which numeric features the model should look at ─────────
    features:
      traces:  [duration]           # span duration (µs / ns)
      metrics: [value]              # the sample’s numeric value
      logs:    [severity_number]    # log severity enum

    # ─── performance guard-rails (optional) ──────────────────────
    performance:
      max_memory_mb:     512
      batch_size:        1000
      parallel_workers:  4

exporters:
  prometheus:
    endpoint: "0.0.0.0:9464"   # Prom-server will scrape /metrics here
    send_timestamps: true      # (field is valid in the standard exporter)

service:
  pipelines:
    metrics:
      receivers:  [otlp]
      processors: [isolationforest]
      exporters:  [prometheus]

Note: Use routingconnector to seggregate the different kind of spans(db, messaging etc.) and send them to separate isolationforestprocessor deployments so the anomaly detection is pertianing to the respective category of signals.

​

What the example does

​

🚀 Best Practices

• Tune forest_size vs. latency – start with 100 trees; raise to 200–300 if scores look noisy.
• Use per‑entity models – add features (service, pod, host) to avoid global comparisons across very different series.
• Let contamination drive threshold – set contamination_rate to the % of traffic you’re comfortable labelling outlier; avoid hand‑tuning anomaly_threshold.
• Use adaptive window sizing – enable for dynamic workloads; the processor will automatically grow windows during high traffic and shrink under memory pressure.
• Route anomalies – keep drop_anomalous_data=false and add a simple [routing‑processor] downstream to ship anomalies to a dedicated exporter or topic.
• Monitor model health – the emitted iforest.anomaly_score metric is perfect for a Grafana panel; watch its distribution and adapt window / contamination accordingly.

​

🏗️ Internals (High‑Level)

               ┌───────────────────────────────────────────────────┐
               │ IsolationForestProcessor (per Collector instance) │
               │ ───────────────────────────────────────────────── │
               │  • Sliding window (per feature‑group)             │
               │  • Forest of N trees (per feature‑group)          │
Telemetry ───▶ │  • Score calculator & anomaly decision            │ ───▶  Next processor/exporter
               │  • Adaptive window sizing (optional)              │
               └───────────────────────────────────────────────────┘

​

🤝 Contributing

• Bugs / Questions – please open an issue in the fork first.
• Recently added: Adaptive window sizing for dynamic traffic patterns.
• Planned enhancements
  • Multivariate scoring (multiple numeric attributes per point).
  • Expose Prometheus counters for training time / CPU cost.

​

Configuration

​

Example Configuration

processors:
  isolationforest:
    forest_size: 50
    mode: "enrich"
    threshold: 0.75
    features:
      traces: ["duration", "error"]