This is a cache of https://docs.openshift.com/container-platform/4.17/observability/otel/otel-collector/otel-collector-receivers.html. It is a snapshot of the page at 2024-11-26T07:36:42.023+0000.
Receivers - Red Hat build of OpenTelemetry | Observability | OpenShift Container Platform 4.17
    OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history bug_report picture_as_pdf
×

Receivers

Receivers get data into the Collector. A receiver can be push or pull based. Generally, a receiver accepts data in a specified format, translates it into the internal format, and passes it to processors and exporters defined in the applicable pipelines. By default, no receivers are configured. One or more receivers must be configured. Receivers may support one or more data sources.

OTLP Receiver

The OTLP Receiver ingests traces, metrics, and logs by using the OpenTelemetry Protocol (OTLP). The OTLP Receiver ingests traces and metrics using the OpenTelemetry protocol (OTLP).

OpenTelemetry Collector custom resource with an enabled OTLP Receiver
# ...
  config: |
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317 (1)
            tls: (2)
              ca_file: ca.pem
              cert_file: cert.pem
              key_file: key.pem
              client_ca_file: client.pem (3)
              reload_interval: 1h (4)
          http:
            endpoint: 0.0.0.0:4318 (5)
            tls: (6)

    service:
      pipelines:
        traces:
          receivers: [otlp]
        metrics:
          receivers: [otlp]
# ...
1 The OTLP gRPC endpoint. If omitted, the default 0.0.0.0:4317 is used.
2 The server-side TLS configuration. Defines paths to TLS certificates. If omitted, the TLS is disabled.
3 The path to the TLS certificate at which the server verifies a client certificate. This sets the value of ClientCAs and ClientAuth to RequireAndVerifyClientCert in the TLSConfig. For more information, see the Config of the Golang TLS package.
4 Specifies the time interval at which the certificate is reloaded. If the value is not set, the certificate is never reloaded. The reload_interval field accepts a string containing valid units of time such as ns, us (or µs), ms, s, m, h.
5 The OTLP HTTP endpoint. The default value is 0.0.0.0:4318.
6 The server-side TLS configuration. For more information, see the grpc protocol configuration section.

Jaeger Receiver

The Jaeger Receiver ingests traces in the Jaeger formats.

OpenTelemetry Collector custom resource with an enabled Jaeger Receiver
# ...
  config: |
    receivers:
      jaeger:
        protocols:
          grpc:
            endpoint: 0.0.0.0:14250 (1)
          thrift_http:
            endpoint: 0.0.0.0:14268 (2)
          thrift_compact:
            endpoint: 0.0.0.0:6831 (3)
          thrift_binary:
            endpoint: 0.0.0.0:6832 (4)
          tls: (5)

    service:
      pipelines:
        traces:
          receivers: [jaeger]
# ...
1 The Jaeger gRPC endpoint. If omitted, the default 0.0.0.0:14250 is used.
2 The Jaeger Thrift HTTP endpoint. If omitted, the default 0.0.0.0:14268 is used.
3 The Jaeger Thrift Compact endpoint. If omitted, the default 0.0.0.0:6831 is used.
4 The Jaeger Thrift Binary endpoint. If omitted, the default 0.0.0.0:6832 is used.
5 The server-side TLS configuration. See the OTLP Receiver configuration section for more details.

Host Metrics Receiver

The Host Metrics Receiver ingests metrics in the OTLP format.

The Host Metrics Receiver is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
OpenTelemetry Collector custom resource with an enabled Host Metrics Receiver
apiVersion: v1
kind: serviceAccount
metadata:
  name: otel-hostfs-daemonset
  namespace: <namespace>
# ...
---
apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
allowHostDirVolumePlugin: true
allowHostIPC: false
allowHostNetwork: false
allowHostPID: true
allowHostPorts: false
allowPrivilegeEscalation: true
allowPrivilegedContainer: true
allowedCapabilities: null
defaultAddCapabilities:
- SYS_ADMIN
fsGroup:
  type: RunAsAny
groups: []
metadata:
  name: otel-hostmetrics
readOnlyRootFilesystem: true
runAsUser:
  type: RunAsAny
seLinuxContext:
  type: RunAsAny
supplementalGroups:
  type: RunAsAny
users:
- system:serviceaccount:<namespace>:otel-hostfs-daemonset
volumes:
- configMap
- emptyDir
- hostPath
- projected
# ...
---
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: otel
  namespace: <namespace>
spec:
  serviceAccount: otel-hostfs-daemonset
  mode: daemonset
  volumeMounts:
    - mountPath: /hostfs
      name: host
      readOnly: true
  volumes:
    - hostPath:
        path: /
      name: host
  config: |
    receivers:
      hostmetrics:
        collection_interval: 10s (1)
        initial_delay: 1s (2)
        root_path: / (3)
        scrapers: (4)
          cpu:
          memory:
          disk:
    service:
      pipelines:
        metrics:
          receivers: [hostmetrics]
# ...
1 Sets the time interval for host metrics collection. If omitted, the default value is 1m.
2 Sets the initial time delay for host metrics collection. If omitted, the default value is 1s.
3 Configures the root_path so that the Host Metrics Receiver knows where the root filesystem is. If running multiple instances of the Host Metrics Receiver, set the same root_path value for each instance.
4 Lists the enabled host metrics scrapers. Available scrapers are cpu, disk, load, filesystem, memory, network, paging, processes, and process.

Kubernetes Objects Receiver

The Kubernetes Objects Receiver pulls or watches objects to be collected from the Kubernetes API server. This receiver watches primarily Kubernetes events, but it can collect any type of Kubernetes objects. This receiver gathers telemetry for the cluster as a whole, so only one instance of this receiver suffices for collecting all the data.

The Kubernetes Objects Receiver is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
OpenTelemetry Collector custom resource with an enabled Kubernetes Objects Receiver
apiVersion: v1
kind: serviceAccount
metadata:
  name: otel-k8sobj
  namespace: <namespace>
# ...
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: otel-k8sobj
  namespace: <namespace>
rules:
- apiGroups:
  - ""
  resources:
  - events
  - pods
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - "events.k8s.io"
  resources:
  - events
  verbs:
  - watch
  - list
# ...
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: otel-k8sobj
subjects:
  - kind: serviceAccount
    name: otel-k8sobj
    namespace: <namespace>
roleRef:
  kind: ClusterRole
  name: otel-k8sobj
  apiGroup: rbac.authorization.k8s.io
# ...
---
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: otel-k8s-obj
  namespace: <namespace>
spec:
  serviceAccount: otel-k8sobj
  image: ghcr.io/os-observability/redhat-opentelemetry-collector/redhat-opentelemetry-collector:main
  mode: deployment
  config: |
    receivers:
      k8sobjects:
        auth_type: serviceAccount
        objects:
          - name: pods (1)
            mode: pull (2)
            interval: 30s (3)
            label_selector: (4)
            field_selector: (5)
            namespaces: [<namespace>,...] (6)
          - name: events
            mode: watch
    exporters:
      debug:
    service:
      pipelines:
        logs:
          receivers: [k8sobjects]
          exporters: [debug]
# ...
1 The Resource name that this receiver observes: for example, pods, deployments, or events.
2 The observation mode that this receiver uses: pull or watch.
3 Only applicable to the pull mode. The request interval for pulling an object. If omitted, the default value is 1h.
4 The label selector to define targets.
5 The field selector to filter targets.
6 The list of namespaces to collect events from. If omitted, the default value is all.

Kubelet Stats Receiver

The Kubelet Stats Receiver extracts metrics related to nodes, pods, containers, and volumes from the kubelet’s API server. These metrics are then channeled through the metrics-processing pipeline for additional analysis.

The Kubelet Stats Receiver is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
OpenTelemetry Collector custom resource with an enabled Kubelet Stats Receiver
# ...
config: |
  receivers:
    kubeletstats:
      collection_interval: 20s
      auth_type: "serviceAccount"
      endpoint: "https://${env:K8S_NODE_NAME}:10250"
      insecure_skip_verify: true
  service:
    pipelines:
      metrics:
        receivers: [kubeletstats]
env:
  - name: K8S_NODE_NAME (1)
    valueFrom:
      fieldRef:
        fieldPath: spec.nodeName
# ...
1 Sets the K8S_NODE_NAME to authenticate to the API.

The Kubelet Stats Receiver requires additional permissions for the service account used for running the OpenTelemetry Collector.

Permissions required by the service account
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: otel-collector
rules:
  - apiGroups: ['']
    resources: ['nodes/stats']
    verbs: ['get', 'watch', 'list']
  - apiGroups: [""]
    resources: ["nodes/proxy"] (1)
    verbs: ["get"]
# ...
1 The permissions required when using the extra_metadata_labels or request_utilization or limit_utilization metrics.

Prometheus Receiver

The Prometheus Receiver scrapes the metrics endpoints.

The Prometheus Receiver is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
OpenTelemetry Collector custom resource with an enabled Prometheus Receiver
# ...
  config: |
    receivers:
        prometheus:
          config:
            scrape_configs: (1)
              - job_name: 'my-app'  (2)
                scrape_interval: 5s (3)
                static_configs:
                  - targets: ['my-app.example.svc.cluster.local:8888'] (4)
    service:
      pipelines:
        metrics:
          receivers: [prometheus]
# ...
1 Scrapes configurations using the Prometheus format.
2 The Prometheus job name.
3 The lnterval for scraping the metrics data. Accepts time units. The default value is 1m.
4 The targets at which the metrics are exposed. This example scrapes the metrics from a my-app application in the example project.

Zipkin Receiver

The Zipkin Receiver ingests traces in the Zipkin v1 and v2 formats.

OpenTelemetry Collector custom resource with the enabled Zipkin Receiver
# ...
  config: |
    receivers:
      zipkin:
        endpoint: 0.0.0.0:9411 (1)
        tls: (2)
    service:
      pipelines:
        traces:
          receivers: [zipkin]
# ...
1 The Zipkin HTTP endpoint. If omitted, the default 0.0.0.0:9411 is used.
2 The server-side TLS configuration. See the OTLP Receiver configuration section for more details.

Kafka Receiver

The Kafka Receiver receives traces, metrics, and logs from Kafka in the OTLP format.

The Kafka Receiver is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
OpenTelemetry Collector custom resource with the enabled Kafka Receiver
# ...
  config: |
    receivers:
      kafka:
        brokers: ["localhost:9092"] (1)
        protocol_version: 2.0.0 (2)
        topic: otlp_spans (3)
        auth:
          plain_text: (4)
            username: example
            password: example
          tls: (5)
            ca_file: ca.pem
            cert_file: cert.pem
            key_file: key.pem
            insecure: false (6)
            server_name_override: kafka.example.corp (7)
    service:
      pipelines:
        traces:
          receivers: [kafka]
# ...
1 The list of Kafka brokers. The default is localhost:9092.
2 The Kafka protocol version. For example, 2.0.0. This is a required field.
3 The name of the Kafka topic to read from. The default is otlp_spans.
4 The plain text authentication configuration. If omitted, plain text authentication is disabled.
5 The client-side TLS configuration. Defines paths to the TLS certificates. If omitted, TLS authentication is disabled.
6 Disables verifying the server’s certificate chain and host name. The default is false.
7 ServerName indicates the name of the server requested by the client to support virtual hosting.

Kubernetes Cluster Receiver

The Kubernetes Cluster Receiver gathers cluster metrics and entity events from the Kubernetes API server. It uses the Kubernetes API to receive information about updates. Authentication for this receiver is only supported through service accounts.

The Kubernetes Cluster Receiver is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
OpenTelemetry Collector custom resource with the enabled Kubernetes Cluster Receiver
# ...
  receivers:
    k8s_cluster:
      distribution: openshift
      collection_interval: 10s
  exporters:
    debug:
  service:
    pipelines:
      metrics:
        receivers: [k8s_cluster]
        exporters: [debug]
      logs/entity_events:
        receivers: [k8s_cluster]
        exporters: [debug]
# ...

This receiver requires a configured service account, RBAC rules for the cluster role, and the cluster role binding that binds the RBAC with the service account.

serviceAccount object
apiVersion: v1
kind: serviceAccount
metadata:
  labels:
    app: otelcontribcol
  name: otelcontribcol
# ...
RBAC rules for the ClusterRole object
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
rules:
- apiGroups:
  - quota.openshift.io
  resources:
  - clusterresourcequotas
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  - namespaces
  - namespaces/status
  - nodes
  - nodes/spec
  - pods
  - pods/status
  - replicationcontrollers
  - replicationcontrollers/status
  - resourcequotas
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - daemonsets
  - deployments
  - replicasets
  - statefulsets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - extensions
  resources:
  - daemonsets
  - deployments
  - replicasets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - jobs
  - cronjobs
  verbs:
  - get
  - list
  - watch
- apiGroups:
    - autoscaling
  resources:
    - horizontalpodautoscalers
  verbs:
    - get
    - list
    - watch
# ...
ClusterRoleBinding object
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: otelcontribcol
subjects:
- kind: serviceAccount
  name: otelcontribcol
  namespace: default
# ...

OpenCensus Receiver

The OpenCensus Receiver provides backwards compatibility with the OpenCensus project for easier migration of instrumented codebases. It receives metrics and traces in the OpenCensus format via gRPC or HTTP and Json.

OpenTelemetry Collector custom resource with the enabled OpenCensus Receiver
# ...
  config: |
    receivers:
      opencensus:
        endpoint: 0.0.0.0:9411 (1)
        tls: (2)
        cors_allowed_origins: (3)
          - https://*.<example>.com
    service:
      pipelines:
        traces:
          receivers: [opencensus]
# ...
1 The OpenCensus endpoint. If omitted, the default is 0.0.0.0:55678.
2 The server-side TLS configuration. See the OTLP Receiver configuration section for more details.
3 You can also use the HTTP JSON endpoint to optionally configure CORS, which is enabled by specifying a list of allowed CORS origins in this field. Wildcards with * are accepted under the cors_allowed_origins. To match any origin, enter only *.

Filelog Receiver

The Filelog Receiver tails and parses logs from files.

The Filelog Receiver is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
OpenTelemetry Collector custom resource with the enabled Filelog Receiver that tails a text file
# ...
receivers:
  filelog:
    include: [ /simple.log ] (1)
    operators: (2)
      - type: regex_parser
        regex: '^(?P<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) (?P<sev>[A-Z]*) (?P<msg>.*)$'
        timestamp:
          parse_from: attributes.time
          layout: '%Y-%m-%d %H:%M:%S'
        severity:
          parse_from: attributes.sev
# ...
1 A list of file glob patterns that match the file paths to be read.
2 An array of Operators. Each Operator performs a simple task such as parsing a timestamp or JSON. To process logs into a desired format, chain the Operators together.

Journald Receiver

The Journald Receiver parses journald events from the systemd journal and sends them as logs.

The Journald Receiver is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
OpenTelemetry Collector custom resource with the enabled Journald Receiver
apiVersion: v1
kind: Namespace
metadata:
  name: otel-journald
  labels:
    security.openshift.io/scc.podSecurityLabelSync: "false"
    pod-security.kubernetes.io/enforce: "privileged"
    pod-security.kubernetes.io/audit: "privileged"
    pod-security.kubernetes.io/warn: "privileged"
# ...
---
apiVersion: v1
kind: serviceAccount
metadata:
  name: privileged-sa
  namespace: otel-journald
# ...
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: otel-journald-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:openshift:scc:privileged
subjects:
- kind: serviceAccount
  name: privileged-sa
  namespace: otel-journald
# ...
---
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: otel-journald-logs
  namespace: otel-journald
spec:
  mode: daemonset
  serviceAccount: privileged-sa
  securityContext:
    allowPrivilegeEscalation: false
    capabilities:
      drop:
      - CHOWN
      - DAC_OVERRIDE
      - FOWNER
      - FSETID
      - KILL
      - NET_BIND_service
      - SETGID
      - SETPCAP
      - SETUID
    readOnlyRootFilesystem: true
    seLinuxOptions:
      type: spc_t
    seccompProfile:
      type: RuntimeDefault
  config: |
    receivers:
      journald:
        files: /var/log/journal/*/*
        priority: info (1)
        units: (2)
          - kubelet
          - crio
          - init.scope
          - dnsmasq
        all: true (3)
        retry_on_failure:
          enabled: true (4)
          initial_interval: 1s (5)
          max_interval: 30s (6)
          max_elapsed_time: 5m (7)
    processors:
    exporters:
      debug:
        verbosity: detailed
    service:
      pipelines:
        logs:
          receivers: [journald]
          exporters: [debug]
  volumeMounts:
  - name: journal-logs
    mountPath: /var/log/journal/
    readOnly: true
  volumes:
  - name: journal-logs
    hostPath:
      path: /var/log/journal
  tolerations:
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule
# ...
1 Filters output by message priorities or priority ranges. The default value is info.
2 Lists the units to read entries from. If empty, entries are read from all units.
3 Includes very long logs and logs with unprintable characters. The default value is false.
4 If set to true, the receiver pauses reading a file and attempts to resend the current batch of logs when encountering an error from downstream components. The default value is false.
5 The time interval to wait after the first failure before retrying. The default value is 1s. The units are ms, s, m, h.
6 The upper bound for the retry backoff interval. When this value is reached, the time interval between consecutive retry attempts remains constant at this value. The default value is 30s. The supported units are ms, s, m, h.
7 The maximum time interval, including retry attempts, for attempting to send a logs batch to a downstream consumer. When this value is reached, the data are discarded. If the set value is 0, retrying never stops. The default value is 5m. The supported units are ms, s, m, h.

Kubernetes Events Receiver

The Kubernetes Events Receiver collects events from the Kubernetes API server. The collected events are converted into logs.

The Kubernetes Events Receiver is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
OpenShift Container Platform permissions required for the Kubernetes Events Receiver
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: otel-collector
  labels:
    app: otel-collector
rules:
- apiGroups:
  - ""
  resources:
  - events
  - namespaces
  - namespaces/status
  - nodes
  - nodes/spec
  - pods
  - pods/status
  - replicationcontrollers
  - replicationcontrollers/status
  - resourcequotas
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - daemonsets
  - deployments
  - replicasets
  - statefulsets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - extensions
  resources:
  - daemonsets
  - deployments
  - replicasets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - jobs
  - cronjobs
  verbs:
  - get
  - list
  - watch
- apiGroups:
    - autoscaling
  resources:
    - horizontalpodautoscalers
  verbs:
    - get
    - list
    - watch
# ...
OpenTelemetry Collector custom resource with the enabled Kubernetes Event Receiver
# ...
  serviceAccount: otel-collector (1)
  config: |
    receivers:
      k8s_events:
        namespaces: [project1, project2] (2)
    service:
      pipelines:
        logs:
          receivers: [k8s_events]
# ...
1 The service account of the Collector that has the required ClusterRole otel-collector RBAC.
2 The list of namespaces to collect events from. The default value is empty, which means that all namespaces are collected.

Additional resources