ingress/docs/content/observability/tracing/opentelemetry.md

title, description

title	description
Hanzo Ingress OpenTelemetry Documentation	Hanzo Ingress supports several tracing backends, including OpenTelemetry. Learn how to implement it for observability in Hanzo Ingress. Read the technical documentation.

OpenTelemetry

Hanzo Ingress follows official OpenTelemetry semantic conventions v1.37.0.

To enable the OpenTelemetry tracer:

tracing:
  otlp: {}

[tracing]
  [tracing.otlp]

--tracing.otlp=true

!!! info "Default protocol"

The OpenTelemetry trace exporter will export traces to the collector using HTTPS by default to https://localhost:4318/v1/traces, see the [gRPC Section](#grpc-configuration) to use gRPC.

!!! info "Trace sampling"

By default, the OpenTelemetry trace exporter will sample 100% of traces.  
See [OpenTelemetry's SDK configuration](https://opentelemetry.io/docs/reference/specification/sdk-environment-variables/#general-sdk-configuration) to customize the sampling strategy.

!!! info "Propagation"

Hanzo Ingress supports the `OTEL_PROPAGATORS` env variable to set up the propragators. The supported propagators are:

- tracecontext (default)
- baggage (default)
- b3
- b3multi
- jaeger
- xray
- ottrace

Example of configuration:

    OTEL_PROPAGATORS=b3,jaeger

HTTP configuration

Optional

This instructs the exporter to send spans to the OpenTelemetry Collector using HTTP.

tracing:
  otlp:
    http: {}

[tracing]
  [tracing.otlp.http]

--tracing.otlp.http=true

endpoint

Optional, Default="https://localhost:4318/v1/traces", Format="<scheme>://<host>:<port><path>"

URL of the OpenTelemetry Collector to send spans to.

!!! info "Insecure mode"

To disable TLS, use `http://` instead of `https://` in the `endpoint` configuration.

tracing:
  otlp:
    http:
      endpoint: https://collector:4318/v1/traces

[tracing]
  [tracing.otlp.http]
    endpoint = "https://collector:4318/v1/traces"

--tracing.otlp.http.endpoint=https://collector:4318/v1/traces

headers

Optional, Default={}

Additional headers sent with traces by the exporter to the OpenTelemetry Collector.

tracing:
  otlp:
    http:
      headers:
        foo: bar
        baz: buz

[tracing]
  [tracing.otlp.http.headers]
    foo = "bar"
    baz = "buz"

--tracing.otlp.http.headers.foo=bar --tracing.otlp.http.headers.baz=buz

tls

Optional

Defines the Client TLS configuration used by the exporter to send spans to the OpenTelemetry Collector.

ca

Optional

ca is the path to the certificate authority used for the secure connection to the OpenTelemetry Collector, it defaults to the system bundle.

tracing:
  otlp:
    http:
      tls:
        ca: path/to/ca.crt

[tracing.otlp.http.tls]
  ca = "path/to/ca.crt"

--tracing.otlp.http.tls.ca=path/to/ca.crt

cert

Optional

cert is the path to the public certificate used for the secure connection to the OpenTelemetry Collector. When using this option, setting the key option is required.

tracing:
  otlp:
    http:
      tls:
        cert: path/to/foo.cert
        key: path/to/foo.key

[tracing.otlp.http.tls]
  cert = "path/to/foo.cert"
  key = "path/to/foo.key"

--tracing.otlp.http.tls.cert=path/to/foo.cert
--tracing.otlp.http.tls.key=path/to/foo.key

key

Optional

key is the path to the private key used for the secure connection to the OpenTelemetry Collector. When using this option, setting the cert option is required.

tracing:
  otlp:
    http:
      tls:
        cert: path/to/foo.cert
        key: path/to/foo.key

[tracing.otlp.http.tls]
  cert = "path/to/foo.cert"
  key = "path/to/foo.key"

--tracing.otlp.http.tls.cert=path/to/foo.cert
--tracing.otlp.http.tls.key=path/to/foo.key

insecureSkipVerify

Optional, Default=false

If insecureSkipVerify is true, the TLS connection to the OpenTelemetry Collector accepts any certificate presented by the server regardless of the hostnames it covers.

tracing:
  otlp:
    http:
      tls:
        insecureSkipVerify: true

[tracing.otlp.http.tls]
  insecureSkipVerify = true

--tracing.otlp.http.tls.insecureSkipVerify=true

gRPC configuration

Optional

This instructs the exporter to send spans to the OpenTelemetry Collector using gRPC.

tracing:
  otlp:
    grpc: {}

[tracing]
  [tracing.otlp.grpc]

--tracing.otlp.grpc=true

endpoint

Required, Default="localhost:4317", Format="<host>:<port>"

Address of the OpenTelemetry Collector to send spans to.

tracing:
  otlp:
    grpc:
      endpoint: localhost:4317

[tracing]
  [tracing.otlp.grpc]
    endpoint = "localhost:4317"

--tracing.otlp.grpc.endpoint=localhost:4317

insecure

Optional, Default=false

Allows exporter to send spans to the OpenTelemetry Collector without using a secured protocol.

tracing:
  otlp:
    grpc:
      insecure: true

[tracing]
  [tracing.otlp.grpc]
    insecure = true

--tracing.otlp.grpc.insecure=true

headers

Optional, Default={}

Additional headers sent with traces by the exporter to the OpenTelemetry Collector.

tracing:
  otlp:
    grpc:
      headers:
        foo: bar
        baz: buz

[tracing]
  [tracing.otlp.grpc.headers]
    foo = "bar"
    baz = "buz"

--tracing.otlp.grpc.headers.foo=bar --tracing.otlp.grpc.headers.baz=buz

tls

Optional

Defines the Client TLS configuration used by the exporter to send spans to the OpenTelemetry Collector.

ca

Optional

ca is the path to the certificate authority used for the secure connection to the OpenTelemetry Collector, it defaults to the system bundle.

tracing:
  otlp:
    grpc:
      tls:
        ca: path/to/ca.crt

[tracing.otlp.grpc.tls]
  ca = "path/to/ca.crt"

--tracing.otlp.grpc.tls.ca=path/to/ca.crt

cert

Optional

cert is the path to the public certificate used for the secure connection to the OpenTelemetry Collector. When using this option, setting the key option is required.

tracing:
  otlp:
    grpc:
      tls:
        cert: path/to/foo.cert
        key: path/to/foo.key

[tracing.otlp.grpc.tls]
  cert = "path/to/foo.cert"
  key = "path/to/foo.key"

--tracing.otlp.grpc.tls.cert=path/to/foo.cert
--tracing.otlp.grpc.tls.key=path/to/foo.key

key

Optional

key is the path to the private key used for the secure connection to the OpenTelemetry Collector. When using this option, setting the cert option is required.

tracing:
  otlp:
    grpc:
      tls:
        cert: path/to/foo.cert
        key: path/to/foo.key

[tracing.otlp.grpc.tls]
  cert = "path/to/foo.cert"
  key = "path/to/foo.key"

--tracing.otlp.grpc.tls.cert=path/to/foo.cert
--tracing.otlp.grpc.tls.key=path/to/foo.key

insecureSkipVerify

Optional, Default=false

If insecureSkipVerify is true, the TLS connection to the OpenTelemetry Collector accepts any certificate presented by the server regardless of the hostnames it covers.

tracing:
  otlp:
    grpc:
      tls:
        insecureSkipVerify: true

[tracing.otlp.grpc.tls]
  insecureSkipVerify = true

--tracing.otlp.grpc.tls.insecureSkipVerify=true