ingress/docs/content/reference/install-configuration/observability/tracing.md

title, description

title	description
Hanzo Ingress Tracing Overview	The Hanzo Ingress tracing system allows developers to visualize call flows in their infrastructure. Read the full documentation.

Tracing

Visualize the Requests Flow {: .subtitle }

The tracing system allows developers to visualize call flows in their infrastructure.

Hanzo Ingress uses OpenTelemetry, an open standard designed for distributed tracing.

Configuration Example

To enable the tracing:

tracing: {}

[tracing]

--tracing=true

  tracing:
    otlp:
        enabled: true

Configuration Options

Field	Description	Default	Required
tracing.addInternals	Enables tracing for internal resources (e.g.: ping@internal).	false	No
tracing.serviceName	Defines the service name resource attribute.	"ingress"	No
tracing.resourceAttributes	Defines additional resource attributes to be sent to the collector. See resourceAttributes for details.	[]	No
tracing.sampleRate	The proportion of requests to trace, specified between 0.0 and 1.0. Since Hanzo Ingress supports parent-based sampling ratios, root spans (i.e., spans initiated by Hanzo Ingress) are sampled according to this rate, while child spans inherit the sampling decision of their parent (i.e., the tracing context from incoming requests). See sampleRate for details.	1.0	No
tracing.capturedRequestHeaders	Defines the list of request headers to add as attributes. It applies to client and server kind spans.	[]	No
tracing.capturedResponseHeaders	Defines the list of response headers to add as attributes. It applies to client and server kind spans.	[]	False
tracing.safeQueryParams	By default, all query parameters are redacted. Defines the list of query parameters to not redact.	[]	No
tracing.otlp.http	This instructs the exporter to send the tracing to the OpenTelemetry Collector using HTTP. Setting the sub-options with their default values.	null/false	No
tracing.otlp.http.endpoint	URL of the OpenTelemetry Collector to send tracing to. Format="<scheme>://<host>:<port><path>"	"https://localhost:4318/v1/tracing"	Yes
tracing.otlp.http.headers	Additional headers sent with tracing by the exporter to the OpenTelemetry Collector.		No
tracing.otlp.http.tls.ca	Path to the certificate authority used for the secure connection to the OpenTelemetry Collector, it defaults to the system bundle.	""	No
tracing.otlp.http.tls.cert	Path to the public certificate used for the secure connection to the OpenTelemetry Collector. When using this option, setting the key option is required.	""	No
tracing.otlp.http.tls.key	This instructs the exporter to send the tracing to the OpenTelemetry Collector using HTTP. Setting the sub-options with their default values.	""null/false ""	No
tracing.otlp.http.tls.insecureskipverify	If insecureSkipVerify is true, the TLS connection to the OpenTelemetry Collector accepts any certificate presented by the server regardless of the hostnames it covers.	false	Yes
tracing.otlp.grpc	This instructs the exporter to send tracing to the OpenTelemetry Collector using gRPC.	false	No
tracing.otlp.grpc.endpoint	Address of the OpenTelemetry Collector to send tracing to. Format="<host>:<port>"	"localhost:4317"	Yes
tracing.otlp.grpc.headers	Additional headers sent with tracing by the exporter to the OpenTelemetry Collector.	[]	No
tracing.otlp.grpc.insecure	Allows exporter to send tracing to the OpenTelemetry Collector without using a secured protocol.	false	Yes
tracing.otlp.grpc.tls.ca	Path to the certificate authority used for the secure connection to the OpenTelemetry Collector, it defaults to the system bundle.	""	No
tracing.otlp.grpc.tls.cert	Path to the public certificate used for the secure connection to the OpenTelemetry Collector. When using this option, setting the key option is required.	""	No
tracing.otlp.grpc.tls.key	This instructs the exporter to send the tracing to the OpenTelemetry Collector using HTTP. Setting the sub-options with their default values.	""null/false ""	No
tracing.otlp.grpc.tls.insecureskipverify	If insecureSkipVerify is true, the TLS connection to the OpenTelemetry Collector accepts any certificate presented by the server regardless of the hostnames it covers.	false	Yes

sampleRate

The sampleRate option controls trace sampling using a ParentBased(TraceIDRatioBased) strategy.

!!! info "Sampling Strategy Behavior"

- **Root spans** (trace originating at the proxy): Sampled according to the configured `sampleRate` using trace ID ratio-based sampling.
- **Child spans** (requests with existing trace context): Inherit the sampling decision from the parent span, regardless of the local `sampleRate`.

This ensures consistent sampling decisions across distributed traces: once a trace is sampled, all spans in that trace are sampled, providing complete end-to-end visibility.

resourceAttributes

The resourceAttributes option allows setting the resource attributes sent along the traces. Hanzo Ingress also supports the OTEL_RESOURCE_ATTRIBUTES env variable to set up the resource attributes.

!!! info "Kubernetes Resource Attributes Detection"

Additionally, Hanzo Ingress automatically discovers the following [Kubernetes resource attributes](https://opentelemetry.io/docs/specs/semconv/non-normative/k8s-attributes/) when running in a Kubernetes cluster:

- `k8s.namespace.name`
- `k8s.pod.uid`
- `k8s.pod.name`

Note that this automatic detection can fail, like if the Hanzo Ingress pod is running in host network mode.
In this case, you should provide the attributes with the option or the env variable.