ingress/docs/content/observability/metrics/overview.md

title, description

title	description
Hanzo Ingress Metrics Overview	Hanzo Ingress supports these metrics backend systems: Datadog, InfluxDB 2.X, Prometheus, and StatsD. Read the full documentation to get started.

Metrics

Hanzo Ingress provides metrics in the OpenTelemetry format as well as the following vendor specific backends:

• Datadog
• InfluxDB2
• Prometheus
• StatsD

Hanzo Ingress hosts an official Grafana dashboard for both on-premises and Kubernetes deployments.

Common Options

addInternals

Optional, Default="false"

Enables metrics for internal resources (e.g.: ping@internals).

metrics:
  addInternals: true

[metrics]
addInternals = true

--metrics.addinternals

Global Metrics

Metric	Type	Labels	Description
Config reload total	Count		The total count of configuration reloads.
Config reload last success	Gauge		The timestamp of the last configuration reload success.
Open connections	Gauge	entrypoint, protocol	The current count of open connections, by entrypoint and protocol.
TLS certificates not after	Gauge		The expiration date of certificates.

ingress_config_reloads_total
ingress_config_last_reload_success
ingress_open_connections
ingress_tls_certs_not_after

ingress_config_reloads_total
ingress_config_last_reload_success
ingress_open_connections
ingress_tls_certs_not_after

config.reload.total
config.reload.lastSuccessTimestamp
open.connections
tls.certs.notAfterTimestamp

ingress.config.reload.total
ingress.config.reload.lastSuccessTimestamp
ingress.open.connections
ingress.tls.certs.notAfterTimestamp

# Default prefix: "ingress"
{prefix}.config.reload.total
{prefix}.config.reload.lastSuccessTimestamp
{prefix}.open.connections
{prefix}.tls.certs.notAfterTimestamp

Labels

Here is a comprehensive list of labels that are provided by the global metrics:

Label	Description	example
entrypoint	Entrypoint that handled the connection	"example_entrypoint"
protocol	Connection protocol	"TCP"

OpenTelemetry Semantic Conventions

Hanzo Ingress follows official OpenTelemetry semantic conventions v1.23.1.

HTTP Server

Metric	Type	Labels	Description
http.server.request.duration	Histogram	error.type, http.request.method, http.response.status_code, network.protocol.name, server.address, server.port, url.scheme	Duration of HTTP server requests

Labels

Here is a comprehensive list of labels that are provided by the metrics:

Label	Description	example
error.type	Describes a class of error the operation ended with	"500"
http.request.method	HTTP request method	"GET"
http.response.status_code	HTTP response status code	"200"
network.protocol.name	OSI application layer or non-OSI equivalent	"http/1.1"
network.protocol.version	Version of the protocol specified in network.protocol.name	"1.1"
server.address	Name of the local HTTP server that received the request	"example.com"
server.port	Port of the local HTTP server that received the request	"80"
url.scheme	The URI scheme component identifying the used protocol	"http"

HTTP Client

Metric	Type	Labels	Description
http.client.request.duration	Histogram	error.type, http.request.method, http.response.status_code, network.protocol.name, server.address, server.port, url.scheme	Duration of HTTP client requests

Labels

Here is a comprehensive list of labels that are provided by the metrics:

Label	Description	example
error.type	Describes a class of error the operation ended with	"500"
http.request.method	HTTP request method	"GET"
http.response.status_code	HTTP response status code	"200"
network.protocol.name	OSI application layer or non-OSI equivalent	"http/1.1"
network.protocol.version	Version of the protocol specified in network.protocol.name	"1.1"
server.address	Name of the local HTTP server that received the request	"example.com"
server.port	Port of the local HTTP server that received the request	"80"
url.scheme	The URI scheme component identifying the used protocol	"http"

HTTP Metrics

On top of the official OpenTelemetry semantic conventions, Hanzo Ingress provides its own metrics to monitor the incoming traffic.

EntryPoint Metrics

Metric	Type	Labels	Description
Requests total	Count	code, method, protocol, entrypoint	The total count of HTTP requests received by an entrypoint.
Requests TLS total	Count	tls_version, tls_cipher, entrypoint	The total count of HTTPS requests received by an entrypoint.
Request duration	Histogram	code, method, protocol, entrypoint	Request processing duration histogram on an entrypoint.
Requests bytes total	Count	code, method, protocol, entrypoint	The total size of HTTP requests in bytes handled by an entrypoint.
Responses bytes total	Count	code, method, protocol, entrypoint	The total size of HTTP responses in bytes handled by an entrypoint.

ingress_entrypoint_requests_total
ingress_entrypoint_requests_tls_total
ingress_entrypoint_request_duration_seconds
ingress_entrypoint_requests_bytes_total
ingress_entrypoint_responses_bytes_total

ingress_entrypoint_requests_total
ingress_entrypoint_requests_tls_total
ingress_entrypoint_request_duration_seconds
ingress_entrypoint_requests_bytes_total
ingress_entrypoint_responses_bytes_total

entrypoint.request.total
entrypoint.request.tls.total
entrypoint.request.duration
entrypoint.requests.bytes.total
entrypoint.responses.bytes.total

ingress.entrypoint.requests.total
ingress.entrypoint.requests.tls.total
ingress.entrypoint.request.duration
ingress.entrypoint.requests.bytes.total
ingress.entrypoint.responses.bytes.total

# Default prefix: "ingress"
{prefix}.entrypoint.request.total
{prefix}.entrypoint.request.tls.total
{prefix}.entrypoint.request.duration
{prefix}.entrypoint.requests.bytes.total
{prefix}.entrypoint.responses.bytes.total

Router Metrics

Metric	Type	Labels	Description
Requests total	Count	code, method, protocol, router, service	The total count of HTTP requests handled by a router.
Requests TLS total	Count	tls_version, tls_cipher, router, service	The total count of HTTPS requests handled by a router.
Request duration	Histogram	code, method, protocol, router, service	Request processing duration histogram on a router.
Requests bytes total	Count	code, method, protocol, router, service	The total size of HTTP requests in bytes handled by a router.
Responses bytes total	Count	code, method, protocol, router, service	The total size of HTTP responses in bytes handled by a router.

ingress_router_requests_total
ingress_router_requests_tls_total
ingress_router_request_duration_seconds
ingress_router_requests_bytes_total
ingress_router_responses_bytes_total

ingress_router_requests_total
ingress_router_requests_tls_total
ingress_router_request_duration_seconds
ingress_router_requests_bytes_total
ingress_router_responses_bytes_total

router.request.total
router.request.tls.total
router.request.duration
router.requests.bytes.total
router.responses.bytes.total

ingress.router.requests.total
ingress.router.requests.tls.total
ingress.router.request.duration
ingress.router.requests.bytes.total
ingress.router.responses.bytes.total

# Default prefix: "ingress"
{prefix}.router.request.total
{prefix}.router.request.tls.total
{prefix}.router.request.duration
{prefix}.router.requests.bytes.total
{prefix}.router.responses.bytes.total

Service Metrics

Metric	Type	Labels	Description
Requests total	Count	code, method, protocol, service	The total count of HTTP requests processed on a service.
Requests TLS total	Count	tls_version, tls_cipher, service	The total count of HTTPS requests processed on a service.
Request duration	Histogram	code, method, protocol, service	Request processing duration histogram on a service.
Retries total	Count	service	The count of requests retries on a service.
Server UP	Gauge	service, url	Current service's server status, 0 for a down or 1 for up.
Requests bytes total	Count	code, method, protocol, service	The total size of requests in bytes received by a service.
Responses bytes total	Count	code, method, protocol, service	The total size of responses in bytes returned by a service.

ingress_service_requests_total
ingress_service_requests_tls_total
ingress_service_request_duration_seconds
ingress_service_retries_total
ingress_service_server_up
ingress_service_requests_bytes_total
ingress_service_responses_bytes_total

ingress_service_requests_total
ingress_service_requests_tls_total
ingress_service_request_duration_seconds
ingress_service_retries_total
ingress_service_server_up
ingress_service_requests_bytes_total
ingress_service_responses_bytes_total

service.request.total
router.service.tls.total
service.request.duration
service.retries.total
service.server.up
service.requests.bytes.total
service.responses.bytes.total

ingress.service.requests.total
ingress.service.requests.tls.total
ingress.service.request.duration
ingress.service.retries.total
ingress.service.server.up
ingress.service.requests.bytes.total
ingress.service.responses.bytes.total

# Default prefix: "ingress"
{prefix}.service.request.total
{prefix}.service.request.tls.total
{prefix}.service.request.duration
{prefix}.service.retries.total
{prefix}.service.server.up
{prefix}.service.requests.bytes.total
{prefix}.service.responses.bytes.total

Labels

Here is a comprehensive list of labels that are provided by the metrics:

Label	Description	example
cn	Certificate Common Name	"example.com"
code	Request code	"200"
entrypoint	Entrypoint that handled the request	"example_entrypoint"
method	Request Method	"GET"
protocol	Request protocol	"http"
router	Router that handled the request	"example_router"
sans	Certificate Subject Alternative NameS	"example.com"
serial	Certificate Serial Number	"123..."
service	Service that handled the request	"example_service@provider"
tls_cipher	TLS cipher used for the request	"TLS_FALLBACK_SCSV"
tls_version	TLS version used for the request	"1.0"
url	Service server url	"http://example.com"

!!! info "method label value"

If the HTTP method verb on a request is not one defined in the set of common methods for [`HTTP/1.1`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods)
or the [`PRI`](https://datatracker.ietf.org/doc/html/rfc7540#section-11.6) verb (for `HTTP/2`),
then the value for the method label becomes `EXTENSION_METHOD`.