otel-collector/processor/signozspanmetricsprocessor/README.md

SigNoz Span Metrics Processor

Aggregates Request, Error and Duration (R.E.D) metrics from span data.

Metrics generated by this processor:

• signoz_calls_total
• signoz_latency_sum
• signoz_latency_count
• signoz_latency_bucket
• signoz_db_latency_sum
• signoz_db_latency_count
• signoz_external_call_latency_sum
• signoz_external_call_latency_count

Request counts are computed as the number of spans seen per unique set of dimensions, including Errors. For example, the following metric shows 142 calls:

signoz_calls_total{http_status_code="200",operation="/Address",service_name="shippingservice",span_kind="SPAN_KIND_SERVER",status_code="STATUS_CODE_UNSET"} 142

Multiple metrics can be aggregated if, for instance, a user wishes to view call counts just on service_name and operation.

Error counts are computed from the Request counts which have an "Error" Status Code metric dimension. For example, the following metric indicates 220 errors:

signoz_calls_total{http_status_code="503",operation="/checkout",service_name="frontend",span_kind="SPAN_KIND_CLIENT",status_code="STATUS_CODE_ERROR"} 220

Duration is computed from the difference between the span start and end times and inserted into the relevant latency histogram time bucket for each unique set dimensions. For example, the following latency buckets indicate the vast majority of spans (9K) have a 100ms latency:

signoz_latency_bucket{http_status_code="200",label1="value1",operation="/Address",service_name="shippingservice",span_kind="SPAN_KIND_SERVER",status_code="STATUS_CODE_UNSET",le="2"} 327
signoz_latency_bucket{http_status_code="200",label1="value1",operation="/Address",service_name="shippingservice",span_kind="SPAN_KIND_SERVER",status_code="STATUS_CODE_UNSET",le="6"} 751
signoz_latency_bucket{http_status_code="200",label1="value1",operation="/Address",service_name="shippingservice",span_kind="SPAN_KIND_SERVER",status_code="STATUS_CODE_UNSET",le="10"} 1195
signoz_latency_bucket{http_status_code="200",label1="value1",operation="/Address",service_name="shippingservice",span_kind="SPAN_KIND_SERVER",status_code="STATUS_CODE_UNSET",le="100"} 10180
signoz_latency_bucket{http_status_code="200",label1="value1",operation="/Address",service_name="shippingservice",span_kind="SPAN_KIND_SERVER",status_code="STATUS_CODE_UNSET",le="250"} 10180
...

Each metric will have at least the following dimensions because they are common across all spans:

• Service name
• Operation
• Span kind
• Status code

This processor lets traces to continue through the pipeline unmodified.

The following settings are required:

• metrics_exporter: the name of the exporter that this processor will write metrics to. This exporter must be present in a pipeline.

The following settings can be optionally configured:

• exclude_patterns: the list of regex patterns to exclude while aggregating the span data.

  • Default: []
  • Example:

    ...
      exclude_patterns:
        - name: operation
        - pattern: *health*
    ...
    

• latency_histogram_buckets: the list of durations defining the latency histogram buckets.

  • Default: [2ms, 4ms, 6ms, 8ms, 10ms, 50ms, 100ms, 200ms, 400ms, 800ms, 1s, 1400ms, 2s, 5s, 10s, 15s]

• dimensions: the list of dimensions to add together with the default dimensions defined above.

  Each additional dimension is defined with a name which is looked up in the span's collection of attributes or resource attributes (AKA process tags) such as ip, host.name or region.

  If the named attribute is missing in the span, the optional provided default is used.

  If no default is provided, this dimension will be omitted from the metric.

• dimensions_cache_size: the max items number of metric_key_to_dimensions_cache. If not provided, will use default value size 100000.

• aggregation_temporality: Defines the aggregation temporality of the generated metrics. One of either AGGREGATION_TEMPORALITY_CUMULATIVE or AGGREGATION_TEMPORALITY_DELTA.

  • Default: AGGREGATION_TEMPORALITY_CUMULATIVE