mirror of
https://github.com/hanzoai/o11y.git
synced 2026-11-03 02:04:48 +00:00
zeekayandGitHub
babbdec4b8
* debrand: signoz/SigNoz/SIGNOZ -> o11y/O11y/O11Y across the tree Rename all SigNoz *branding* to O11y in file contents, file/dir names, package names, env vars, config keys, comments, docs, SDK code, and the ClickHouse schema identifiers the querier reads. Collector dependency: - Repoint github.com/hanzoai/signoz-otel-collector -> github.com/hanzoai/otel-collector and bump v0.144.6 -> v0.144.7 (go.mod + go.sum reconciled via go mod tidy). Internal package paths also debranded upstream: signozschemamigrator -> o11yschemamigrator, signozlogspipelineprocessor -> o11ylogspipelineprocessor. - Fix: v0.144.7 dropped otelconst.DistributedFieldKeysTable from the collector's public constants; pin FieldKeysTable = "distributed_field_keys" locally in pkg/telemetrymetadata/tables.go. Package/dir renames (package decl + all importers): pkg/apiserver/signozapiserver -> o11yapiserver, plus signozalertmanager, signozauthzapi, signozglobal, signozquerier, signozruler, and pkg/signoz -> pkg/o11y. Schema (read plane): signoz_traces/metrics/logs/metadata/analytics/meter and signoz_index_v3/index_v2/error_index_v2/spans (+ distributed_*) -> o11y_*. Data-preserving cutover migration added: deploy/clickhouse/migrations/0001_rename_signoz_to_o11y.sql (metadata-only RENAME DATABASE/TABLE, no drop/recreate). Preserved (attribution / external / wire contracts): - LICENSE + NOTICE verbatim (incl. "software from SigNoz", "Copyright ... SigNoz Inc."). - Upstream github.com/SigNoz/* repo URLs + @SigNoz/* CODEOWNERS teams. - Billing resource-attribute regex "signoz.workspace.*" (external wire contract). go build ./pkg/... ./cmd/... = 0; gofmt clean; renamed+schema pkg tests pass. * o11y: bump collector to v0.144.8 (o11y_* physical schema writer) + lockstep deploy plan Collector v0.144.8's schema-migrator + exporters now CREATE/WRITE the same o11y_* physical databases/tables the querier reads and the RENAME migration (0001_rename_signoz_to_o11y.sql) targets. Cross-checked byte-identical: writer DB set == migration target DB set (6); writer prefixed-table set == migration target table set (8); o11y querier reads and cloud reads are subsets of that set. Zero table-name mismatches between writer, readers, and migration. Collector go.mod unchanged between v0.144.7 and v0.144.8 (identical go.mod hash) — pure source rename, no module-graph change. Adds deploy/clickhouse/DEPLOY_signoz_to_o11y_cutover.md: the lockstep order (collector v0.144.8 -> RENAME migration -> o11y+cloud readers) with a fresh/scratch-ClickHouse pre-flight (create via migrator, emit trace+metric, query back) and rollback. Ships together or telemetry blackholes. go build ./pkg/... ./cmd/community = 0.
3.0 KiB
3.0 KiB
Collect Clickhouse Query Logs
You can configure collection from system.query_log table in clickhouse by providing the required collector config to your collector.
Create collector config file
Save the following config for collecting clickhouse query logs in a file named clickhouse-query-logs-collection-config.yaml
receivers:
clickhousesystemtablesreceiver/query_log:
dsn: "${env:CLICKHOUSE_MONITORING_DSN}"
cluster_name: "${env:CLICKHOUSE_CLUSTER_NAME}"
query_log_scrape_config:
scrape_interval_seconds: ${env:QUERY_LOG_SCRAPE_INTERVAL_SECONDS}
min_scrape_delay_seconds: ${env:QUERY_LOG_SCRAPE_DELAY_SECONDS}
exporters:
# export to O11y cloud
otlp/clickhouse-query-logs:
endpoint: "${env:OTLP_DESTINATION_ENDPOINT}"
tls:
insecure: false
headers:
"o11y-access-token": "${env:O11Y_INGESTION_KEY}"
# export to local collector
# otlp/clickhouse-query-logs:
# endpoint: "localhost:4317"
# tls:
# insecure: true
service:
pipelines:
logs/clickhouse-query-logs:
receivers: [clickhousesystemtablesreceiver/query_log]
processors: []
exporters: [otlp/clickhouse-query-logs]
Set Environment Variables
Set the following environment variables in your otel-collector environment:
# DSN for connecting to clickhouse with the monitoring user
# Replace monitoring:<PASSWORD> with `username:password` for your monitoring user
# Note: The monitoring user must be able to issue select queries on system.query_log table.
export CLICKHOUSE_MONITORING_DSN="tcp://monitoring:<PASSWORD>@clickhouse:9000/"
# If collecting query logs from a clustered deployment, specify a non-empty cluster name.
export CLICKHOUSE_CLUSTER_NAME=""
# Rows from query_log table will be collected periodically based on this setting
export QUERY_LOG_SCRAPE_INTERVAL_SECONDS=20
# Must be configured to a value greater than flush_interval_milliseconds setting for query_log.
# This setting can be found in the clickhouse server config
# For details see https://clickhouse.com/docs/en/operations/server-configuration-parameters/settings#query-log
# Setting a large enough value ensures all query logs for a particular time interval have been
# flushed before an attempt to collect them is made.
export QUERY_LOG_SCRAPE_DELAY_SECONDS=8
# region specific O11y cloud ingestion endpoint
export OTLP_DESTINATION_ENDPOINT="ingest.us.o11y.cloud:443"
# your O11y ingestion key
export O11Y_INGESTION_KEY="o11y-ingestion-key"
Use collector config file
Make the collector config file available to your otel collector and use it by adding the following flag to the command for running your collector
--config clickhouse-query-logs-collection-config.yaml
Note: the collector can use multiple config files, specified by multiple occurrences of the --config flag.
Also note that only 1 collector instance should be configured to collect query_logs.
Using multiple collector instances or replicas with this config will lead to duplicate logs.