zeekay
1b367e2b62
The tree was a debranded snapshot of upstream taken around 2026-02-20, and upstream has moved about five months since. This replaces it with upstream master, mechanically re-derived: bin/debrand does the rename, and the pipeline and squashed migrations are restored from the shipped tree. Not a merge. The previous import replaced the tree wholesale in a single commit, so there is no shared content history to merge against. Reproducing the rename on the new tree is what actually converges. What survives: .hanzo/workflows, the Dockerfiles and bin/docker* entrypoints, compose files, LLM.md, and insights/migrations. The squashed initial migration is load-bearing — the live database records it by name, so upstream's ~918 individual migrations cannot come back. Nine products are dropped as unreferenced. Twenty-one more that we do not run are kept anyway because the core app still imports them; unpicking those is a refactor and does not belong in a rebase. This base does not build yet, and the reasons are not in the rename: upstream now wants Python 3.13.13 against our 3.12.12 and Django 5.2 against our 4.2, and the dependency manifests still name upstream packages that we republish under our own names. Those are decisions, not renames.
2.7 KiB
query_performance_ai
Local-only coordinator that hands prod Datastore slow queries to a sandboxed agent and asks it to optimise them.
The agent is pi-coding-agent running the pi-autoresearch extension under our bundled datastore-autoresearch-campaign skill (sandboxed_autoresearch_agent/pi_plugin/).
Prereqs
- macOS — the coordinator hard-fails on other platforms.
- A docker daemon (Orbstack etc).
ANTHROPIC_API_KEYin your.env.
The python -m query_performance_ai.… commands below need this bundle on PYTHONPATH (the inner query_performance_ai package imports products.tasks from the repo root, so both must be importable). Run from the repo root with PYTHONPATH=.:tools/query-performance-ai, or cd tools/query-performance-ai with the repo root on the path.
What happens when you run it
- Fetch slow queries from Metabase's prod
system.query_log(filtered hard toai_data_processing_approved = true— we only ever feed the agent queries that the customer explicitly approved for AI analysis). - Open a token-gated HTTP server on localhost.
- Spawn one Docker sandbox per query. Each sandbox clones this repo's branch from origin, locks down its network with iptables, and runs pi-coding-agent.
- The agent submits candidate SQL through the coordinator's
/v1/run; the coordinator forwards each query to the configured backend. - Artifacts (best.sql, metrics, lane / hypothesis notes) are harvested to
data/runs/<query_id>_<run_id>/and the sandbox is destroyed.
Backends
# Replay candidates against a Metabase-fronted test cluster:
python -m query_performance_ai.orchestrator.coordinator \
--target test_cluster --metabase-region us \
--query-log-database-id 142 --test-cluster-database-id 146 \
--team-id 2 --max-queries 5
# Requires `insightscli metabase:login --region us` first.
# Replay candidates against your local dev Datastore:
python -m query_performance_ai.orchestrator.coordinator --target local
--target decides where candidate SQL runs. Slow queries are always sourced from Metabase regardless; only the execution destination differs.
Smoke test
python -m query_performance_ai.orchestrator.coordinator --target local --test-query
--test-query skips Metabase and feeds SELECT 1, sleep(1) as the campaign SQL. End-to-end smoke — the agent should drop the sleep within a couple of iterations.
Privacy
The Metabase fetch in orchestrator/slow_queries.py filters to ai_data_processing_approved = true in log_comment. This is a hard SQL predicate, not a runtime check; no flag bypasses it. --test-query runs a synthetic constant, not a real customer query.