hanzo-dev
6db7fa9837
CI/CD / containment (push) Successful in 3m4s
Hanzo CI/CD / cicd (push) Failing after 29m12s
CI/CD / gate (push) Failing after 29m15s
CI/CD / image (push) Skipped
CI/CD / rollout (push) Skipped
CI/CD / reach (push) Skipped
CI/CD / fanout (push) Skipped
CI/CD / receipt (push) Skipped
The sink commits one row per message, and insertSettings asks the store to batch it (async_insert) while wait_for_async_insert holds the call open until the block is durable. With a single puller those two deadlock: the one in-flight row IS the whole batch, so every insert pays a full materialized-view cascade to land a single fact, and the next message cannot start until it finishes. Throughput collapses to 1/insert-latency — measured at ~24 rows/min against a ~2.5s cascade — no matter how much is queued behind it. The warehouse was ~3.6 hours behind live traffic and losing ground, so every dashboard, heat map and rollup read empty: they filter on `time`, and `time` had not advanced since the backlog formed. Nothing was lost — the door commits to JetStream before it answers, and the stream holds 72h — but the window is finite. A JetStream pull consumer load-balances across every client bound to it, so binding the durable N times is concurrency without a second cursor. Each message is still delivered once and still acked only after ITS OWN row lands; redelivery, MaxDeliver and the abandoned-fact advisory are untouched. That gives async_insert real blocks to coalesce and lets the store overlap the cascades, so the ceiling becomes the store's rather than the round-trip's. Also drops insights.raw_sessions_v3_mv in the warehouse (done out-of-band): 854ms and 9.2MiB of every insert spent projecting a table with no readers. Co-authored-by: Hanzo Dev <dev@hanzo.ai>