chore(sqlite): bump hanzoai/sqlite v0.2.4 → v0.3.2, drop modernc from the graph (#18 )

s3's SQLite filer already imports github.com/hanzoai/sqlite on main. v0.3.2 is self-contained (vendored pure-Go engine), so modernc.org/{sqlite,libc,memory, mathutil} are no longer needed and are dropped from go.mod (verified: go build ./... stays green, cgo + nocgo, without them). csqlite + sqlcipher (v0.3.2's cgo backend) enter as indirect. modernc removal is done via go mod edit rather than go mod tidy: a pre-existing, unrelated go.sum re-tag drift on github.com/luxfi/pq@v1.0.3 blocks full-graph ops on this branch's base (origin/main) — out of scope for the sqlite migration. Co-authored-by: hanzo-dev <dev@hanzo.ai>
fix(s3api): bound gateway startup filer reads so an unresponsive filer can't hang :8333
2026-07-18 09:15:14 -07:00 · 2026-07-17 23:08:44 -07:00 · 2026-07-17 22:33:55 -07:00 · 2026-07-17 21:56:49 -07:00 · 2026-07-17 21:16:55 -07:00 · 2026-07-13 16:05:26 -07:00
11 changed files with 99 additions and 50 deletions
@@ -81,28 +81,31 @@ dials ZAP via `brokerPool`. The last gRPC broker dial (`admin/dash/admin_server.
 (`mq/broker/broker_grpc_*.go`) keep their `mq_pb.HanzoMessaging_*Server` signatures —
 those names are now plain `Send/Recv/Context` interfaces, not grpc generic streams.

-gRPC is still **load-bearing** in (these legitimately import grpc — leave them):
- **Volume server RPC** — still gRPC-served (`command/volume.go`, `server/volume_grpc_*.go`,
-  `pb/volume_server_pb/volume_server_grpc.pb.go`). Its `status.Errorf(codes.X)` producers
-  must stay so gRPC clients get real codes; `peer.FromContext` works over gRPC.
- **Raft transport** — hashicorp/seaweedfs raft rides the master's gRPC listener
-  (`server/raft_hashicorp.go`, the `pb.NewGrpcServer` + `reflection.Register` in
-  `command/master.go` and `master_follower.go`). The master's own service is ZAP; the
-  gRPC listener carries raft only.
- **Strangler client seam** `volumezap/client.go` — implements the
-  `volume_server_pb.*Client` interface, whose generated signatures use `grpc.{CallOption,
-  *StreamingClient}` and `metadata.MD`. Bound to grpc until that `*_pb` interface is
-  regenerated grpc-free. (`mqzap/client.go` is now grpc-free — see above.)
- **Core dial/server + request-id** — `pb/grpc_client_server.go` (volume dials,
-  `NewGrpcServer`), `operation/grpc_client.go`, `server/common.go` (grpc metadata).
- **TLS cert hot-reload** — `security/tls.go`, `security/certreload/certreload.go`,
-  `security/tls_reload.go`, `util/http/client/http_client.go` reuse grpc's
-  `credentials/tls/certprovider` + `pemfile` as a generic file-watching cert provider;
-  `tls.go` also returns `grpc.ServerOption` for the remaining gRPC servers.
+gRPC has been ripped from **every** RPC path. **Zero `.go` files in the main module
+import `google.golang.org/grpc`** — `grep -rl google.golang.org/grpc s3/` returns 0.
+The items previously listed here as "still load-bearing" (volume RPC, raft, cert
+hot-reload, core dial/server, request-id) are all now native/ZAP:
+- **Volume server RPC** rides ZAP: `pb/volume_server_pb/volume_server_grpc.pb.go` is
+  grpc-free (client iface returns `pb/rpc` stream seams; server iface is a plain method
+  set). `volumezap/client.go` and `server/raft_hashicorp.go` are gone.
+- **Cert hot-reload** (`security/certreload/certreload.go`) is a native file-watching
+  provider — no grpc `certprovider`/`pemfile`; the surviving "grpc" mentions there are
+  comments naming only what it replaced.
+- Error semantics still cross the wire as the PascalCase code-name STRING (see above),
+  classified by `strings.Contains` — never reintroduce `google.golang.org/grpc/{codes,status}`.

-Full `go.mod` grpc removal is unreachable until volume RPC is ported to ZAP (regen its
-`*_pb` interface grpc-free, switch the server to ZAP dispatch) and cert hot-reload drops
-grpc's certprovider. (Filer, master, and MQ broker are already ported.)
+**The residual `google.golang.org/grpc v1.x // indirect` in `go.mod` is not an RPC-path
+dep.** It is pulled transitively by the optional Google Cloud client family — `go mod why
+grpc` traces `s3/kms/gcp → cloud.google.com/go/kms/apiv1 → grpc`; `cloud.google.com/go/pubsub`
+and `cloud.google.com/go/storage` pull it the same way, and `otelgrpc` rides in via
+`google.golang.org/api/transport/grpc`. These SDKs are grpc-transport-native but are
+**never dialed in a Hanzo (no-GCP) deployment** — the wire is 100% ZAP. Reaching literal
+zero grpc in `go.mod` means ripping the whole GCP integration surface (5 files: `kms/gcp`,
+`notification/google_pub_sub`, `replication/sub/notification_google_pub_sub.go`,
+`remote_storage/gcs`, `replication/sink/gcssink`; blank-imported from
+`s3api/auth_credentials.go`, `server/filer_server.go`, `command/imports.go`) plus dropping
+the `cloud.google.com/go/{kms,pubsub,storage}` + `google.golang.org/api` direct deps — a
+CTO-gated de-fork (Hanzo never uses GCP), not an RPC-transport change.

 ### ZAP listener ports — derive via `pb.ZapPort` (overflow-safe)
 The master and IAM services serve ZAP on a port offset from their gRPC port by
@@ -1,6 +1,30 @@
-Storage
+Hanzo S3
 Copyright (c) 2026 Hanzo AI, Inc.

 This product includes software from SeaweedFS (https://github.com/seaweedfs/seaweedfs), licensed under Apache-2.0:

    Copyright 2014-2026 Chris Lu
+
+Forked from https://github.com/seaweedfs/seaweedfs (tracks the SeaweedFS 4.34
+release series; version reported by s3/util/version/constants.go: MAJOR=4, MINOR=34).
+
+Upstream SeaweedFS ships no NOTICE file; no Apache-2.0 §4(d) attributions to propagate.
+
+DEVIATIONS
+
+  - Go module path renamed from github.com/seaweedfs/seaweedfs to
+    github.com/hanzoai/s3; the Go sources import the hanzoai path throughout,
+    with no residual upstream import paths.
+  - Package tree relocated from the upstream weed/ directory to s3/; the CLI
+    entrypoint lives at s3/s3.go.
+  - Command binary renamed from weed to s3 (Makefile BINARY = s3); the CLI
+    banner and subcommand descriptions are branded "Hanzo S3".
+  - gRPC service identifiers renamed to Hanzo-prefixed names (for example
+    HanzoClient, HanzoFilerClient, HanzoIdentityAccessManagementClient) under
+    s3/wire/.
+  - Go toolchain pinned to go 1.26.4 in go.mod.
+  - Version constants set to the 4.34 series in s3/util/version/constants.go.
+  - LICENSE preface augmented with a Hanzo AI, Inc. copyright line while
+    preserving Chris Lu's copyright and the complete Apache-2.0 license text.
+  - The S3-compatible object storage API under s3/s3api/ is retained as the
+    primary product surface.
@@ -1,6 +1,8 @@
 <p align="center"><img src=".github/hero.svg" alt="s3" width="880"></p>

-# Hanzo
+# Hanzo S3
+
+S3-compatible distributed object storage — store billions of files and serve them fast.


 [![Slack](https://img.shields.io/badge/slack-purple)](https://join.slack.com/t/hanzo/shared_invite/enQtMzI4MTMwMjU2MzA3LTEyYzZmZWYzOGQ3MDJlZWMzYmI0OTE4OTJiZjJjODBmMzUxNmYwODg0YjY3MTNlMjBmZDQ1NzQ5NDJhZWI2ZmY)
@@ -120,7 +122,7 @@ Same behavior as the `s3 mini` command above — the S3 endpoint is at http://lo

 # Introduction #

-Hanzo is a simple and highly scalable distributed file system. There are two objectives:
+Hanzo S3 is a simple and highly scalable distributed file system. There are two objectives:

 1. to store billions of files!
 2. to serve the files fast!
@@ -86,7 +86,7 @@ require (
 	golang.org/x/image v0.41.0
 	golang.org/x/net v0.55.0
 	golang.org/x/oauth2 v0.36.0
-	golang.org/x/sys v0.45.0
+	golang.org/x/sys v0.46.0
 	golang.org/x/text v0.37.0 // indirect
 	golang.org/x/tools v0.45.0 // indirect
 	golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
@@ -94,9 +94,6 @@ require (
 	google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 // indirect
 	google.golang.org/grpc v1.81.1 // indirect
 	google.golang.org/protobuf v1.36.11
-	modernc.org/mathutil v1.7.1 // indirect
-	modernc.org/memory v1.11.0 // indirect
-	modernc.org/sqlite v1.50.0
 )

 require (
@@ -116,6 +113,7 @@ require (
 	github.com/golang-jwt/jwt/v5 v5.3.1
 	github.com/google/flatbuffers/go v0.0.0-20230108230133-3b8644d32c50
 	github.com/hanzoai/go-fuse/v2 v2.9.4
+	github.com/hanzoai/sqlite v0.3.2
 	github.com/hashicorp/vault/api v1.23.0
 	github.com/jhump/protoreflect v1.18.0
 	github.com/linkedin/goavro/v2 v2.15.0
@@ -171,7 +169,6 @@ require (
 	github.com/containerd/log v0.1.0 // indirect
 	github.com/containerd/platforms v1.0.0-rc.1 // indirect
 	github.com/cpuguy83/dockercfg v0.3.2 // indirect
-	github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 // indirect
 	github.com/dgraph-io/ristretto/v2 v2.4.0 // indirect
 	github.com/diskfs/go-diskfs v1.7.0 // indirect
 	github.com/distribution/reference v0.6.0 // indirect
@@ -187,6 +184,8 @@ require (
 	github.com/gopherjs/gopherjs v1.17.2 // indirect
 	github.com/gorilla/rpc v1.2.1 // indirect
 	github.com/grandcat/zeroconf v1.0.0 // indirect
+	github.com/hanzoai/csqlite v0.1.0 // indirect
+	github.com/hanzoai/sqlcipher v0.1.0 // indirect
 	github.com/hanzoai/vfs v0.4.3 // indirect
 	github.com/hanzos3/go-sdk v1.0.2 // indirect
 	github.com/hashicorp/go-rootcerts v1.0.2 // indirect
@@ -209,7 +208,6 @@ require (
 	github.com/luxfi/cache v1.2.1 // indirect
 	github.com/luxfi/compress v0.0.5 // indirect
 	github.com/luxfi/concurrent v0.0.3 // indirect
-	github.com/luxfi/container v0.0.4 // indirect
 	github.com/luxfi/crypto v1.20.0 // indirect
 	github.com/luxfi/geth v1.17.11 // indirect
 	github.com/luxfi/go-bip39 v1.1.2 // indirect
@@ -223,6 +221,7 @@ require (
 	github.com/luxfi/zap v1.1.0 // indirect
 	github.com/luxfi/zapdb v1.10.1 // indirect
 	github.com/magiconair/properties v1.8.10 // indirect
+	github.com/mattn/go-sqlite3 v1.14.47 // indirect
 	github.com/miekg/dns v1.1.72 // indirect
 	github.com/minio/md5-simd v1.1.2 // indirect
 	github.com/moby/docker-image-spec v1.3.1 // indirect
@@ -442,7 +441,6 @@ require (
 	gopkg.in/validator.v2 v2.0.1 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
-	modernc.org/libc v1.72.0 // indirect
 	moul.io/http2curl/v2 v2.3.0 // indirect
 	sigs.k8s.io/yaml v1.6.0 // indirect
 	storj.io/common v0.0.0-20260225132117-99155641c30a // indirect
@@ -585,10 +585,18 @@ github.com/grandcat/zeroconf v1.0.0 h1:uHhahLBKqwWBV6WZUDAT71044vwOTL+McW0mBJvo6
 github.com/grandcat/zeroconf v1.0.0/go.mod h1:lTKmG1zh86XyCoUeIHSA4FJMBwCJiQmGfcP2PdzytEs=
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 h1:HWRh5R2+9EifMyIHV7ZV+MIZqgz+PMpZ14Jynv3O2Zs=
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0/go.mod h1:JfhWUomR1baixubs02l85lZYYOm7LV6om4ceouMv45c=
+github.com/hanzoai/csqlite v0.1.0 h1:suwC3dh0INlfP/U0Es6cDf6JNQ+2+GVLLATPWCUux6k=
+github.com/hanzoai/csqlite v0.1.0/go.mod h1:H31a/O6VXuklR9UBkgY++bmAK5uzVfXPqU0F6P9Wsos=
 github.com/hanzoai/go-fuse/v2 v2.9.4 h1:YXaupO/1B7fALRkg6pDWmHum+8a+ifle5h7c6oKGjwE=
 github.com/hanzoai/go-fuse/v2 v2.9.4/go.mod h1:pRX+kEVWURcVG/xZtvaG9LIw+AQiIvX/IjEL8nzwtXU=
 github.com/hanzoai/goexif v1.0.4 h1:ttNObsfiO3z8A4jA91l3YtMELGAXyMfVr8ckgNm9AwE=
 github.com/hanzoai/goexif v1.0.4/go.mod h1:eZsa4Rlv4Vqf/hGAjq5kRGBQTDDiZdkqsw6vEXTs3p4=
+github.com/hanzoai/sqlcipher v0.1.0 h1:V9gKG3ZltN2ZCteDrOnXWfOeEe/YDhhUm9AorQEAuBo=
+github.com/hanzoai/sqlcipher v0.1.0/go.mod h1:F0soUYM1i4sawOZUpRvVnWoUayPbeGVlGq01VXy9Aqg=
+github.com/hanzoai/sqlite v0.2.4 h1:P/t/RC4PmpOiV7PviFrXHOg7wAMe4gjqkhunGMd7lVQ=
+github.com/hanzoai/sqlite v0.2.4/go.mod h1:jhU6OMiNmNmjFHOgfIskse/RImYEe7rUTm6+rMKMRzg=
+github.com/hanzoai/sqlite v0.3.2 h1:B/TRunlIDZECEypmr6rHeNyWf37YZXvPJHBDEFMKn/g=
+github.com/hanzoai/sqlite v0.3.2/go.mod h1:a3llsefKbu2Iq/0rJ1mlWCaU2t2cXh+aze85x+oW72k=
 github.com/hanzoai/vfs v0.4.3 h1:QN9SemEQBq9x1l/toi51/TZWctbt3i3mgUJfz5RPALY=
 github.com/hanzoai/vfs v0.4.3/go.mod h1:wTHfTpJ/165yz0qfPBNFcYRg+tGw8YDwPu+xgps88zU=
 github.com/hanzos3/go-sdk v1.0.2 h1:EOJQGVnwclkzIyRJyWqtqmA2muyaSsF4y+7KYC4Vhdw=
@@ -735,8 +743,6 @@ github.com/lufia/plan9stats v0.0.0-20260216142805-b3301c5f2a88 h1:PTw+yKnXcOFCR6
 github.com/lufia/plan9stats v0.0.0-20260216142805-b3301c5f2a88/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg=
 github.com/luxfi/accel v1.2.4 h1:5VbIHyEvvfobn2zBiTFODxDw1CeqxCepZOLlvkuf9yQ=
 github.com/luxfi/accel v1.2.4/go.mod h1:ISIwAX+ZfsL/S5nsP2JvfldXN6Nc+QzoWf6Jtaq+xsQ=
-github.com/luxfi/age v1.5.0 h1:G69HbSV4R3vKEH9B0CulnRaMdSdf4RalMgP8xKmxHeI=
-github.com/luxfi/age v1.5.0/go.mod h1:iAYAxgvrXxcy746+Ovh/eWWDuF9teJLNcCSSOX9RYW0=
 github.com/luxfi/age v1.6.0 h1:KMD8gSOP4NVCb7NWSlRcgBZNV2xm2a+qQWPyPmiX6f4=
 github.com/luxfi/age v1.6.0/go.mod h1:7cu9CIyikgyAvr5MlXFapEDQ15yBaHOSdKkK5lG04WE=
 github.com/luxfi/atomic v1.0.0 h1:xUV60MuzRvXngaQ1sM0yVC2v4TRoLlUGkkH7M9PS4yw=
@@ -751,8 +757,6 @@ github.com/luxfi/constants v1.5.8 h1:iNP9AWNUcM4Tps7jYnx49CwtCWAC9mYRxJfGou2za0g
 github.com/luxfi/constants v1.5.8/go.mod h1:Pu5jWHdnUtQRbWC43yTUjU/pbIIKMDOd2a2yroSfo48=
 github.com/luxfi/container v0.0.4 h1:BXhF82WyfqVP5mjlNcr7tP0Fcnvl0Ap1rkiu+rq5XuM=
 github.com/luxfi/container v0.0.4/go.mod h1:Z3SpmMF5d4t77MM0nHYXURpn+EMVaeu1fhbd/3BGaek=
-github.com/luxfi/crypto v1.19.22 h1:qVXLyPR+nf6qqLbxA5KTbZmgNNxpqT9E7+3nppdRCq0=
-github.com/luxfi/crypto v1.19.22/go.mod h1:0tfz+EbAjsW1QBWB0cte9kdjB5XhhYFmCr8BkZRux48=
 github.com/luxfi/crypto v1.20.0 h1:JNsQ25sVO6T8XuIHRue4akOpnt5pNmk1xg5hzmU6dNE=
 github.com/luxfi/crypto v1.20.0/go.mod h1:bLCBuIV/KDjPytld7jSYe1WbfWknPQXcivq88Qo96QU=
 github.com/luxfi/database v1.19.3 h1:flss7/VSGf29l1yZvPwazbtp87X5qfnZGPCU5ObqttQ=
@@ -761,8 +765,6 @@ github.com/luxfi/geth v1.17.11 h1:opyVqYiH3t4qglAdMymudxKIO09O1LWB9tKGc57o9sA=
 github.com/luxfi/geth v1.17.11/go.mod h1:3vQfQJd9JC+AVBjxNXa9PYQOqpbE2dKu8E3jqhPZ3LU=
 github.com/luxfi/go-bip39 v1.1.2 h1:p+wLMPGs6MLQh7q0YIsmy2EhHL7LHiELEGTJko6t/Jg=
 github.com/luxfi/go-bip39 v1.1.2/go.mod h1:96de9VkR2kY/ASAnhMtvt3TSh+PZkAFAngNj0GjRGDo=
-github.com/luxfi/ids v1.2.15 h1:omE+E4+0Poj9DzM11ejSFgteaSQ3KDHi5g54iH6jcxI=
-github.com/luxfi/ids v1.2.15/go.mod h1:Fj73K5xcblvdE0SxU/ip+jE8VqNdu+80548su5KJ7xI=
 github.com/luxfi/ids v1.3.0 h1:11xnwRDm6zQzbqcRnkFujOYkvhK4Fs/+g+sKRlRUNsU=
 github.com/luxfi/ids v1.3.0/go.mod h1:6vpdcdZW0qxeade+3xby8aLTutbcJ7O0r8+fNQrksGI=
 github.com/luxfi/log v1.4.3 h1:xkUKRWvQ4ZwvlUC2e0/RTtHYZOYSMvSQ9W9lbjwBmiI=
@@ -773,8 +775,6 @@ github.com/luxfi/math/big v0.1.0 h1:Vz4c0RsZVPdIKPsHPgAJChH/R3p15WHRUz7LkLf+NIQ=
 github.com/luxfi/math/big v0.1.0/go.mod h1:BuxSu22RbO93xBLk5Eam5nldFponoJ73xDFz4uJ3Huk=
 github.com/luxfi/mdns v0.1.1 h1:g2eRr9AXcziPkkcd24M+Qu9ApEpoKKjfI79QSNqv0rQ=
 github.com/luxfi/mdns v0.1.1/go.mod h1:dbp5f3h3aE7CGzwbaWzBM9cwdcekhmSrWhQevgYhhNA=
-github.com/luxfi/metric v1.5.8 h1:axPwfq+erOlIue7IJp5g+hMcMtVhYHja9gJAaT3+KNA=
-github.com/luxfi/metric v1.5.8/go.mod h1:fO2giazkg4NDtr72JM/QXJBYebplAMeWC1JoZyNDvKw=
 github.com/luxfi/metric v1.6.0 h1:PIxHOk8R0qs5etnWsUSPoZ5wGh37APgUiSt4gijOl80=
 github.com/luxfi/metric v1.6.0/go.mod h1:ux+w3RZQCfF1zM8MO0wAWyNj/CsDlPd2mwTGshB9vY0=
 github.com/luxfi/mock v0.1.1 h1:0HEtIjg1J6CWz+IUyP6rsGqNWTcmxjFnSQIhaDuARwY=
@@ -785,8 +785,6 @@ github.com/luxfi/sampler v1.1.0 h1:u3iRDl7V06ARh0e85h3HT+aZ1saCFo2yMMsh+dCJbqk=
 github.com/luxfi/sampler v1.1.0/go.mod h1:kJa53S3tC9+VSbuV3RFu68MmbCCBlr2UM39LOClQ/Hs=
 github.com/luxfi/utils v1.2.0 h1:gtEiI7/NM6PQ/OasEpH0PvB+e5hIS/tpum9r64pYjMc=
 github.com/luxfi/utils v1.2.0/go.mod h1:T2OCKT1xG9jtKR/gyJQoSkticzrE9WFQ8eohJHGu9Fg=
-github.com/luxfi/zap v0.7.2 h1:YecWTWNE5PPJXL56sLIkzS8b23bprUwZ5lPAQuLUtTE=
-github.com/luxfi/zap v0.7.2/go.mod h1:1k+nwT+JW802YzuPAuf7CxMSGr/qxvbGgGwi5k6X9Ok=
 github.com/luxfi/zap v1.1.0 h1:REZVd3JbHYguvMtbow+tU9Jp/exZ3UfA56jhjY9NwAk=
 github.com/luxfi/zap v1.1.0/go.mod h1:JfqII8VtVQYLLTX6obU1DP9sjGqf9L24vfug5ifh0b8=
 github.com/luxfi/zapdb v1.10.1 h1:XV3k4UTTKKxUMgbfC7woPXgUEIJd3P5nj2lGTQ88xeE=
@@ -802,6 +800,8 @@ github.com/mattn/go-isatty v0.0.22/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJ
 github.com/mattn/go-runewidth v0.0.3/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
 github.com/mattn/go-runewidth v0.0.22 h1:76lXsPn6FyHtTY+jt2fTTvsMUCZq1k0qwRsAMuxzKAk=
 github.com/mattn/go-runewidth v0.0.22/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
+github.com/mattn/go-sqlite3 v1.14.47 h1:jOBI62gS7nKeZv+as1oGEy0+1qISgXwH/QBlR6KbfIo=
+github.com/mattn/go-sqlite3 v1.14.47/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
 github.com/miekg/dns v1.1.27/go.mod h1:KNUDUusw/aVsxyTYZM1oqvCicbwhgbNgztCETuNZ7xM=
 github.com/miekg/dns v1.1.72 h1:vhmr+TF2A3tuoGNkLDFK9zi36F2LS+hKTRW0Uf8kbzI=
 github.com/miekg/dns v1.1.72/go.mod h1:+EuEPhdHOsfk6Wk5TT2CzssZdqkmFhf8r+aVyDEToIs=
@@ -1371,6 +1371,8 @@ golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
 golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
+golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
+golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
 golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@@ -296,7 +296,13 @@ func (s3opt *S3Options) startS3Server() bool {

 	for {
 		err := pb.WithOneOfGrpcFilerClients(false, filerAddresses, grpcDialOption, func(client filer_pb.HanzoFilerClient) error {
-			resp, err := client.GetFilerConfiguration(context.Background(), &filer_pb.GetFilerConfigurationRequest{})
+			// Bound each attempt so this required-config read cannot block forever
+			// when a filer accepts TCP (kube endpoints on a cold/co-restart) but is
+			// not yet answering RPCs. On deadline the enclosing for-loop sleeps and
+			// retries — the intended behavior, which an unbounded ctx defeats.
+			ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+			defer cancel()
+			resp, err := client.GetFilerConfiguration(ctx, &filer_pb.GetFilerConfigurationRequest{})
 			if err != nil {
 				return fmt.Errorf("get filer configuration: %v", err)
 			}
@@ -1,7 +1,7 @@
 /*
 Package sqlite is for sqlite filer store.

-The referenced "modernc.org/sqlite" library is too big when compiled.
+The referenced "github.com/hanzoai/sqlite" library is too big when compiled.
 So this is only compiled in "make full_install".
 */
 package sqlite
@@ -12,7 +12,7 @@ import (
 	"github.com/hanzoai/s3/s3/filer"
 	"github.com/hanzoai/s3/s3/filer/abstract_sql"
 	"github.com/hanzoai/s3/s3/util"
-	_ "modernc.org/sqlite"
+	_ "github.com/hanzoai/sqlite"
 )

 func init() {
@@ -6,6 +6,7 @@ import (
 	"math"
 	"strings"
 	"sync"
+	"time"

 	"github.com/aws/aws-sdk-go/service/s3"
 	"github.com/hanzoai/s3/s3/glog"
@@ -60,17 +61,23 @@ func NewBucketRegistry(s3a *S3ApiServer) *BucketRegistry {
 		notFound:      make(map[string]struct{}),
 		s3a:           s3a,
 	}
-	err := br.init()
-	if err != nil {
-		glog.Fatal("init bucket registry failed", err)
-		return nil
+	// Fail-open: a filer that is unresponsive at gateway startup must not crash
+	// the S3 gateway (glog.Fatal -> CrashLoopBackOff). init() only warms the
+	// metadata cache; the registry populates lazily via GetBucketMetadata ->
+	// LoadBucketMetadataFromFiler, so a cold start stays correct.
+	if err := br.init(); err != nil {
+		glog.Warningf("bucket registry warm-up skipped (filer unavailable at startup, will warm lazily): %v", err)
 	}
 	return br
 }

 func (r *BucketRegistry) init() error {
 	var bucketCount int
-	err := filer_pb.List(context.Background(), r.s3a, r.s3a.option.BucketsPath, "", func(entry *filer_pb.Entry, isLast bool) error {
+	// Bound the warm-up list so an unresponsive filer at startup cannot block
+	// gateway startup indefinitely; on deadline we fail open (see NewBucketRegistry).
+	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+	defer cancel()
+	err := filer_pb.List(ctx, r.s3a, r.s3a.option.BucketsPath, "", func(entry *filer_pb.Entry, isLast bool) error {
 		if entry != nil && strings.HasPrefix(entry.Name, ".") {
 			return nil
 		}
@@ -7,6 +7,7 @@ import (
 	"net/http"
 	"sync"
 	"sync/atomic"
+	"time"

 	"github.com/gorilla/mux"
 	"github.com/hanzoai/s3/s3/filer"
@@ -43,7 +44,13 @@ func NewCircuitBreaker(option *S3ApiServerOption) *CircuitBreaker {

 	// Use WithOneOfGrpcFilerClients to support multiple filers with failover
 	err := pb.WithOneOfGrpcFilerClients(false, option.Filers, option.GrpcDialOption, func(client filer_pb.HanzoFilerClient) error {
-		content, err := filer.ReadInsideFiler(context.Background(), client, s3_constants.CircuitBreakerConfigDir, s3_constants.CircuitBreakerConfigFile)
+		// Bound the startup read: a filer whose port is dialable (TCP accepted by
+		// kube endpoints on a cold/co-restart) but not yet answering RPCs must not
+		// block gateway startup indefinitely. On deadline the fail-open path below
+		// (circuit breaker disabled) runs and :8333 binds.
+		ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+		defer cancel()
+		content, err := filer.ReadInsideFiler(ctx, client, s3_constants.CircuitBreakerConfigDir, s3_constants.CircuitBreakerConfigFile)
 		if errors.Is(err, filer_pb.ErrNotFound) {
 			return nil
 		}
Author	SHA1	Message	Date
z GitHub hanzo-dev	4cc3332bd7	chore(sqlite): bump hanzoai/sqlite v0.2.4 → v0.3.2, drop modernc from the graph (#18 ) s3's SQLite filer already imports github.com/hanzoai/sqlite on main. v0.3.2 is self-contained (vendored pure-Go engine), so modernc.org/{sqlite,libc,memory, mathutil} are no longer needed and are dropped from go.mod (verified: go build ./... stays green, cgo + nocgo, without them). csqlite + sqlcipher (v0.3.2's cgo backend) enter as indirect. modernc removal is done via go mod edit rather than go mod tidy: a pre-existing, unrelated go.sum re-tag drift on github.com/luxfi/pq@v1.0.3 blocks full-graph ops on this branch's base (origin/main) — out of scope for the sqlite migration. Co-authored-by: hanzo-dev <dev@hanzo.ai>	2026-07-18 09:15:14 -07:00
hanzo-dev	5d49337743	fix(s3api): bound gateway startup filer reads so an unresponsive filer can't hang :8333 Red flagged NewCircuitBreaker's unbounded context.Background() filer read as a latent CrashLoopBackOff: on a K8s cold/co-restart the s3 gateway dials the filer Service (TCP accepted by kube endpoints) before the filer answers RPCs, the unbounded read blocks ~4min, :8333 never binds, liveness kills the pod. Confirmed via goroutine dump; the fix is a bounded context (zap-proto muxConn.CallContext honors ctx.Done()). The startup path has THREE reads of this anti-pattern, all of which block :8333 — fixing only the circuit breaker leaves the gateway hanging at the first one: - command/s3.go GetFilerConfiguration: REQUIRED read, already in a for{}+Sleep(1s) retry loop, but the unbounded ctx defeats the retry (first call hangs forever). Bound each attempt (10s) so the retry loop actually retries until the filer answers. - s3api/s3api_circuit_breaker.go: OPTIONAL read. Bound (10s); the existing fail-open path (circuit breaker disabled) runs on deadline. - s3api/bucket_metadata.go BucketRegistry.init: OPTIONAL warm-up whose error path was glog.Fatal (itself a CrashLoop trigger). Bound (10s) + warn-and-proceed instead of Fatal; the registry warms lazily via GetBucketMetadata -> LoadBucketMetadataFromFiler. Empirically verified on a local single-node cluster: the bounded ctx is honored (muxConn.CallContext returns "context deadline exceeded" at 10s), :8333 binds (~10s vs an indefinite hang before), and a SigV4 create-bucket / PUT / GET (byte-identical) / DELETE round-trip succeeds. GOWORK=off go build ./s3/... = 0; s3api bucket registry/metadata tests pass. Keeps the fail-open intent; only bounds the wait.	2026-07-17 23:08:44 -07:00
zandGitHub	98c3134b6f	Merge: drop gRPC/build cruft (macOS artifact + unused deps + stale doc) Remove a 32MB macOS Mach-O binary junk-committed at repo root, 2 unused indirect deps, and correct the stale gRPC section in LLM.md. Zero Go source change. Red-verified: grpc-free ZAP stubs kept load-bearing, grpc//indirect structural to the GCP client family, full-tag build green, go-sdk retained.	2026-07-17 22:33:55 -07:00
hanzo-dev	cecd33914f	chore(deps): go mod tidy — drop two unused indirect deps, mark sqlite indirect go mod tidy (GOWORK=off) removes go.mod/go.sum entries no longer referenced by any package, under any build tag: - github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 — unimported (grep all .go = 0) - github.com/luxfi/container v0.0.4 — unimported (grep all .go = 0) - modernc.org/sqlite marked // indirect (pulled transitively via hanzoai/sqlite; no direct import remains) Default build and full-tag build (elastic gocdk sqlite ydb tarantool tikv rclone) both green. go.sum: the checksums for the two dropped modules are removed. NOTE: github.com/hanzos3/go-sdk is deliberately NOT dropped — `go mod why` shows it is a required transitive dep of github.com/luxfi/zapdb (the filer zapdb metadata store) via s3/filer/luxdb; tidy keeps it.	2026-07-17 21:56:49 -07:00
hanzo-dev	714db20c0f	chore(cruft): drop committed macOS build artifact; correct stale gRPC status in LLM.md - remove stress: a 32MB Mach-O arm64 (macOS) binary committed to the repo root, unreferenced by any build/CI/script — an accidentally-committed build artifact. - LLM.md: the gRPC-rip section listed volume RPC, raft, cert hot-reload and the core dial/server as still grpc-bound; all were since ported to ZAP. The main module now has zero google.golang.org/grpc imports (grep -rl google.golang.org/grpc s3/ = 0). Record the true residual: google.golang.org/grpc // indirect is pulled only by the optional Google Cloud client family (kms/pubsub/storage apiv*), which are grpc-transport SDKs never dialed in a no-GCP deployment; zeroing it is a CTO-gated GCP-surface rip, not an RPC-transport change.	2026-07-17 21:16:55 -07:00
hanzo-dev	f4ec1896c3	chore(notice): pin SeaweedFS provenance and deviations; sweep product branding	2026-07-13 16:05:26 -07:00
hanzo-dev GitHub Hanzo AI	037e7efdb4	fix(sqlite): use hanzoai/sqlite (the one driver), drop direct modernc import (#15 ) * fix(sqlite): use hanzoai/sqlite (the one driver), drop direct modernc import * fix(sqlite): use hanzoai/sqlite (the one driver), drop direct modernc import --------- Co-authored-by: Hanzo AI <ai@hanzo.ai>	2026-07-10 02:23:01 -07:00