Hanzo RedandHanzo Dev
c5f918ac74
OnBucketCreation and OnBucketDeletion take the bucket name straight from a
filer metadata event and hand it to CreateTable/deleteTable, where the SQL
generator interpolates it as a table identifier — a position no bind
parameter can occupy. Their sibling getTxOrDB gates the same name through
isValidBucket before it reaches SQL; these two hooks did not, so a name
carrying stacked statements executed verbatim.
A directory planted under /buckets/ drives these hooks. The filer
mkdir/gRPC create path only runs VerifyS3BucketName when it has to
auto-create a missing /buckets/<name> parent, so creating the bucket
directory itself as a leaf leaves the name unchecked; the event then
crosses to every subscribed filer, each of which runs the hook against its
own store. Emitted DROP was `drop table `realbucket`;drop table `victim``;
emitted CREATE stacked a DROP after the interpolated identifier. Without a
guard OnBucketDeletion("filemeta") also dropped the shared default table.
Gate both hooks through isValidBucket, the same check getTxOrDB uses, so a
malformed name is refused before any SQL runs. Regression test asserts the
bystander table survives the payloads and that a well-formed bucket still
creates and drops its table.
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
Hanzo S3
Hanzo S3 is S3-compatible distributed object storage: a master, volume servers, a filer and
an S3 gateway, all in one Go binary called s3.
Install
Download a binary for your platform from
Releases — s3-linux-amd64,
s3-linux-arm64, s3-darwin-amd64, s3-darwin-arm64, s3-windows-amd64.exe — or build
it yourself:
go install github.com/hanzoai/s3/s3@latest
There is also an image at ghcr.io/hanzoai/s3.
On macOS, if the downloaded binary is quarantined: xattr -d com.apple.quarantine ./s3.
First run
s3 mini brings up the whole stack — master, volume, filer, S3 gateway — with credentials
and a bucket already created:
AWS_ACCESS_KEY_ID=admin \
AWS_SECRET_ACCESS_KEY=secret \
S3_BUCKET=my-bucket \
s3 mini -dir=/data
The same run in Docker:
docker run -p 8333:8333 \
-e AWS_ACCESS_KEY_ID=admin \
-e AWS_SECRET_ACCESS_KEY=secret \
-e S3_BUCKET=my-bucket \
ghcr.io/hanzoai/s3
Point any S3 client at it:
import boto3
s3 = boto3.client(
"s3", endpoint_url="http://localhost:8333",
aws_access_key_id="admin", aws_secret_access_key="secret",
region_name="us-east-1",
)
s3.put_object(Bucket="my-bucket", Key="hello.txt", Body=b"hi")
print(s3.get_object(Bucket="my-bucket", Key="hello.txt")["Body"].read())
What s3 mini starts:
| Service | Address |
|---|---|
| S3 endpoint | http://localhost:8333 |
| Master UI | http://localhost:9333 |
| Volume server | http://localhost:9340 |
| Filer UI | http://localhost:8888 |
| WebDAV | http://localhost:7333 |
| Iceberg REST catalog | http://localhost:8181 |
| Admin UI | http://localhost:23646 |
S3_BUCKET takes a comma-separated list. S3_TABLE_BUCKET creates S3 Tables (Iceberg)
buckets. Leave the AWS keys out and the S3 gateway runs unauthenticated, which is fine for
local development and nothing else.
Running it for real
s3 mini is one process for convenience. In a cluster you run the components separately:
s3 master -mdir=/data/master
s3 volume -dir=/data/vol1 -master=<master_host>:9333 -port=8081
s3 filer -master=<master_host>:9333
s3 s3 -filer=<filer_host>:8888 -port=8333
Add volume servers to add capacity — each one registers with the master and starts taking
writes. s3 -h lists every command; s3 <command> -h lists its flags. s3 scaffold
writes starter config files.
Beyond the S3 API, the filer gives you POSIX-shaped directories over HTTP, FUSE mounting
(s3 mount), WebDAV, cross-cluster replication (s3 filer.sync), tiering to remote object
stores, erasure coding for warm data, and a built-in Iceberg REST catalog so Spark, Trino,
DuckDB and friends can read tables without a separate metastore. Kubernetes manifests are
in k8s/, Docker Compose files in docker/.
Clients
Any S3 client works. Ours for Go is hanzoai/s3-go,
module github.com/hanzos3/go. This repository is the server.
One name
The product is Hanzo S3. hanzoai/storage and hanzoai/storage-go are old paths that
redirect here and to hanzoai/s3-go — renames, not separate products. "Hanzo Storage"
anywhere in our copy is stale, not a second thing.
Docs
LLM.md in this repository is the deep reference: architecture, the filer
metadata store, the ZAP transport, and how S3 fits the rest of the platform.
docs.hanzo.ai covers the platform around it.
Lineage
Hanzo S3 is a fork of SeaweedFS at its 4.34
release series, Apache-2.0, copyright Chris Lu — see NOTICE. The upstream
binary is weed; ours is s3, and every import path is github.com/hanzoai/s3. The
storage design it inherits comes from Facebook's Haystack paper, with erasure coding after
f4. Two support libraries are forked alongside it, each keeping its own upstream licence:
hanzoai/goexif (BSD-2-Clause) and hanzoai/go-fuse (BSD-3-Clause).
License
Apache-2.0 — see LICENSE.