mirror of
https://github.com/hanzoai/s3-csi.git
synced 2026-09-30 23:52:32 +00:00
e8b89e78c3
* fix(docker): clone the fork that exists, at a commit that exists
The s3-mount image has never built. A global rename turned the upstream
seaweedfs reference into `github.com/s3/s3`, which is not a repository, so the
build died on `git clone ... exit code 128` before compiling anything.
Three more things were wrong behind it, each of which would have failed the next
step in turn:
clone github.com/s3/s3 -> github.com/hanzoai/s3
path /go/src/github.com/s3/s3 -> /go/src/github.com/hanzoai/s3
dir cd weed -> cd s3 (the fork renamed it)
commit 19ce7c0b6fdb -> 6dda4e4d6bca
That commit is the real one. 19ce7c0b6fdb is an UPSTREAM seaweedfs hash and does
not exist in our fork — the API 422s on it — so the checkout could never have
succeeded even with the URL fixed. The ARG stays so a build can still pin.
SEAWEEDFS_COMMIT is renamed S3_COMMIT while we are here; nothing consumed it yet.
* chore: no seaweed, no weed — it is s3
369 occurrences across 33 files, plus nine files whose names carried it. All of
it gone: manifests, helm chart, nomad jobs, workflows, docs and code.
Two of these were not cosmetic.
The driver spawned a binary called "weed":
DefaultWeedBinary = "weed"
Our fork builds it as `s3` — the running filer pod has /usr/bin/s3 and no weed
at all — so every mount would have failed with an executable that is not there,
however well the image built.
And the crash-recovery integration test killed processes matching `pgrep -x
weed`. Nothing is named that any more, so it would have found no pids, killed
nothing, and passed while testing exactly nothing. It now matches `s3`. The test
already had a guard that fails when there is nothing to kill, which is the only
reason this would have been noticed at all.
The rest follows the same rename the Dockerfile got: SEAWEEDFS_COMMIT is
S3_COMMIT, and the release/sync workflows that patch that ARG were updated with
it or they would silently stop pinning anything.
No upstream import paths were touched — go.mod depends on github.com/hanzoai/s3,
not on anything upstream, so nothing here was load-bearing for the build.
Verified: go build clean, all three test packages pass.
---------
Co-authored-by: hanzo-dev <[email protected]>
Co-authored-by: Hanzo Dev <[email protected]>
Example of using s3 with HashiCorp Nomad
Running s3 cluster
You can skip this part if you have already running s3.
Assumptions:
- Running Nomad cluster
- At least 3 nodes with static IP addresses
- Enabled memory oversubscription (https://learn.hashicorp.com/tutorials/nomad/memory-oversubscription?in=nomad%2Fadvanced-scheduling)
- Running PostgreSQL instance for filer
export NOMAD_ADDR=http://nomad.service.consul:4646
nomad run s3.hcl
S3 master will be available on http://s3-master.service.consul:9333/
S3 filer will be available on http://s3-filer.service.consul:8888/
Running CSI
The CSI driver is split into two components that register under the same
csi_plugin id (s3):
- controller (
s3-csi-controller.hcl) — a singleservicejob that implements the volume lifecycle RPCs. Nomad calls it fornomad volume create/nomad volume delete. It only talks to the filer. - node (
s3-csi.hcl) — asystemjob that runs on every worker and stages/publishes volumes into allocations, using thes3-mountsidecar over a shared unix socket.
You need both running. With only the node plugin, nomad volume create
fails with plugin has no controller.
export NOMAD_ADDR=http://nomad.service.consul:4646
# Start CSI controller (one instance) and node (one per worker)
nomad run s3-csi-controller.hcl
nomad run s3-csi.hcl
# Wait until the plugin reports a healthy controller and the expected node count
nomad plugin status s3
# Create volume
nomad volume create example-s3-volume.hcl
# Start sample app
nomad run example-s3-app.hcl
If you only run the node plugin (no controller), you cannot
nomad volume create. Instead, pre-create the bucket/directory in S3 yourself andnomad volume register example-s3-volume.hclto register the existing volume.