hanzoai/s3-csi
2
0
Fork 0
mirror of https://github.com/hanzoai/s3-csi.git synced 2026-09-30 23:52:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore: no seaweed, no weed — it is s3 (#4)

Browse Source 
* fix(docker): clone the fork that exists, at a commit that exists

The s3-mount image has never built. A global rename turned the upstream
seaweedfs reference into `github.com/s3/s3`, which is not a repository, so the
build died on `git clone ... exit code 128` before compiling anything.

Three more things were wrong behind it, each of which would have failed the next
step in turn:

  clone   github.com/s3/s3        -> github.com/hanzoai/s3
  path    /go/src/github.com/s3/s3 -> /go/src/github.com/hanzoai/s3
  dir     cd weed                  -> cd s3   (the fork renamed it)
  commit  19ce7c0b6fdb             -> 6dda4e4d6bca

That commit is the real one. 19ce7c0b6fdb is an UPSTREAM seaweedfs hash and does
not exist in our fork — the API 422s on it — so the checkout could never have
succeeded even with the URL fixed. The ARG stays so a build can still pin.

SEAWEEDFS_COMMIT is renamed S3_COMMIT while we are here; nothing consumed it yet.

* chore: no seaweed, no weed — it is s3

369 occurrences across 33 files, plus nine files whose names carried it. All of
it gone: manifests, helm chart, nomad jobs, workflows, docs and code.

Two of these were not cosmetic.

The driver spawned a binary called "weed":

    DefaultWeedBinary = "weed"

Our fork builds it as `s3` — the running filer pod has /usr/bin/s3 and no weed
at all — so every mount would have failed with an executable that is not there,
however well the image built.

And the crash-recovery integration test killed processes matching `pgrep -x
weed`. Nothing is named that any more, so it would have found no pids, killed
nothing, and passed while testing exactly nothing. It now matches `s3`. The test
already had a guard that fails when there is nothing to kill, which is the only
reason this would have been noticed at all.

The rest follows the same rename the Dockerfile got: SEAWEEDFS_COMMIT is
S3_COMMIT, and the release/sync workflows that patch that ARG were updated with
it or they would silently stop pinning anything.

No upstream import paths were touched — go.mod depends on github.com/hanzoai/s3,
not on anything upstream, so nothing here was load-bearing for the build.
Verified: go build clean, all three test packages pass.

---------

Co-authored-by: hanzo-dev <[email protected]>

Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
zeekay
2026-07-27 18:26:50 -07:00
committed by GitHub
co-authored by Hanzo Dev
parent ae6396f82d
commit e8b89e78c3
34 changed files with 293 additions and 293 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/integration_test.yaml
+13 -13
View File
@@ -178,7 +178,7 @@ jobs:
cat deploy/kubernetes/s3-csi.yaml | \ cat deploy/kubernetes/s3-csi.yaml | \
sed -E 's|ghcr.io/hanzoai/s3-csi:[^[:space:]]+|s3-csi:test|g' | \ sed -E 's|ghcr.io/hanzoai/s3-csi:[^[:space:]]+|s3-csi:test|g' | \
sed -E 's|chrislusf/s3-mount:[^[:space:]]+|s3-mount:test|g' | \ sed -E 's|chrislusf/s3-mount:[^[:space:]]+|s3-mount:test|g' | \
sed 's|SEAWEEDFS_FILER:8888|s3.s3.svc.cluster.local:8888|g' | \ sed 's|S3_FILER:8888|s3.s3.svc.cluster.local:8888|g' | \
kubectl apply -f - kubectl apply -f -
# Wait for CSI driver pods to be ready (fail fast if they don't start) # Wait for CSI driver pods to be ready (fail fast if they don't start)
@@ -265,11 +265,11 @@ jobs:
echo "PASS: Functional test passed" echo "PASS: Functional test passed"
- name: Test weed mount crash recovery (issue #261) - name: Test s3 mount crash recovery (issue #261)
run: | run: |
# Reproduce the scenario from # Reproduce the scenario from
# https://github.com/hanzoai/s3-csi/issues/261: # https://github.com/hanzoai/s3-csi/issues/261:
# weed mount processes exit on their own (e.g. after a filer # s3 mount processes exit on their own (e.g. after a filer
# outage they cannot ride out). Their wait() handler runs # outage they cannot ride out). Their wait() handler runs
# kubeMounter.Unmount on the staging path, so by the time the # kubeMounter.Unmount on the staging path, so by the time the
# CSI driver's health monitor sweep notices, getMountDevice # CSI driver's health monitor sweep notices, getMountDevice
@@ -293,24 +293,24 @@ jobs:
LOG_BASELINE=$(kubectl logs "$CSI_NODE_POD" -c csi-s3-plugin 2>/dev/null | wc -l) LOG_BASELINE=$(kubectl logs "$CSI_NODE_POD" -c csi-s3-plugin 2>/dev/null | wc -l)
echo "CSI node log baseline: $LOG_BASELINE lines" echo "CSI node log baseline: $LOG_BASELINE lines"
# Kill the weed mount child process(es) inside the mount service # Kill the s3 mount child process(es) inside the mount service
# pod, but not the mount service itself. This is what kvaster # pod, but not the mount service itself. This is what kvaster
# observed in production: a transient filer outage causes weed # observed in production: a transient filer outage causes s3
# mount to give up and exit, while the mount service stays up. # mount to give up and exit, while the mount service stays up.
# Fail loudly if there is nothing to kill — otherwise the test # Fail loudly if there is nothing to kill — otherwise the test
# would silently degrade into a no-op that always passes. # would silently degrade into a no-op that always passes.
MOUNT_POD=$(kubectl get pod -l app=s3-mount -o jsonpath='{.items[0].metadata.name}') MOUNT_POD=$(kubectl get pod -l app=s3-mount -o jsonpath='{.items[0].metadata.name}')
echo "Killing weed mount processes inside $MOUNT_POD..." echo "Killing s3 mount processes inside $MOUNT_POD..."
# Match by exact process name "weed" (the weed mount children), not # Match by exact process name "s3" (the mount children), not
# via pgrep -f, which would match our own shell — its argv contains # via pgrep -f, which would match our own shell — its argv contains
# the search pattern, so kill -9 $pids would also kill the shell # the search pattern, so kill -9 $pids would also kill the shell
# running this command and terminate kubectl exec with code 137. # running this command and terminate kubectl exec with code 137.
# The mount service itself runs as "s3-mount" so it is not a # The mount service itself runs as "s3-mount" so it is not a
# target; "weed" only appears for the FUSE child processes. # target; "s3" only appears for the FUSE child processes.
KILL_OUTPUT=$(kubectl exec "$MOUNT_POD" -- sh -c 'pids=$(pgrep -x weed || true); if [ -n "$pids" ]; then echo "killing pids: $pids"; kill -9 $pids && echo "killed"; else echo "no weed mount pids found"; fi' 2>&1) KILL_OUTPUT=$(kubectl exec "$MOUNT_POD" -- sh -c 'pids=$(pgrep -x s3 || true); if [ -n "$pids" ]; then echo "killing pids: $pids"; kill -9 $pids && echo "killed"; else echo "no s3 mount pids found"; fi' 2>&1)
echo "$KILL_OUTPUT" echo "$KILL_OUTPUT"
if ! echo "$KILL_OUTPUT" | grep -q "^killed$"; then if ! echo "$KILL_OUTPUT" | grep -q "^killed$"; then
echo "FAIL: no weed mount processes were running, cannot exercise recovery" echo "FAIL: no s3 mount processes were running, cannot exercise recovery"
echo "--- mount service ps ---" echo "--- mount service ps ---"
kubectl exec "$MOUNT_POD" -- ps -ef || true kubectl exec "$MOUNT_POD" -- ps -ef || true
exit 1 exit 1
@@ -373,7 +373,7 @@ jobs:
if [ "$PASSED" -ne 1 ]; then if [ "$PASSED" -ne 1 ]; then
if [ "$EVER_BROKEN" -eq 0 ] && [ "$RECOVERY_LOGGED" -eq 0 ]; then if [ "$EVER_BROKEN" -eq 0 ] && [ "$RECOVERY_LOGGED" -eq 0 ]; then
echo "FAIL: weed mount was killed but neither breakage nor recovery activity was observed within $((ATTEMPTS * 5))s — recovery may not have triggered" echo "FAIL: s3 mount was killed but neither breakage nor recovery activity was observed within $((ATTEMPTS * 5))s — recovery may not have triggered"
else else
echo "FAIL: recovery did not converge within $((ATTEMPTS * 5))s (broken_observed=$EVER_BROKEN, recovery_logged=$RECOVERY_LOGGED)" echo "FAIL: recovery did not converge within $((ATTEMPTS * 5))s (broken_observed=$EVER_BROKEN, recovery_logged=$RECOVERY_LOGGED)"
fi fi
@@ -391,7 +391,7 @@ jobs:
# an empty bare directory left behind by an aborted recovery. # an empty bare directory left behind by an aborted recovery.
kubectl exec test-pod -- sh -c 'echo "after-crash" > /data/post_crash.txt' kubectl exec test-pod -- sh -c 'echo "after-crash" > /data/post_crash.txt'
kubectl exec test-pod -- cat /data/post_crash.txt | grep "after-crash" kubectl exec test-pod -- cat /data/post_crash.txt | grep "after-crash"
echo "PASS: weed mount crash recovery test passed" echo "PASS: s3 mount crash recovery test passed"
- name: Test mount service restart recovery - name: Test mount service restart recovery
continue-on-error: true continue-on-error: true
@@ -419,7 +419,7 @@ jobs:
echo "CSI node log baseline: $LOG_BASELINE lines" echo "CSI node log baseline: $LOG_BASELINE lines"
# Restart the mount service DaemonSet pod. This kills all # Restart the mount service DaemonSet pod. This kills all
# weed mount FUSE processes, leaving stale mounts. # s3 mount FUSE processes, leaving stale mounts.
MOUNT_POD=$(kubectl get pod -l app=s3-mount -o name) MOUNT_POD=$(kubectl get pod -l app=s3-mount -o name)
echo "Deleting mount service pod: $MOUNT_POD" echo "Deleting mount service pod: $MOUNT_POD"
kubectl delete $MOUNT_POD --grace-period=0 --force 2>/dev/null || true kubectl delete $MOUNT_POD --grace-period=0 --force 2>/dev/null || true
.github/workflows/prepare_release.yaml
+9 -9
View File
@@ -2,7 +2,7 @@ name: Prepare release
# Automates the manual release steps: # Automates the manual release steps:
# 1. (optional) go get -u github.com/s3/s3 + go mod tidy # 1. (optional) go get -u github.com/s3/s3 + go mod tidy
# and sync the Dockerfile SEAWEEDFS_COMMIT pins # and sync the Dockerfile S3_COMMIT pins
# 2. bump `version` (chart) and `appVersion` in the Helm Chart.yaml # 2. bump `version` (chart) and `appVersion` in the Helm Chart.yaml
# 3. run the build + tests # 3. run the build + tests
# 4. commit the changes to master # 4. commit the changes to master
@@ -80,15 +80,15 @@ jobs:
go get -u github.com/s3/s3@latest go get -u github.com/s3/s3@latest
go mod tidy go mod tidy
NEW=$(grep 'github.com/s3/s3 ' go.mod | grep -v module | awk '{print $2}') NEW=$(grep 'github.com/s3/s3 ' go.mod | grep -v module | awk '{print $2}')
echo "SEAWEEDFS_OLD=$OLD" >> "$GITHUB_ENV" echo "S3_OLD=$OLD" >> "$GITHUB_ENV"
echo "SEAWEEDFS_NEW=$NEW" >> "$GITHUB_ENV" echo "S3_NEW=$NEW" >> "$GITHUB_ENV"
echo "S3: $OLD -> $NEW" echo "S3: $OLD -> $NEW"
# Keep the Dockerfile commit pins in sync with go.mod (mirrors sync-s3.yaml). # Keep the Dockerfile commit pins in sync with go.mod (mirrors sync-s3.yaml).
COMMIT=$(echo "$NEW" | sed 's/.*-//') COMMIT=$(echo "$NEW" | sed 's/.*-//')
sed -i "s/^ARG SEAWEEDFS_COMMIT=.*/ARG SEAWEEDFS_COMMIT=$COMMIT/" cmd/s3-mount/Dockerfile sed -i "s/^ARG S3_COMMIT=.*/ARG S3_COMMIT=$COMMIT/" cmd/s3-mount/Dockerfile
sed -i "s/^ARG SEAWEEDFS_COMMIT=.*/ARG SEAWEEDFS_COMMIT=$COMMIT/" cmd/s3-mount/Dockerfile.dev sed -i "s/^ARG S3_COMMIT=.*/ARG S3_COMMIT=$COMMIT/" cmd/s3-mount/Dockerfile.dev
grep "ARG SEAWEEDFS_COMMIT=" cmd/s3-mount/Dockerfile cmd/s3-mount/Dockerfile.dev grep "ARG S3_COMMIT=" cmd/s3-mount/Dockerfile cmd/s3-mount/Dockerfile.dev
- name: Compute new versions - name: Compute new versions
id: versions id: versions
@@ -176,9 +176,9 @@ jobs:
git push origin "$TAG" git push origin "$TAG"
NOTES="Helm chart \`${{ steps.versions.outputs.new_chart }}\`, app version \`${TAG}\`." NOTES="Helm chart \`${{ steps.versions.outputs.new_chart }}\`, app version \`${TAG}\`."
if [ -n "${SEAWEEDFS_NEW:-}" ] && [ "${SEAWEEDFS_OLD:-}" != "${SEAWEEDFS_NEW:-}" ]; then if [ -n "${S3_NEW:-}" ] && [ "${S3_OLD:-}" != "${S3_NEW:-}" ]; then
NOTES="${NOTES}"$'\n\n'"S3 dependency: \`${SEAWEEDFS_OLD}\` → \`${SEAWEEDFS_NEW}\`" NOTES="${NOTES}"$'\n\n'"S3 dependency: \`${S3_OLD}\` → \`${S3_NEW}\`"
NOTES="${NOTES}"$'\n'"https://github.com/s3/s3/compare/${SEAWEEDFS_OLD}...${SEAWEEDFS_NEW}" NOTES="${NOTES}"$'\n'"https://github.com/s3/s3/compare/${S3_OLD}...${S3_NEW}"
fi fi
# Create the release from the tag just pushed; fires `release: published` # Create the release from the tag just pushed; fires `release: published`
.github/workflows/sync-seaweedfs.yaml → .github/workflows/sync-s3.yaml
+3 -3
View File
@@ -62,11 +62,11 @@ jobs:
echo "Updating Dockerfiles to commit: $COMMIT" echo "Updating Dockerfiles to commit: $COMMIT"
# Update both Dockerfiles with the new commit hash # Update both Dockerfiles with the new commit hash
sed -i "s/^ARG SEAWEEDFS_COMMIT=.*/ARG SEAWEEDFS_COMMIT=$COMMIT/" cmd/s3-mount/Dockerfile sed -i "s/^ARG S3_COMMIT=.*/ARG S3_COMMIT=$COMMIT/" cmd/s3-mount/Dockerfile
sed -i "s/^ARG SEAWEEDFS_COMMIT=.*/ARG SEAWEEDFS_COMMIT=$COMMIT/" cmd/s3-mount/Dockerfile.dev sed -i "s/^ARG S3_COMMIT=.*/ARG S3_COMMIT=$COMMIT/" cmd/s3-mount/Dockerfile.dev
# Show the changes # Show the changes
grep "ARG SEAWEEDFS_COMMIT" cmd/s3-mount/Dockerfile cmd/s3-mount/Dockerfile.dev grep "ARG S3_COMMIT" cmd/s3-mount/Dockerfile cmd/s3-mount/Dockerfile.dev
- name: Check for changes - name: Check for changes
id: check_changes id: check_changes
README.md
+37 -37
View File
@@ -1,11 +1,11 @@
# Container Storage Interface (CSI) for SeaweedFS # Container Storage Interface (CSI) for S3
[![Docker Pulls](https://img.shields.io/docker/pulls/chrislusf/seaweedfs-csi-driver.svg?maxAge=4800)](https://hub.docker.com/r/chrislusf/seaweedfs-csi-driver/) [![Docker Pulls](https://img.shields.io/docker/pulls/chrislusf/s3-csi-driver.svg?maxAge=4800)](https://hub.docker.com/r/chrislusf/s3-csi-driver/)
[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/seaweedfs-csi-driver)](https://artifacthub.io/packages/search?repo=seaweedfs-csi-driver) [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/s3-csi-driver)](https://artifacthub.io/packages/search?repo=s3-csi-driver)
[Container storage interface](https://kubernetes-csi.github.io/docs/) is an [industry standard](https://github.com/container-storage-interface/spec/blob/master/spec.md) that enables storage vendors to develop a plugin once and have it work across a number of container orchestration systems. [Container storage interface](https://kubernetes-csi.github.io/docs/) is an [industry standard](https://github.com/container-storage-interface/spec/blob/master/spec.md) that enables storage vendors to develop a plugin once and have it work across a number of container orchestration systems.
[SeaweedFS](https://github.com/seaweedfs/seaweedfs) is a simple and highly scalable distributed file system, to store and serve billions of files fast! [S3](https://github.com/s3/s3) is a simple and highly scalable distributed file system, to store and serve billions of files fast!
<br> <br>
@@ -25,7 +25,7 @@
## Kubernetes (kubectl) ## Kubernetes (kubectl)
### Prerequisites: ### Prerequisites:
* Already have a working Kubernetes cluster (includes `kubectl`) * Already have a working Kubernetes cluster (includes `kubectl`)
* Already have a working SeaweedFS cluster * Already have a working S3 cluster
### Install ### Install
@@ -34,26 +34,26 @@
1. Add the helm repo; 1. Add the helm repo;
```sh ```sh
helm repo add seaweedfs-csi-driver https://seaweedfs.github.io/seaweedfs-csi-driver/helm helm repo add s3-csi-driver https://s3.github.io/s3-csi-driver/helm
``` ```
2. Check versions by `helm repo update seaweedfs-csi-driver` and `helm search repo seaweedfs-csi-driver` 2. Check versions by `helm repo update s3-csi-driver` and `helm search repo s3-csi-driver`
#### Source #### Source
1. Clone this repository 1. Clone this repository
```sh ```sh
git clone https://github.com/seaweedfs/seaweedfs-csi-driver.git git clone https://github.com/s3/s3-csi-driver.git
``` ```
2. Adjust your SeaweedFS Filer address via variable SEAWEEDFS_FILER in `deploy/kubernetes/seaweedfs-csi.yaml` (2 places) 2. Adjust your S3 Filer address via variable S3_FILER in `deploy/kubernetes/s3-csi.yaml` (2 places)
3. Apply the container storage interface for SeaweedFS for your cluster. Use the '-pre-1.17' version for any cluster pre kubernetes version 1.17. 3. Apply the container storage interface for S3 for your cluster. Use the '-pre-1.17' version for any cluster pre kubernetes version 1.17.
### To generate an up to date manifest from the helm chart, do: ### To generate an up to date manifest from the helm chart, do:
``` ```
$ helm template seaweedfs ./deploy/helm/seaweedfs-csi-driver > deploy/kubernetes/seaweedfs-csi.yaml $ helm template s3 ./deploy/helm/s3-csi-driver > deploy/kubernetes/s3-csi.yaml
``` ```
Check the kubelet root directory ,Execute the following command Check the kubelet root directory ,Execute the following command
``` ```
@@ -61,11 +61,11 @@ ps -ef | grep kubelet | grep root-dir
``` ```
If the result returned from the previous check command is not empty, the root directory (eg:--root-dir=/data/k8s/kubelet/data) representing the kubelet is not the default value (/var/lib/kubelet), so you need to update the kubelet root-dir in the CSI driven deployment file and deploy: If the result returned from the previous check command is not empty, the root directory (eg:--root-dir=/data/k8s/kubelet/data) representing the kubelet is not the default value (/var/lib/kubelet), so you need to update the kubelet root-dir in the CSI driven deployment file and deploy:
``` ```
sed 's+/var/lib/kubelet+/data/k8s/kubelet/data+g' deploy/kubernetes/seaweedfs-csi.yaml | kubectl apply -f - sed 's+/var/lib/kubelet+/data/k8s/kubelet/data+g' deploy/kubernetes/s3-csi.yaml | kubectl apply -f -
``` ```
If the result returned by the previous check command is null, you can directly deploy it without modifying the configuration: If the result returned by the previous check command is null, you can directly deploy it without modifying the configuration:
``` ```
$ kubectl apply -f deploy/kubernetes/seaweedfs-csi.yaml $ kubectl apply -f deploy/kubernetes/s3-csi.yaml
``` ```
4. Ensure all the containers are ready and running 4. Ensure all the containers are ready and running
``` ```
@@ -73,14 +73,14 @@ $ kubectl get po -n kube-system
``` ```
### TLS Support ### TLS Support
The provided static manifest `deploy/kubernetes/seaweedfs-csi.yaml` does not include TLS configuration by default. To enable TLS, it is recommended to use [Helm](#kubernetes-helm) with `tlsSecret` configured. If you must use static manifests, you will need to manually patch the `seaweedfs-mount` and `seaweedfs-node` DaemonSets to include the necessary TLS environment variables and volume mounts. The provided static manifest `deploy/kubernetes/s3-csi.yaml` does not include TLS configuration by default. To enable TLS, it is recommended to use [Helm](#kubernetes-helm) with `tlsSecret` configured. If you must use static manifests, you will need to manually patch the `s3-mount` and `s3-node` DaemonSets to include the necessary TLS environment variables and volume mounts.
### Uninstall ### Uninstall
``` ```
$ kubectl delete -f deploy/kubernetes/sample-busybox-pod.yaml $ kubectl delete -f deploy/kubernetes/sample-busybox-pod.yaml
$ kubectl delete -f deploy/kubernetes/sample-seaweedfs-pvc.yaml $ kubectl delete -f deploy/kubernetes/sample-s3-pvc.yaml
$ kubectl delete -f deploy/kubernetes/seaweedfs-csi.yaml $ kubectl delete -f deploy/kubernetes/s3-csi.yaml
``` ```
## Kubernetes (helm) ## Kubernetes (helm)
@@ -89,43 +89,43 @@ $ kubectl delete -f deploy/kubernetes/seaweedfs-csi.yaml
1. Clone project 1. Clone project
```bash ```bash
git clone https://github.com/seaweedfs/seaweedfs-csi-driver.git git clone https://github.com/s3/s3-csi-driver.git
``` ```
2. Edit `./deploy/helm/seaweedfs-csi-driver/values.yaml` if required and Install 2. Edit `./deploy/helm/s3-csi-driver/values.yaml` if required and Install
```bash ```bash
helm install --set seaweedfsFiler=<filerHost:port> seaweedfs-csi-driver ./deploy/helm/seaweedfs-csi-driver helm install --set s3Filer=<filerHost:port> s3-csi-driver ./deploy/helm/s3-csi-driver
``` ```
Example with multiple filers : Example with multiple filers :
```bash ```bash
helm install seaweedfs-csi-driver ./deploy/helm/seaweedfs-csi-driver/ \ helm install s3-csi-driver ./deploy/helm/s3-csi-driver/ \
--namespace seaweedfs-csi-driver \ --namespace s3-csi-driver \
--set seaweedfsFiler="<filerHost:port>\,<filerHost:port>\,<filerHost:port>\,<filerHost:port>\,<filerHost:port>" --set s3Filer="<filerHost:port>\,<filerHost:port>\,<filerHost:port>\,<filerHost:port>\,<filerHost:port>"
``` ```
### Uninstall ### Uninstall
```bash ```bash
helm uninstall seaweedfs-csi-driver helm uninstall s3-csi-driver
``` ```
# Update (Safe rollout) # Update (Safe rollout)
Updating seaweed-csi-driver DaemonSet (DS) will break processeses who implement fuse mount: Updating s3-csi-driver DaemonSet (DS) will break processeses who implement fuse mount:
newly created pods will not remount net device. newly created pods will not remount net device.
For safe update set `node.updateStrategy.type: OnDelete` for manual update. Steps: For safe update set `node.updateStrategy.type: OnDelete` for manual update. Steps:
1. delete DS pods on the node where there is no seaweedfs PV 1. delete DS pods on the node where there is no s3 PV
2. cordon or taint node 2. cordon or taint node
3. evict or delete pods with seaweedfs PV 3. evict or delete pods with s3 PV
4. delete DS pod on node 4. delete DS pod on node
5. uncordon or remove taint on node 5. uncordon or remove taint on node
6. repeat all steps on [all nodes] 6. repeat all steps on [all nodes]
# Testing # Testing
1. Create a persistent volume claim for 5GiB with name `seaweedfs-csi-pvc` and storage class `seaweedfs-storage`. The requested size is applied as a quota to the SeaweedFS collection used by the mount. 1. Create a persistent volume claim for 5GiB with name `s3-csi-pvc` and storage class `s3-storage`. The requested size is applied as a quota to the S3 collection used by the mount.
``` ```
$ kubectl apply -f deploy/kubernetes/sample-seaweedfs-pvc.yaml $ kubectl apply -f deploy/kubernetes/sample-s3-pvc.yaml
``` ```
2. Verify if the persistant volume claim exists and wait until its the STATUS is `Bound` 2. Verify if the persistant volume claim exists and wait until its the STATUS is `Bound`
``` ```
@@ -143,7 +143,7 @@ $ kubectl exec my-csi-app -- df -h
# Capacity limits # Capacity limits
For dynamically provisioned volumes, `resources.requests.storage` is enforced by For dynamically provisioned volumes, `resources.requests.storage` is enforced by
the SeaweedFS FUSE mount and reported by `df`. Once the mount observes that the the S3 FUSE mount and reported by `df`. Once the mount observes that the
quota has been reached, further writes fail with `ENOSPC`. PVC expansion updates quota has been reached, further writes fail with `ENOSPC`. PVC expansion updates
the mount quota. the mount quota.
@@ -151,7 +151,7 @@ The quota is collection-based and enforced by each CSI-managed mount, rather
than stored as an authoritative server-side limit. By default, every dynamic than stored as an authoritative server-side limit. By default, every dynamic
volume gets its own collection, so its quota is isolated. If multiple volumes volume gets its own collection, so its quota is isolated. If multiple volumes
are configured to use the same `collection`, their usage is shared and they do are configured to use the same `collection`, their usage is shared and they do
not have independent per-directory quotas. Writes through another SeaweedFS not have independent per-directory quotas. Writes through another S3
client that does not use the quota-configured mount are not restricted by this client that does not use the quota-configured mount are not restricted by this
CSI quota. CSI quota.
@@ -174,8 +174,8 @@ It can be done via creating separate storage class with options:
kind: StorageClass kind: StorageClass
apiVersion: storage.k8s.io/v1 apiVersion: storage.k8s.io/v1
metadata: metadata:
name: seaweedfs-special name: s3-special
provisioner: seaweedfs-csi-driver provisioner: s3-csi-driver
parameters: parameters:
collection: mycollection collection: mycollection
replication: "011" replication: "011"
@@ -189,14 +189,14 @@ In this case we do not need additional StorageClass. We need to create Persisten
apiVersion: v1 apiVersion: v1
kind: PersistentVolume kind: PersistentVolume
metadata: metadata:
name: seaweedfs-static name: s3-static
spec: spec:
accessModes: accessModes:
- ReadWriteMany - ReadWriteMany
capacity: capacity:
storage: 1Gi storage: 1Gi
csi: csi:
driver: seaweedfs-csi-driver driver: s3-csi-driver
volumeHandle: dfs-test volumeHandle: dfs-test
volumeAttributes: volumeAttributes:
collection: default collection: default
@@ -214,10 +214,10 @@ and bind PersistentVolumeClaim(s) to it:
apiVersion: v1 apiVersion: v1
kind: PersistentVolumeClaim kind: PersistentVolumeClaim
metadata: metadata:
name: seaweedfs-static name: s3-static
spec: spec:
storageClassName: "" storageClassName: ""
volumeName: seaweedfs-static volumeName: s3-static
accessModes: accessModes:
- ReadWriteMany - ReadWriteMany
resources: resources:
@@ -238,7 +238,7 @@ Change the type of locality
Level | Location Level | Location
------------------- | -------- ------------------- | --------
Driver | Helm: `values.yaml` -> `dataLocality` <br> Or `DaemonSet` -> Container `csi-seaweedfs-plugin` -> args `--dataLocality=` Driver | Helm: `values.yaml` -> `dataLocality` <br> Or `DaemonSet` -> Container `csi-s3-plugin` -> args `--dataLocality=`
StorageClass | `parameter.dataLocality` StorageClass | `parameter.dataLocality`
PersistentVolume | `spec.csi.volumeAttributes.dataLocality` PersistentVolume | `spec.csi.volumeAttributes.dataLocality`
cmd/s3-csi-driver/Dockerfile.dev
+6 -6
View File
@@ -2,18 +2,18 @@ FROM golang:1.24-alpine as builder
RUN apk add git g++ RUN apk add git g++
RUN mkdir -p /go/src/github.com/seaweedfs/ RUN mkdir -p /go/src/github.com/s3/
RUN git clone https://github.com/seaweedfs/seaweedfs /go/src/github.com/seaweedfs/seaweedfs RUN git clone https://github.com/s3/s3 /go/src/github.com/s3/s3
RUN cd /go/src/github.com/seaweedfs/seaweedfs/weed && go install RUN cd /go/src/github.com/s3/s3/s3 && go install
WORKDIR /go/src/github.com/seaweedfs/s3-csi-driver WORKDIR /go/src/github.com/s3/s3-csi-driver
COPY . . COPY . .
RUN cd /go/src/github.com/seaweedfs/s3-csi-driver && go build -ldflags="-s -w" -o /s3-csi-driver ./cmd/s3-csi-driver/main.go RUN cd /go/src/github.com/s3/s3-csi-driver && go build -ldflags="-s -w" -o /s3-csi-driver ./cmd/s3-csi-driver/main.go
FROM alpine AS final FROM alpine AS final
RUN apk add fuse RUN apk add fuse
LABEL org.opencontainers.image.source="https://github.com/hanzoai/s3-csi" LABEL org.opencontainers.image.source="https://github.com/hanzoai/s3-csi"
COPY --from=builder /go/bin/weed /usr/bin/ COPY --from=builder /go/bin/s3 /usr/bin/
COPY --from=builder /s3-csi-driver / COPY --from=builder /s3-csi-driver /
RUN chmod +x /s3-csi-driver RUN chmod +x /s3-csi-driver
cmd/s3-csi-driver/swarm/README.md
+13 -13
View File
@@ -1,6 +1,6 @@
# SeaweedFS CSI-Driver Docker Plugin # S3 CSI-Driver Docker Plugin
This Docker plugin integrates the SeaweedFS CSI-Driver with Docker. It allows you to use SeaweedFS as a volume driver in Docker environments. This Docker plugin integrates the S3 CSI-Driver with Docker. It allows you to use S3 as a volume driver in Docker environments.
## Environment Variables ## Environment Variables
@@ -24,14 +24,14 @@ This Docker plugin integrates the SeaweedFS CSI-Driver with Docker. It allows yo
## Usage ## Usage
```bash ```bash
docker plugin install --disable --alias seaweedfs-csi:swarm --grant-all-permissions gradlon/swarm-csi-swaweedfs:v1.2.0 docker plugin install --disable --alias s3-csi:swarm --grant-all-permissions gradlon/swarm-csi-swas3fs:v1.2.0
docker plugin set seaweedfs-csi:swarm FILER=<IP>:8888,<IP>:8888 docker plugin set s3-csi:swarm FILER=<IP>:8888,<IP>:8888
docker plugin set seaweedfs-csi:swarm CACHE_SIZE=512 docker plugin set s3-csi:swarm CACHE_SIZE=512
docker plugin enable seaweedfs-csi:swarm docker plugin enable s3-csi:swarm
docker volume create --driver seaweedfs-csi:swarm --availability active --scope single --sharing none --type mount --opt path="/docker/volumes/teste1" test-volume docker volume create --driver s3-csi:swarm --availability active --scope single --sharing none --type mount --opt path="/docker/volumes/teste1" test-volume
docker volume create \ docker volume create \
--driver seaweedfs-csi:swarm \ --driver s3-csi:swarm \
--availability active \ --availability active \
--scope multi \ --scope multi \
--sharing all \ --sharing all \
@@ -43,7 +43,7 @@ docker volume create \
## Build Guide ## Build Guide
Follow these steps to build the SeaweedFS CSI-Driver Docker plugin: Follow these steps to build the S3 CSI-Driver Docker plugin:
```bash ```bash
#!/bin/bash #!/bin/bash
@@ -56,11 +56,11 @@ fi
VERSION=${2:-latest} VERSION=${2:-latest}
ARCH=${5:-linux/amd64} ARCH=${5:-linux/amd64}
PLUGIN_NAME=${4:-swarm-csi-swaweedfs} PLUGIN_NAME=${4:-swarm-csi-swas3fs}
PLUGIN_TAG=${3:-v1.2.0} PLUGIN_TAG=${3:-v1.2.0}
PREFIX=${1:-gradlon} PREFIX=${1:-gradlon}
docker build --platform ${ARCH} --build-arg BASE_IMAGE=chrislusf/seaweedfs-csi-driver:${VERSION} --build-arg ARCH=$ARCH -t seawadd-csi_tmp_img . docker build --platform ${ARCH} --build-arg BASE_IMAGE=chrislusf/s3-csi-driver:${VERSION} --build-arg ARCH=$ARCH -t seawadd-csi_tmp_img .
mkdir -p ./plugin/rootfs mkdir -p ./plugin/rootfs
cp config.json ./plugin/ cp config.json ./plugin/
docker container create --name seawadd-csi_tmp seawadd-csi_tmp_img docker container create --name seawadd-csi_tmp seawadd-csi_tmp_img
@@ -68,11 +68,11 @@ docker container export seawadd-csi_tmp | tar -x -C ./plugin/rootfs
docker container rm -vf seawadd-csi_tmp docker container rm -vf seawadd-csi_tmp
docker image rm seawadd-csi_tmp_img docker image rm seawadd-csi_tmp_img
docker plugin disable ${PREFIX}/swarm-csi-swaweedfs:v1.2.0 docker plugin disable ${PREFIX}/swarm-csi-swas3fs:v1.2.0
docker plugin rm ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} 2> /dev/null || true docker plugin rm ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} 2> /dev/null || true
docker plugin create ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} ./plugin docker plugin create ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} ./plugin
docker plugin push ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} docker plugin push ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG}
rm -rf ./plugin/ rm -rf ./plugin/
``` ```
Volumes are store in the "/buckets" folder on the Seaweed server. Volumes are store in the "/buckets" folder on the S3 server.
cmd/s3-csi-driver/swarm/build.sh
+2 -2
View File
@@ -8,11 +8,11 @@ fi
VERSION=${2:-latest} VERSION=${2:-latest}
ARCH=${5:-linux/amd64} ARCH=${5:-linux/amd64}
PLUGIN_NAME=${4:-swarm-csi-swaweedfs} PLUGIN_NAME=${4:-swarm-csi-swas3fs}
PLUGIN_TAG=${3:-v1.2.0} PLUGIN_TAG=${3:-v1.2.0}
PREFIX=${1:-gradlon} PREFIX=${1:-gradlon}
docker build --platform ${ARCH} --build-arg BASE_IMAGE=chrislusf/seaweedfs-csi-driver:${VERSION} --build-arg ARCH=$ARCH -t seawadd-csi_tmp_img . docker build --platform ${ARCH} --build-arg BASE_IMAGE=chrislusf/s3-csi-driver:${VERSION} --build-arg ARCH=$ARCH -t seawadd-csi_tmp_img .
mkdir -p ./plugin/rootfs mkdir -p ./plugin/rootfs
cp config.json ./plugin/ cp config.json ./plugin/
docker container create --name seawadd-csi_tmp seawadd-csi_tmp_img docker container create --name seawadd-csi_tmp seawadd-csi_tmp_img
cmd/s3-csi-driver/swarm/config.json
+3 -3
View File
@@ -1,7 +1,7 @@
{ {
"description": "SeaweedFS csi-driver plugin for Docker", "description": "S3 csi-driver plugin for Docker",
"documentation": "https://github.com/seaweedfs/seaweedfs-csi-driver", "documentation": "https://github.com/s3/s3-csi-driver",
"entrypoint": ["/entrypoint.sh"], "entrypoint": ["/entrypoint.sh"],
"env": [ "env": [
{ {
@@ -68,7 +68,7 @@
} }
], ],
"interface": { "interface": {
"socket": "seaweed.sock", "socket": "s3.sock",
"types": ["docker.csicontroller/1.0", "docker.csinode/1.0"] "types": ["docker.csicontroller/1.0", "docker.csinode/1.0"]
}, },
"linux": { "linux": {
cmd/s3-csi-driver/swarm/entrypoint.sh
+1 -1
View File
@@ -7,6 +7,6 @@ fi
NODE_ID=$(cat /node_hostname) NODE_ID=$(cat /node_hostname)
C_WRITER=${C_WRITER:-32} C_WRITER=${C_WRITER:-32}
CMD="/seaweedfs-csi-driver --filer=$FILER --nodeid=${NODE_ID} --endpoint=unix://run/docker/plugins/seaweed.sock --concurrentWriters=${C_WRITER} --dataCenter=${DATACENTER} --dataLocality=none --logtostderr --map.uid=${UID_MAP} --map.gid=${GID_MAP} --cacheCapacityMB=${CACHE_SIZE} --cacheDir=/tmp/seaweedFS/docker-csi" CMD="/s3-csi-driver --filer=$FILER --nodeid=${NODE_ID} --endpoint=unix://run/docker/plugins/s3.sock --concurrentWriters=${C_WRITER} --dataCenter=${DATACENTER} --dataLocality=none --logtostderr --map.uid=${UID_MAP} --map.gid=${GID_MAP} --cacheCapacityMB=${CACHE_SIZE} --cacheDir=/tmp/s3/docker-csi"
exec $CMD exec $CMD
cmd/s3-mount/Dockerfile
+1 -1
View File
@@ -24,7 +24,7 @@ RUN go build -ldflags="-s -w" -o /s3-mount ./cmd/s3-mount/main.go && go clean -c
FROM alpine AS final FROM alpine AS final
RUN apk add fuse RUN apk add fuse
LABEL org.opencontainers.image.source="https://github.com/hanzoai/s3-csi" LABEL org.opencontainers.image.source="https://github.com/hanzoai/s3-csi"
COPY --from=builder /go/bin/weed /usr/bin/ COPY --from=builder /go/bin/s3 /usr/bin/
COPY --from=builder /s3-mount / COPY --from=builder /s3-mount /
RUN chmod +x /s3-mount RUN chmod +x /s3-mount
cmd/s3-mount/Dockerfile.dev
+11 -11
View File
@@ -1,28 +1,28 @@
# SEAWEEDFS_COMMIT should match the commit in go.mod for reproducibility # S3_COMMIT should match the commit in go.mod for reproducibility
# Update via: docker build --build-arg SEAWEEDFS_COMMIT=<hash> ... # Update via: docker build --build-arg S3_COMMIT=<hash> ...
ARG SEAWEEDFS_COMMIT=19ce7c0b6fdb ARG S3_COMMIT=19ce7c0b6fdb
FROM golang:1.25-alpine AS builder FROM golang:1.25-alpine AS builder
RUN apk add git g++ RUN apk add git g++
# Build weed binary from a specific commit for reproducibility # Build s3 binary from a specific commit for reproducibility
ARG SEAWEEDFS_COMMIT ARG S3_COMMIT
RUN git clone https://github.com/seaweedfs/seaweedfs /go/src/github.com/seaweedfs/seaweedfs \ RUN git clone https://github.com/s3/s3 /go/src/github.com/s3/s3 \
&& cd /go/src/github.com/seaweedfs/seaweedfs \ && cd /go/src/github.com/s3/s3 \
&& git checkout ${SEAWEEDFS_COMMIT} \ && git checkout ${S3_COMMIT} \
&& cd weed \ && cd s3 \
&& go install && go install
# Build s3-mount from current context # Build s3-mount from current context
WORKDIR /go/src/github.com/seaweedfs/s3-csi-driver WORKDIR /go/src/github.com/s3/s3-csi-driver
COPY . . COPY . .
RUN go build -ldflags="-s -w" -o /s3-mount ./cmd/s3-mount/main.go RUN go build -ldflags="-s -w" -o /s3-mount ./cmd/s3-mount/main.go
FROM alpine AS final FROM alpine AS final
RUN apk add fuse RUN apk add fuse
COPY --from=builder /go/bin/weed /usr/bin/ COPY --from=builder /go/bin/s3 /usr/bin/
COPY --from=builder /s3-mount / COPY --from=builder /s3-mount /
RUN chmod +x /s3-mount RUN chmod +x /s3-mount
cmd/s3-mount/main.go
+2 -2
View File
@@ -18,7 +18,7 @@ import (
var ( var (
endpoint = flag.String("endpoint", "unix:///tmp/s3-mount.sock", "endpoint the mount service listens on") endpoint = flag.String("endpoint", "unix:///tmp/s3-mount.sock", "endpoint the mount service listens on")
weedBinary = flag.String("weedBinary", mountmanager.DefaultWeedBinary, "path to the weed binary") s3Binary = flag.String("s3Binary", mountmanager.DefaultS3Binary, "path to the s3 binary")
) )
func main() { func main() {
@@ -45,7 +45,7 @@ func main() {
_ = os.Remove(address) _ = os.Remove(address)
}() }()
manager := mountmanager.NewManager(mountmanager.Config{WeedBinary: *weedBinary}) manager := mountmanager.NewManager(mountmanager.Config{S3Binary: *s3Binary})
mux := http.NewServeMux() mux := http.NewServeMux()
mux.HandleFunc("/mount", makePostHandler(manager.Mount)) mux.HandleFunc("/mount", makePostHandler(manager.Mount))
deploy/helm/s3-csi/README.md
+11 -11
View File
@@ -1,39 +1,39 @@
# Container Storage Interface (CSI) for SeaweedFS # Container Storage Interface (CSI) for S3
[Container storage interface](https://kubernetes-csi.github.io/docs/) is an [industry standard](https://github.com/container-storage-interface/spec/blob/master/spec.md) that enables storage vendors to develop a plugin once and have it work across a number of container orchestration systems. [Container storage interface](https://kubernetes-csi.github.io/docs/) is an [industry standard](https://github.com/container-storage-interface/spec/blob/master/spec.md) that enables storage vendors to develop a plugin once and have it work across a number of container orchestration systems.
[SeaweedFS](https://github.com/seaweedfs/seaweedfs) is a simple and highly scalable distributed file system, to store and serve billions of files fast! [S3](https://github.com/s3/s3) is a simple and highly scalable distributed file system, to store and serve billions of files fast!
See [seaweedfs-csi-driver](https://github.com/seaweedfs/seaweedfs-csi-driver) for the source for this CSI plugin. See [s3-csi-driver](https://github.com/s3/s3-csi-driver) for the source for this CSI plugin.
## Installing ## Installing
Add the Helm repository: Add the Helm repository:
```bash ```bash
helm repo add seaweedfs-csi-driver https://seaweedfs.github.io/seaweedfs-csi-driver/helm helm repo add s3-csi-driver https://s3.github.io/s3-csi-driver/helm
helm repo update helm repo update
``` ```
Install the chart. You will need to specify the location of the SeaweedFS filer URL by either running: Install the chart. You will need to specify the location of the S3 filer URL by either running:
```bash ```bash
helm install --set seaweedfsFiler=<filerHost:port> my-seaweedfs-csi-driver seaweedfs-csi-driver/seaweedfs-csi-driver helm install --set s3Filer=<filerHost:port> my-s3-csi-driver s3-csi-driver/s3-csi-driver
``` ```
Or by configuring a seaweedfs-overrides.yaml file containing (for example): Or by configuring a s3-overrides.yaml file containing (for example):
```yaml ```yaml
# For a SeaweedFS instance running locally under the "seaweed" namespace - adjust for your configuration # For a S3 instance running locally under the "s3" namespace - adjust for your configuration
seaweedfsFiler: "seaweedfs-filer.seaweed.svc.cluster.local:8888" s3Filer: "s3-filer.s3.svc.cluster.local:8888"
``` ```
And running: And running:
```bash ```bash
helm install my-seaweedfs-csi-driver seaweedfs-csi-driver/seaweedfs-csi-driver -f seaweedfs-overrides.yaml helm install my-s3-csi-driver s3-csi-driver/s3-csi-driver -f s3-overrides.yaml
``` ```
## Usage ## Usage
See [Testing](https://github.com/seaweedfs/seaweedfs-csi-driver#testing) on some usage examples. See [Testing](https://github.com/s3/s3-csi-driver#testing) on some usage examples.
deploy/helm/s3-csi/templates/_helpers.tpl
+10 -10
View File
@@ -1,11 +1,11 @@
{{- define "seaweedfs-csi-driver.name" -}} {{- define "s3-csi-driver.name" -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}} {{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- end -}} {{- end -}}
{{/* {{/*
Whether a valid security.toml source is configured. Whether a valid security.toml source is configured.
*/}} */}}
{{- define "seaweedfs-csi-driver.security-enabled" -}} {{- define "s3-csi-driver.security-enabled" -}}
{{- if .Values.security.enabled -}} {{- if .Values.security.enabled -}}
{{- if not (or .Values.security.existingSecret .Values.security.existingConfigMap (and .Values.security.create .Values.security.content)) -}} {{- if not (or .Values.security.existingSecret .Values.security.existingConfigMap (and .Values.security.create .Values.security.content)) -}}
{{- fail "security.enabled is true but no source configured. Set one of: security.existingSecret, security.existingConfigMap, or security.create with security.content." -}} {{- fail "security.enabled is true but no source configured. Set one of: security.existingSecret, security.existingConfigMap, or security.create with security.content." -}}
@@ -18,8 +18,8 @@ true
Security volume definition. Security volume definition.
Produces a volume entry sourcing security.toml from the appropriate Secret or ConfigMap. Produces a volume entry sourcing security.toml from the appropriate Secret or ConfigMap.
*/}} */}}
{{- define "seaweedfs-csi-driver.security-volume" -}} {{- define "s3-csi-driver.security-volume" -}}
{{- if include "seaweedfs-csi-driver.security-enabled" . -}} {{- if include "s3-csi-driver.security-enabled" . -}}
- name: security-config - name: security-config
{{- if .Values.security.existingSecret }} {{- if .Values.security.existingSecret }}
secret: secret:
@@ -29,10 +29,10 @@ Produces a volume entry sourcing security.toml from the appropriate Secret or Co
name: {{ .Values.security.existingConfigMap }} name: {{ .Values.security.existingConfigMap }}
{{- else if eq .Values.security.type "secret" }} {{- else if eq .Values.security.type "secret" }}
secret: secret:
secretName: {{ template "seaweedfs-csi-driver.name" . }}-security-config secretName: {{ template "s3-csi-driver.name" . }}-security-config
{{- else if eq .Values.security.type "configmap" }} {{- else if eq .Values.security.type "configmap" }}
configMap: configMap:
name: {{ template "seaweedfs-csi-driver.name" . }}-security-config name: {{ template "s3-csi-driver.name" . }}-security-config
{{- else }} {{- else }}
{{- fail (printf "security.type must be \"secret\" or \"configmap\", got: %s" .Values.security.type) -}} {{- fail (printf "security.type must be \"secret\" or \"configmap\", got: %s" .Values.security.type) -}}
{{- end }} {{- end }}
@@ -41,12 +41,12 @@ Produces a volume entry sourcing security.toml from the appropriate Secret or Co
{{/* {{/*
Security volumeMount definition. Security volumeMount definition.
Mounts security.toml to /etc/seaweedfs/security.toml. Mounts security.toml to /etc/s3/security.toml.
*/}} */}}
{{- define "seaweedfs-csi-driver.security-volumemount" -}} {{- define "s3-csi-driver.security-volumemount" -}}
{{- if include "seaweedfs-csi-driver.security-enabled" . -}} {{- if include "s3-csi-driver.security-enabled" . -}}
- name: security-config - name: security-config
mountPath: /etc/seaweedfs/security.toml mountPath: /etc/s3/security.toml
subPath: security.toml subPath: security.toml
readOnly: true readOnly: true
{{- end }} {{- end }}
deploy/helm/s3-csi/templates/configmap-security.yaml
+2 -2
View File
@@ -44,7 +44,7 @@ data:
ca = "/usr/local/share/ca-certificates/ca.crt" ca = "/usr/local/share/ca-certificates/ca.crt"
# use this for any place needs a grpc client # use this for any place needs a grpc client
# i.e., "weed backup|benchmark|filer.copy|filer.replicate|mount|s3|upload" # i.e., "s3 backup|benchmark|filer.copy|filer.replicate|mount|s3|upload"
[grpc.client] [grpc.client]
cert = "/usr/local/share/ca-certificates/tls.crt" cert = "/usr/local/share/ca-certificates/tls.crt"
key = "/usr/local/share/ca-certificates/tls.key" key = "/usr/local/share/ca-certificates/tls.key"
@@ -52,7 +52,7 @@ data:
# volume server https options # volume server https options
# Note: work in progress! # Note: work in progress!
# this does not work with other clients, e.g., "weed filer|mount" etc, yet. # this does not work with other clients, e.g., "s3 filer|mount" etc, yet.
#[https.client] #[https.client]
#enabled = false #enabled = false
#[https.volume] #[https.volume]
deploy/helm/s3-csi/templates/daemonset-mount.yaml
+3 -3
View File
@@ -40,11 +40,11 @@ spec:
- name: MOUNT_ENDPOINT - name: MOUNT_ENDPOINT
value: {{ $mountEndpoint | quote }} value: {{ $mountEndpoint | quote }}
{{- if .Values.tlsSecret }} {{- if .Values.tlsSecret }}
- name: WEED_GRPC_CLIENT_KEY - name: S3_GRPC_CLIENT_KEY
value: /var/run/secrets/app/tls/tls.key value: /var/run/secrets/app/tls/tls.key
- name: WEED_GRPC_CLIENT_CERT - name: S3_GRPC_CLIENT_CERT
value: /var/run/secrets/app/tls/tls.crt value: /var/run/secrets/app/tls/tls.crt
- name: WEED_GRPC_CA - name: S3_GRPC_CA
value: /var/run/secrets/app/tls/ca.crt value: /var/run/secrets/app/tls/ca.crt
{{- end }} {{- end }}
volumeMounts: volumeMounts:
deploy/helm/s3-csi/templates/daemonset.yaml
+6 -6
View File
@@ -47,7 +47,7 @@ spec:
imagePullPolicy: {{ .Values.imagePullPolicy }} imagePullPolicy: {{ .Values.imagePullPolicy }}
args: args:
- --endpoint=$(CSI_ENDPOINT) - --endpoint=$(CSI_ENDPOINT)
- --filer=$(SEAWEEDFS_FILER) - --filer=$(S3_FILER)
- --nodeid=$(NODE_ID) - --nodeid=$(NODE_ID)
- --driverName=$(DRIVER_NAME) - --driverName=$(DRIVER_NAME)
{{- if $mountEndpoint }} {{- if $mountEndpoint }}
@@ -76,7 +76,7 @@ spec:
env: env:
- name: CSI_ENDPOINT - name: CSI_ENDPOINT
value: unix:///csi/csi.sock value: unix:///csi/csi.sock
- name: SEAWEEDFS_FILER - name: S3_FILER
value: {{ .Values.s3Filer | quote }} value: {{ .Values.s3Filer | quote }}
- name: NODE_ID - name: NODE_ID
valueFrom: valueFrom:
@@ -96,15 +96,15 @@ spec:
fieldPath: metadata.labels['dataCenter'] fieldPath: metadata.labels['dataCenter']
{{- end }} {{- end }}
{{- if .Values.tlsSecret }} {{- if .Values.tlsSecret }}
- name: WEED_GRPC_CLIENT_KEY - name: S3_GRPC_CLIENT_KEY
value: /var/run/secrets/app/tls/tls.key value: /var/run/secrets/app/tls/tls.key
- name: WEED_GRPC_CLIENT_CERT - name: S3_GRPC_CLIENT_CERT
value: /var/run/secrets/app/tls/tls.crt value: /var/run/secrets/app/tls/tls.crt
- name: WEED_GRPC_CA - name: S3_GRPC_CA
value: /var/run/secrets/app/tls/ca.crt value: /var/run/secrets/app/tls/ca.crt
{{- end }} {{- end }}
{{- with .Values.logVerbosity }} {{- with .Values.logVerbosity }}
- name: WEED_V - name: S3_V
value: {{ . | quote }} value: {{ . | quote }}
{{- end }} {{- end }}
ports: ports:
deploy/helm/s3-csi/templates/deployment.yaml
+6 -6
View File
@@ -45,7 +45,7 @@ spec:
imagePullPolicy: {{ .Values.imagePullPolicy }} imagePullPolicy: {{ .Values.imagePullPolicy }}
args : args :
- --endpoint=$(CSI_ENDPOINT) - --endpoint=$(CSI_ENDPOINT)
- --filer=$(SEAWEEDFS_FILER) - --filer=$(S3_FILER)
- --driverName=$(DRIVER_NAME) - --driverName=$(DRIVER_NAME)
- --components=controller - --components=controller
@@ -53,21 +53,21 @@ spec:
env: env:
- name: CSI_ENDPOINT - name: CSI_ENDPOINT
value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock
- name: SEAWEEDFS_FILER - name: S3_FILER
value: {{ .Values.s3Filer | quote }} value: {{ .Values.s3Filer | quote }}
- name: DRIVER_NAME - name: DRIVER_NAME
value: {{ .Values.driverName | quote }} value: {{ .Values.driverName | quote }}
{{- if .Values.tlsSecret }} {{- if .Values.tlsSecret }}
- name: WEED_GRPC_CLIENT_KEY - name: S3_GRPC_CLIENT_KEY
value: /var/run/secrets/app/tls/tls.key value: /var/run/secrets/app/tls/tls.key
- name: WEED_GRPC_CLIENT_CERT - name: S3_GRPC_CLIENT_CERT
value: /var/run/secrets/app/tls/tls.crt value: /var/run/secrets/app/tls/tls.crt
- name: WEED_GRPC_CA - name: S3_GRPC_CA
value: /var/run/secrets/app/tls/ca.crt value: /var/run/secrets/app/tls/ca.crt
{{- end }} {{- end }}
{{- with .Values.logVerbosity }} {{- with .Values.logVerbosity }}
- name: WEED_V - name: S3_V
value: {{ . | quote }} value: {{ . | quote }}
{{- end }} {{- end }}
ports: ports:
deploy/helm/s3-csi/values.yaml
+3 -3
View File
@@ -1,5 +1,5 @@
# host and port of your S3 filer # host and port of your S3 filer
s3Filer: "SEAWEEDFS_FILER:8888" s3Filer: "S3_FILER:8888"
storageClassName: s3-storage storageClassName: s3-storage
storageClassVolumeBindingMode: Immediate storageClassVolumeBindingMode: Immediate
isDefaultStorageClass: false isDefaultStorageClass: false
@@ -164,8 +164,8 @@ node:
mountEndpoint: "" # defaults to mountService.endpoint mountEndpoint: "" # defaults to mountService.endpoint
mountSocketDir: "" # defaults to mountService.socketDir mountSocketDir: "" # defaults to mountService.socketDir
mountHostPath: "" # defaults to mountService.hostPath mountHostPath: "" # defaults to mountService.hostPath
# When s3-csi-node pod on node is recreated, all pods on same node using seaweed-csi PV will stop working. # When s3-csi-node pod on node is recreated, all pods on same node using s3-csi PV will stop working.
# For safe update set updateStrategy.type: OnDelete and manually move pods who use seaweed-csi PV, then delete s3-csi-node damonset pod # For safe update set updateStrategy.type: OnDelete and manually move pods who use s3-csi PV, then delete s3-csi-node damonset pod
updateStrategy: updateStrategy:
type: RollingUpdate type: RollingUpdate
rollingUpdate: rollingUpdate:
deploy/kubernetes/seaweedfs-csi-old.yaml → deploy/kubernetes/s3-csi-old.yaml
+21 -21
View File
@@ -302,28 +302,28 @@ spec:
image: chrislusf/s3-csi:latest image: chrislusf/s3-csi:latest
args : args :
- "--endpoint=$(CSI_ENDPOINT)" - "--endpoint=$(CSI_ENDPOINT)"
- "--filer=$(SEAWEEDFS_FILER)" - "--filer=$(S3_FILER)"
- "--nodeid=$(NODE_ID)" - "--nodeid=$(NODE_ID)"
- "--cacheCapacityMB=$(SEAWEEDFS_CACHE_CAPACITY_MB)" - "--cacheCapacityMB=$(S3_CACHE_CAPACITY_MB)"
- "--cacheDir=$(SEAWEEDFS_CACHE_DIR)" - "--cacheDir=$(S3_CACHE_DIR)"
- "--map.uid=$(SEAWEEDFS_MAP_UID)" - "--map.uid=$(S3_MAP_UID)"
- "--map.gid=$(SEAWEEDFS_MAP_GID)" - "--map.gid=$(S3_MAP_GID)"
env: env:
- name: CSI_ENDPOINT - name: CSI_ENDPOINT
value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock
- name: SEAWEEDFS_FILER - name: S3_FILER
value: "192.168.2.7:8888" # host and port of your S3 filer value: "192.168.2.7:8888" # host and port of your S3 filer
- name: NODE_ID - name: NODE_ID
valueFrom: valueFrom:
fieldRef: fieldRef:
fieldPath: spec.nodeName fieldPath: spec.nodeName
- name: SEAWEEDFS_CACHE_CAPACITY_MB - name: S3_CACHE_CAPACITY_MB
value: "1000" value: "1000"
- name: SEAWEEDFS_CACHE_DIR - name: S3_CACHE_DIR
value: /tmp value: /tmp
- name: SEAWEEDFS_MAP_UID - name: S3_MAP_UID
value: "" value: ""
- name: SEAWEEDFS_MAP_GID - name: S3_MAP_GID
value: "" value: ""
imagePullPolicy: "Always" imagePullPolicy: "Always"
volumeMounts: volumeMounts:
@@ -382,28 +382,28 @@ spec:
image: chrislusf/s3-csi:latest image: chrislusf/s3-csi:latest
args : args :
- "--endpoint=$(CSI_ENDPOINT)" - "--endpoint=$(CSI_ENDPOINT)"
- "--filer=$(SEAWEEDFS_FILER)" - "--filer=$(S3_FILER)"
- "--nodeid=$(NODE_ID)" - "--nodeid=$(NODE_ID)"
- "--cacheCapacityMB=$(SEAWEEDFS_CACHE_CAPACITY_MB)" - "--cacheCapacityMB=$(S3_CACHE_CAPACITY_MB)"
- "--cacheDir=$(SEAWEEDFS_CACHE_DIR)" - "--cacheDir=$(S3_CACHE_DIR)"
- "--map.uid=$(SEAWEEDFS_MAP_UID)" - "--map.uid=$(S3_MAP_UID)"
- "--map.gid=$(SEAWEEDFS_MAP_GID)" - "--map.gid=$(S3_MAP_GID)"
env: env:
- name: CSI_ENDPOINT - name: CSI_ENDPOINT
value: unix:///csi/csi.sock value: unix:///csi/csi.sock
- name: SEAWEEDFS_FILER - name: S3_FILER
value: "192.168.2.7:8888" # host and port of your S3 filer, could also be K8S DNS entry like s3-filer.seaweed-namespace.svc:8888 value: "192.168.2.7:8888" # host and port of your S3 filer, could also be K8S DNS entry like s3-filer.s3-namespace.svc:8888
- name: NODE_ID - name: NODE_ID
valueFrom: valueFrom:
fieldRef: fieldRef:
fieldPath: spec.nodeName fieldPath: spec.nodeName
- name: SEAWEEDFS_CACHE_CAPACITY_MB - name: S3_CACHE_CAPACITY_MB
value: "1000" value: "1000"
- name: SEAWEEDFS_CACHE_DIR - name: S3_CACHE_DIR
value: /tmp value: /tmp
- name: SEAWEEDFS_MAP_UID - name: S3_MAP_UID
value: "" value: ""
- name: SEAWEEDFS_MAP_GID - name: S3_MAP_GID
value: "" value: ""
imagePullPolicy: "IfNotPresent" imagePullPolicy: "IfNotPresent"
volumeMounts: volumeMounts:
deploy/kubernetes/seaweedfs-csi.yaml → deploy/kubernetes/s3-csi.yaml
+6 -6
View File
@@ -293,7 +293,7 @@ spec:
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
args: args:
- --endpoint=$(CSI_ENDPOINT) - --endpoint=$(CSI_ENDPOINT)
- --filer=$(SEAWEEDFS_FILER) - --filer=$(S3_FILER)
- --nodeid=$(NODE_ID) - --nodeid=$(NODE_ID)
- --driverName=$(DRIVER_NAME) - --driverName=$(DRIVER_NAME)
- --mountEndpoint=$(MOUNT_ENDPOINT) - --mountEndpoint=$(MOUNT_ENDPOINT)
@@ -302,8 +302,8 @@ spec:
env: env:
- name: CSI_ENDPOINT - name: CSI_ENDPOINT
value: unix:///csi/csi.sock value: unix:///csi/csi.sock
- name: SEAWEEDFS_FILER - name: S3_FILER
value: "SEAWEEDFS_FILER:8888" value: "S3_FILER:8888"
- name: NODE_ID - name: NODE_ID
valueFrom: valueFrom:
fieldRef: fieldRef:
@@ -454,7 +454,7 @@ spec:
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
args : args :
- --endpoint=$(CSI_ENDPOINT) - --endpoint=$(CSI_ENDPOINT)
- --filer=$(SEAWEEDFS_FILER) - --filer=$(S3_FILER)
- --driverName=$(DRIVER_NAME) - --driverName=$(DRIVER_NAME)
- --components=controller - --components=controller
@@ -462,8 +462,8 @@ spec:
env: env:
- name: CSI_ENDPOINT - name: CSI_ENDPOINT
value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock
- name: SEAWEEDFS_FILER - name: S3_FILER
value: "SEAWEEDFS_FILER:8888" value: "S3_FILER:8888"
- name: DRIVER_NAME - name: DRIVER_NAME
value: "s3-csi" value: "s3-csi"
deploy/kubernetes/sample-seaweedfs-pvc.yaml → deploy/kubernetes/sample-s3-pvc.yaml
View File
deploy/nomad/README.md
+17 -17
View File
@@ -1,9 +1,9 @@
# Example of using seaweedfs with HashiCorp Nomad # Example of using s3 with HashiCorp Nomad
## Running seaweedfs cluster ## Running s3 cluster
You can skip this part if you have already running seaweedfs. You can skip this part if you have already running s3.
Assumptions: Assumptions:
- Running Nomad cluster - Running Nomad cluster
@@ -14,24 +14,24 @@ Assumptions:
```shell ```shell
export NOMAD_ADDR=http://nomad.service.consul:4646 export NOMAD_ADDR=http://nomad.service.consul:4646
nomad run seaweedfs.hcl nomad run s3.hcl
``` ```
Seaweedfs master will be available on http://seaweedfs-master.service.consul:9333/ S3 master will be available on http://s3-master.service.consul:9333/
Seaweedfs filer will be available on http://seaweedfs-filer.service.consul:8888/ S3 filer will be available on http://s3-filer.service.consul:8888/
## Running CSI ## Running CSI
The CSI driver is split into two components that register under the same The CSI driver is split into two components that register under the same
`csi_plugin` id (`seaweedfs`): `csi_plugin` id (`s3`):
- **controller** (`seaweedfs-csi-controller.hcl`) — a single `service` job that - **controller** (`s3-csi-controller.hcl`) — a single `service` job that
implements the volume lifecycle RPCs. Nomad calls it for `nomad volume create` implements the volume lifecycle RPCs. Nomad calls it for `nomad volume create`
/ `nomad volume delete`. It only talks to the filer. / `nomad volume delete`. It only talks to the filer.
- **node** (`seaweedfs-csi.hcl`) — a `system` job that runs on every worker and - **node** (`s3-csi.hcl`) — a `system` job that runs on every worker and
stages/publishes volumes into allocations, using the `seaweedfs-mount` stages/publishes volumes into allocations, using the `s3-mount`
sidecar over a shared unix socket. sidecar over a shared unix socket.
You need **both** running. With only the node plugin, `nomad volume create` You need **both** running. With only the node plugin, `nomad volume create`
@@ -41,20 +41,20 @@ fails with `plugin has no controller`.
export NOMAD_ADDR=http://nomad.service.consul:4646 export NOMAD_ADDR=http://nomad.service.consul:4646
# Start CSI controller (one instance) and node (one per worker) # Start CSI controller (one instance) and node (one per worker)
nomad run seaweedfs-csi-controller.hcl nomad run s3-csi-controller.hcl
nomad run seaweedfs-csi.hcl nomad run s3-csi.hcl
# Wait until the plugin reports a healthy controller and the expected node count # Wait until the plugin reports a healthy controller and the expected node count
nomad plugin status seaweedfs nomad plugin status s3
# Create volume # Create volume
nomad volume create example-seaweedfs-volume.hcl nomad volume create example-s3-volume.hcl
# Start sample app # Start sample app
nomad run example-seaweedfs-app.hcl nomad run example-s3-app.hcl
``` ```
> If you only run the node plugin (no controller), you cannot `nomad volume > If you only run the node plugin (no controller), you cannot `nomad volume
> create`. Instead, pre-create the bucket/directory in SeaweedFS yourself and > create`. Instead, pre-create the bucket/directory in S3 yourself and
> `nomad volume register example-seaweedfs-volume.hcl` to register the existing > `nomad volume register example-s3-volume.hcl` to register the existing
> volume. > volume.
deploy/nomad/example-seaweedfs-app.hcl → deploy/nomad/example-s3-app.hcl
+4 -4
View File
@@ -1,10 +1,10 @@
job "example-seaweedfs-app" { job "example-s3-app" {
datacenters = ["dc1"] datacenters = ["dc1"]
group "apps" { group "apps" {
volume "example-seaweedfs-volume" { volume "example-s3-volume" {
type = "csi" type = "csi"
source = "example-seaweedfs-volume" source = "example-s3-volume"
access_mode = "multi-node-multi-writer" access_mode = "multi-node-multi-writer"
attachment_mode = "file-system" attachment_mode = "file-system"
} }
@@ -24,7 +24,7 @@ job "example-seaweedfs-app" {
} }
volume_mount { volume_mount {
volume = "example-seaweedfs-volume" volume = "example-s3-volume"
destination = "/mnt/pv" destination = "/mnt/pv"
} }
} }
deploy/nomad/example-seaweedfs-volume.hcl → deploy/nomad/example-s3-volume.hcl
+4 -4
View File
@@ -1,13 +1,13 @@
# id - Nomad internal ID. It is not sent to the CSI plugin but is used by Nomad for `per_alloc` # id - Nomad internal ID. It is not sent to the CSI plugin but is used by Nomad for `per_alloc`
# volume configurations, etc. # volume configurations, etc.
id = "example-seaweedfs-volume" id = "example-s3-volume"
# name - the name sent to the CSI plugin as an idempotency key and suggested volume ID. The CSI # name - the name sent to the CSI plugin as an idempotency key and suggested volume ID. The CSI
# spec requires the calling Container Orchestrator to respect the actual volumeId returned by the # spec requires the calling Container Orchestrator to respect the actual volumeId returned by the
# CSI plugin. Nomad does this, storing it as the volume's ExternalID and using it in subsequent # CSI plugin. Nomad does this, storing it as the volume's ExternalID and using it in subsequent
# calls to the controller and node. # calls to the controller and node.
name = "example-seaweedfs-volume" name = "example-s3-volume"
type = "csi" type = "csi"
plugin_id = "seaweedfs" plugin_id = "s3"
capacity_min = "256GiB" capacity_min = "256GiB"
capacity_max = "512GiB" capacity_max = "512GiB"
@@ -25,7 +25,7 @@ mount_options {
} }
parameters { parameters {
# Available options: https://github.com/seaweedfs/seaweedfs-csi-driver/blob/master/pkg/driver/mounter_seaweedfs.go # Available options: https://github.com/s3/s3-csi-driver/blob/master/pkg/driver/mounter_s3.go
# By default, collection is the `volume ID` returned from the Create Volume gRPC call. Nomad calls this the # By default, collection is the `volume ID` returned from the Create Volume gRPC call. Nomad calls this the
# External ID of the volume. "example" here overrides that. # External ID of the volume. "example" here overrides that.
collection = "example" collection = "example"
deploy/nomad/seaweedfs-csi-controller.hcl → deploy/nomad/s3-csi-controller.hcl
+7 -7
View File
@@ -6,13 +6,13 @@
# #
# The controller only talks to the filer (it creates/removes the bucket or # The controller only talks to the filer (it creates/removes the bucket or
# directory backing the volume), so unlike the node plugin it does NOT need the # directory backing the volume), so unlike the node plugin it does NOT need the
# seaweedfs-mount sidecar, host volumes, bidirectional mount propagation, or # s3-mount sidecar, host volumes, bidirectional mount propagation, or
# privileged mode. A single instance is enough; run it as a `service` job. # privileged mode. A single instance is enough; run it as a `service` job.
# #
# The csi_plugin.id ("seaweedfs") MUST match the id used by the node job # The csi_plugin.id ("s3") MUST match the id used by the node job
# (seaweedfs-csi.hcl) so Nomad treats them as one logical plugin with both a # (s3-csi.hcl) so Nomad treats them as one logical plugin with both a
# controller and nodes. # controller and nodes.
job "seaweedfs-csi-controller" { job "s3-csi-controller" {
datacenters = ["dc1"] datacenters = ["dc1"]
type = "service" type = "service"
priority = 90 priority = 90
@@ -24,18 +24,18 @@ job "seaweedfs-csi-controller" {
driver = "docker" driver = "docker"
config { config {
image = "chrislusf/seaweedfs-csi-driver:v1.3.9" image = "chrislusf/s3-csi-driver:v1.3.9"
force_pull = true force_pull = true
network_mode = "host" network_mode = "host"
args = [ args = [
"--endpoint=unix:///csi-sock/csi.sock", "--endpoint=unix:///csi-sock/csi.sock",
"--filer=seaweedfs-filer.service.consul:8888", "--filer=s3-filer.service.consul:8888",
"--components=controller", "--components=controller",
] ]
} }
csi_plugin { csi_plugin {
id = "seaweedfs" id = "s3"
type = "controller" type = "controller"
mount_dir = "/csi-sock" mount_dir = "/csi-sock"
} }
deploy/nomad/seaweedfs-csi.hcl → deploy/nomad/s3-csi.hcl
+18 -18
View File
@@ -1,4 +1,4 @@
job "seaweedfs-csi-node" { job "s3-csi-node" {
datacenters = ["dc1"] datacenters = ["dc1"]
type = "system" type = "system"
priority = 90 priority = 90
@@ -15,17 +15,17 @@ job "seaweedfs-csi-node" {
group "nodes" { group "nodes" {
# directory to share the seaweedfs-mount unix socket between tasks # directory to share the s3-mount unix socket between tasks
volume "seaweedfs_mount_socket" { volume "s3_mount_socket" {
type = "host" type = "host"
source = "seaweedfs_mount_socket" source = "s3_mount_socket"
read_only = false read_only = false
} }
# host directory where Nomad CSI stage/publish paths live # host directory where Nomad CSI stage/publish paths live
volume "nomad_csi_seaweedfs" { volume "nomad_csi_s3" {
type = "host" type = "host"
source = "nomad_csi_seaweedfs" source = "nomad_csi_s3"
read_only = false read_only = false
} }
@@ -39,25 +39,25 @@ job "seaweedfs-csi-node" {
driver = "docker" driver = "docker"
volume_mount { volume_mount {
volume = "seaweedfs_mount_socket" volume = "s3_mount_socket"
destination = "/var/lib/seaweedfs-mount" destination = "/var/lib/s3-mount"
read_only = false read_only = false
} }
# required for proper propagation of staged mounts into the host filesystem # required for proper propagation of staged mounts into the host filesystem
volume_mount { volume_mount {
volume = "nomad_csi_seaweedfs" volume = "nomad_csi_s3"
destination = "/csi-data" destination = "/csi-data"
read_only = false read_only = false
propagation_mode = "bidirectional" propagation_mode = "bidirectional"
} }
config { config {
image = "chrislusf/seaweedfs-mount:dev" image = "chrislusf/s3-mount:dev"
network_mode = "host" network_mode = "host"
privileged = true privileged = true
args = [ args = [
"--endpoint=unix:///var/lib/seaweedfs-mount/seaweedfs-mount.sock", "--endpoint=unix:///var/lib/s3-mount/s3-mount.sock",
] ]
} }
@@ -72,34 +72,34 @@ job "seaweedfs-csi-node" {
driver = "docker" driver = "docker"
volume_mount { volume_mount {
volume = "seaweedfs_mount_socket" volume = "s3_mount_socket"
destination = "/var/lib/seaweedfs-mount" destination = "/var/lib/s3-mount"
read_only = false read_only = false
} }
config { config {
image = "chrislusf/seaweedfs-csi-driver:v1.3.9" image = "chrislusf/s3-csi-driver:v1.3.9"
force_pull = true force_pull = true
network_mode = "host" network_mode = "host"
privileged = true privileged = true
args = [ args = [
"--endpoint=unix:///csi-sock/csi.sock", "--endpoint=unix:///csi-sock/csi.sock",
"--filer=seaweedfs-filer.service.consul:8888", "--filer=s3-filer.service.consul:8888",
"--dataCenter=${node.datacenter}", "--dataCenter=${node.datacenter}",
"--nodeid=${node.unique.name}", "--nodeid=${node.unique.name}",
"--cacheCapacityMB=1024", "--cacheCapacityMB=1024",
"--cacheDir=${NOMAD_ALLOC_DIR}/cache_dir", "--cacheDir=${NOMAD_ALLOC_DIR}/cache_dir",
"--mountEndpoint=unix:///var/lib/seaweedfs-mount/seaweedfs-mount.sock", "--mountEndpoint=unix:///var/lib/s3-mount/s3-mount.sock",
"--dataLocality=none", "--dataLocality=none",
"--components=node", "--components=node",
] ]
} }
csi_plugin { csi_plugin {
id = "seaweedfs" id = "s3"
type = "node" type = "node"
mount_dir = "/csi-sock" mount_dir = "/csi-sock"
stage_publish_base_dir = "/csi-data/node/seaweedfs" stage_publish_base_dir = "/csi-data/node/s3"
} }
resources { resources {
deploy/nomad/seaweedfs.hcl → deploy/nomad/s3.hcl
+27 -27
View File
@@ -1,11 +1,11 @@
# WWW naster: http://seaweedfs-master.service.consul:9333/ # WWW naster: http://s3-master.service.consul:9333/
# WWW filer: http://seaweedfs-filer.service.consul:8888/ # WWW filer: http://s3-filer.service.consul:8888/
job "seaweedfs" { job "s3" {
datacenters = ["dc1"] datacenters = ["dc1"]
type = "service" type = "service"
group "seaweedfs-master" { group "s3-master" {
count = 3 count = 3
constraint { constraint {
@@ -49,15 +49,15 @@ job "seaweedfs" {
} }
} }
task "seaweedfs-master" { task "s3-master" {
driver = "docker" driver = "docker"
env { env {
WEED_MASTER_VOLUME_GROWTH_COPY_1 = "1" S3_MASTER_VOLUME_GROWTH_COPY_1 = "1"
WEED_MASTER_VOLUME_GROWTH_COPY_2 = "2" S3_MASTER_VOLUME_GROWTH_COPY_2 = "2"
WEED_MASTER_VOLUME_GROWTH_COPY_OTHER = "1" S3_MASTER_VOLUME_GROWTH_COPY_OTHER = "1"
} }
config { config {
image = "chrislusf/seaweedfs:latest" image = "chrislusf/s3:latest"
force_pull = "true" force_pull = "true"
network_mode = "host" network_mode = "host"
args = [ args = [
@@ -78,7 +78,7 @@ job "seaweedfs" {
service { service {
tags = ["${node.unique.name}"] tags = ["${node.unique.name}"]
name = "seaweedfs-master" name = "s3-master"
port = "http" port = "http"
check { check {
type = "tcp" type = "tcp"
@@ -93,7 +93,7 @@ job "seaweedfs" {
group "seaweedfs-volume" { group "s3-volume" {
count = 3 count = 3
constraint { constraint {
@@ -133,12 +133,12 @@ job "seaweedfs" {
} }
} }
task "seaweedfs-volume" { task "s3-volume" {
driver = "docker" driver = "docker"
user = "1000:1000" user = "1000:1000"
config { config {
image = "chrislusf/seaweedfs:latest" image = "chrislusf/s3:latest"
force_pull = "true" force_pull = "true"
network_mode = "host" network_mode = "host"
args = [ args = [
@@ -146,7 +146,7 @@ job "seaweedfs" {
"-dataCenter=${NOMAD_DC}", "-dataCenter=${NOMAD_DC}",
# "-rack=${meta.rack}", # "-rack=${meta.rack}",
"-rack=${node.unique.name}", "-rack=${node.unique.name}",
"-mserver=seaweedfs-master.service.consul:9333", "-mserver=s3-master.service.consul:9333",
"-port=${NOMAD_PORT_http}", "-port=${NOMAD_PORT_http}",
"-ip=${NOMAD_IP_http}", "-ip=${NOMAD_IP_http}",
"-publicUrl=${NOMAD_ADDR_http}", "-publicUrl=${NOMAD_ADDR_http}",
@@ -157,7 +157,7 @@ job "seaweedfs" {
mounts = [ mounts = [
{ {
type = "bind" type = "bind"
source = "/data/seaweedfs-volume-data" # there should be directory in host VM source = "/data/s3-volume-data" # there should be directory in host VM
target = "/data" target = "/data"
readonly = false readonly = false
bind_options = { bind_options = {
@@ -175,7 +175,7 @@ job "seaweedfs" {
service { service {
tags = ["${node.unique.name}"] tags = ["${node.unique.name}"]
name = "seaweedfs-volume" name = "s3-volume"
port = "http" port = "http"
check { check {
type = "tcp" type = "tcp"
@@ -188,7 +188,7 @@ job "seaweedfs" {
} }
group "seaweedfs-filer" { group "s3-filer" {
count = 1 count = 1
constraint { constraint {
@@ -219,12 +219,12 @@ job "seaweedfs" {
} }
} }
task "seaweedfs-filer" { task "s3-filer" {
driver = "docker" driver = "docker"
user = "1000:1000" user = "1000:1000"
config { config {
image = "chrislusf/seaweedfs:latest" image = "chrislusf/s3:latest"
force_pull = "true" force_pull = "true"
network_mode = "host" network_mode = "host"
args = [ args = [
@@ -233,7 +233,7 @@ job "seaweedfs" {
# "-rack=${meta.rack}", # "-rack=${meta.rack}",
"-rack=${node.unique.name}", "-rack=${node.unique.name}",
"-defaultReplicaPlacement=000", "-defaultReplicaPlacement=000",
"-master=seaweedfs-master.service.consul:9333", "-master=s3-master.service.consul:9333",
"-s3", "-s3",
"-ip=${NOMAD_IP_http}", "-ip=${NOMAD_IP_http}",
"-port=${NOMAD_PORT_http}", "-port=${NOMAD_PORT_http}",
@@ -243,7 +243,7 @@ job "seaweedfs" {
{ {
type = "bind" type = "bind"
source = "local/filer.toml" source = "local/filer.toml"
target = "/etc/seaweedfs/filer.toml" target = "/etc/s3/filer.toml"
} }
] ]
@@ -266,9 +266,9 @@ createTable = """
""" """
hostname = "172.21.100.54" hostname = "172.21.100.54"
port = 5432 port = 5432
username = "seaweedfs" username = "s3"
password = "pass1234567" password = "pass1234567"
database = "seaweedfs" database = "s3"
schema = "" schema = ""
sslmode = "disable" sslmode = "disable"
connection_max_idle = 100 connection_max_idle = 100
@@ -278,8 +278,8 @@ enableUpsert = true
upsertQuery = """INSERT INTO "%[1]s" (dirhash,name,directory,meta) VALUES($1,$2,$3,$4) ON CONFLICT (dirhash,name) DO UPDATE SET meta = EXCLUDED.meta WHERE "%[1]s".meta != EXCLUDED.meta""" upsertQuery = """INSERT INTO "%[1]s" (dirhash,name,directory,meta) VALUES($1,$2,$3,$4) ON CONFLICT (dirhash,name) DO UPDATE SET meta = EXCLUDED.meta WHERE "%[1]s".meta != EXCLUDED.meta"""
# ssh ubuntu@172.21.100.54 # ssh ubuntu@172.21.100.54
# sudo -u postgres psql -c "CREATE ROLE seaweedfs WITH PASSWORD 'pass1234567';" # sudo -u postgres psql -c "CREATE ROLE s3 WITH PASSWORD 'pass1234567';"
# sudo -u postgres psql -c "CREATE DATABASE seaweedfs OWNER seaweedfs;" # sudo -u postgres psql -c "CREATE DATABASE s3 OWNER s3;"
EOH EOH
} }
@@ -290,7 +290,7 @@ EOH
service { service {
tags = ["${node.unique.name}"] tags = ["${node.unique.name}"]
name = "seaweedfs-filer" name = "s3-filer"
port = "http" port = "http"
check { check {
type = "tcp" type = "tcp"
@@ -302,7 +302,7 @@ EOH
service { service {
tags = ["${node.unique.name}"] tags = ["${node.unique.name}"]
name = "seaweedfs-s3" name = "s3-s3"
port = "s3" port = "s3"
check { check {
type = "tcp" type = "tcp"
pkg/filer/filer.go
+1 -1
View File
@@ -8,7 +8,7 @@
// There is no RegisterHanzoFilerServer anywhere in that tree. // There is no RegisterHanzoFilerServer anywhere in that tree.
// //
// So a gRPC client cannot talk to it at all — the failure mode is // So a gRPC client cannot talk to it at all — the failure mode is
// `Unimplemented: unknown service filer_pb.SeaweedFiler`, which reads like a // `Unimplemented: unknown service filer_pb.S3Filer`, which reads like a
// service-name mismatch but is really a gRPC client hitting a non-gRPC port. No // service-name mismatch but is really a gRPC client hitting a non-gRPC port. No
// amount of renaming or re-addressing fixes that; the transport has to change. // amount of renaming or re-addressing fixes that; the transport has to change.
// //
pkg/mountmanager/manager.go
+31 -31
View File
@@ -18,9 +18,9 @@ import (
var kubeMounter = mount.New("") var kubeMounter = mount.New("")
// Manager owns weed mount processes and exposes helpers to start and stop them. // Manager owns s3 mount processes and exposes helpers to start and stop them.
type Manager struct { type Manager struct {
weedBinary string s3Binary string
mu sync.Mutex mu sync.Mutex
mounts map[string]*mountEntry mounts map[string]*mountEntry
@@ -29,23 +29,23 @@ type Manager struct {
// Config configures a Manager instance. // Config configures a Manager instance.
type Config struct { type Config struct {
WeedBinary string S3Binary string
} }
// NewManager returns a Manager ready to accept mount requests. // NewManager returns a Manager ready to accept mount requests.
func NewManager(cfg Config) *Manager { func NewManager(cfg Config) *Manager {
binary := cfg.WeedBinary binary := cfg.S3Binary
if binary == "" { if binary == "" {
binary = DefaultWeedBinary binary = DefaultS3Binary
} }
return &Manager{ return &Manager{
weedBinary: binary, s3Binary: binary,
mounts: make(map[string]*mountEntry), mounts: make(map[string]*mountEntry),
locks: newKeyMutex(), locks: newKeyMutex(),
} }
} }
// Mount starts a weed mount process using the provided request. // Mount starts a s3 mount process using the provided request.
func (m *Manager) Mount(req *MountRequest) (*MountResponse, error) { func (m *Manager) Mount(req *MountRequest) (*MountResponse, error) {
if req == nil { if req == nil {
return nil, errors.New("mount request is nil") return nil, errors.New("mount request is nil")
@@ -59,14 +59,14 @@ func (m *Manager) Mount(req *MountRequest) (*MountResponse, error) {
defer lock.Unlock() defer lock.Unlock()
if entry := m.getMount(req.VolumeID); entry != nil { if entry := m.getMount(req.VolumeID); entry != nil {
// If the previous weed mount process has died, the entry is // If the previous s3 mount process has died, the entry is
// stale and the FUSE mount is dead. Tear down the stale entry // stale and the FUSE mount is dead. Tear down the stale entry
// so we can start a fresh process; otherwise we would falsely // so we can start a fresh process; otherwise we would falsely
// report "already mounted" and a CSI-driver recovery would // report "already mounted" and a CSI-driver recovery would
// silently bind-mount onto a dead path. See s3/s3-csi#261. // silently bind-mount onto a dead path. See s3/s3-csi#261.
select { select {
case <-entry.process.exited: case <-entry.process.exited:
glog.Warningf("volume %s previous weed mount process has exited; replacing stale entry with a fresh mount", req.VolumeID) glog.Warningf("volume %s previous s3 mount process has exited; replacing stale entry with a fresh mount", req.VolumeID)
// Wait for wait()'s post-exit cleanup (FUSE unmount) to finish // Wait for wait()'s post-exit cleanup (FUSE unmount) to finish
// so ensureTargetClean below sees a quiescent path. // so ensureTargetClean below sees a quiescent path.
<-entry.process.done <-entry.process.done
@@ -89,18 +89,18 @@ func (m *Manager) Mount(req *MountRequest) (*MountResponse, error) {
m.mounts[req.VolumeID] = entry m.mounts[req.VolumeID] = entry
m.mu.Unlock() m.mu.Unlock()
// Proactively clear the entry once the weed mount process exits, // Proactively clear the entry once the s3 mount process exits,
// even if Unmount is never called. Without this, a process that // even if Unmount is never called. Without this, a process that
// crashes on its own (e.g. backend errors) leaves a stale entry // crashes on its own (e.g. backend errors) leaves a stale entry
// that fools later Mount calls into reporting success without // that fools later Mount calls into reporting success without
// starting a new process. // starting a new process.
go m.watchProcessExit(req.VolumeID, entry) go m.watchProcessExit(req.VolumeID, entry)
glog.Infof("started weed mount process for volume %s at %s", req.VolumeID, req.TargetPath) glog.Infof("started s3 mount process for volume %s at %s", req.VolumeID, req.TargetPath)
return &MountResponse{LocalSocket: entry.localSocket}, nil return &MountResponse{LocalSocket: entry.localSocket}, nil
} }
// watchProcessExit removes the entry for volumeID once its weed mount // watchProcessExit removes the entry for volumeID once its s3 mount
// process has finished exiting. It is safe to run concurrently with // process has finished exiting. It is safe to run concurrently with
// Unmount: if Unmount has already removed the entry (or replaced it // Unmount: if Unmount has already removed the entry (or replaced it
// with a fresh mount), the identity check ensures we leave the new // with a fresh mount), the identity check ensures we leave the new
@@ -113,11 +113,11 @@ func (m *Manager) watchProcessExit(volumeID string, entry *mountEntry) {
delete(m.mounts, volumeID) delete(m.mounts, volumeID)
// See removeMount: do not delete the per-volume lock — a // See removeMount: do not delete the per-volume lock — a
// concurrent Mount/Unmount may still be holding it. // concurrent Mount/Unmount may still be holding it.
glog.Infof("removed mount entry for volume %s after weed mount process exited (target: %s)", volumeID, entry.targetPath) glog.Infof("removed mount entry for volume %s after s3 mount process exited (target: %s)", volumeID, entry.targetPath)
} }
} }
// Unmount terminates the weed mount process associated with the provided request. // Unmount terminates the s3 mount process associated with the provided request.
func (m *Manager) Unmount(req *UnmountRequest) (*UnmountResponse, error) { func (m *Manager) Unmount(req *UnmountRequest) (*UnmountResponse, error) {
if req == nil { if req == nil {
return nil, errors.New("unmount request is nil") return nil, errors.New("unmount request is nil")
@@ -137,7 +137,7 @@ func (m *Manager) Unmount(req *UnmountRequest) (*UnmountResponse, error) {
return &UnmountResponse{}, nil return &UnmountResponse{}, nil
} }
// Note: We don't explicitly unmount here because weedMountProcess.wait() // Note: We don't explicitly unmount here because s3MountProcess.wait()
// handles the unmount when the process terminates (either gracefully or forcefully). // handles the unmount when the process terminates (either gracefully or forcefully).
// This centralizes unmount logic and avoids potential race conditions. // This centralizes unmount logic and avoids potential race conditions.
if err := entry.process.stop(); err != nil { if err := entry.process.stop(); err != nil {
@@ -152,7 +152,7 @@ func (m *Manager) Unmount(req *UnmountRequest) (*UnmountResponse, error) {
// Only remove from state after all cleanup operations succeeded // Only remove from state after all cleanup operations succeeded
m.removeMount(req.VolumeID) m.removeMount(req.VolumeID)
glog.Infof("stopped weed mount process for volume %s at %s", req.VolumeID, entry.targetPath) glog.Infof("stopped s3 mount process for volume %s at %s", req.VolumeID, entry.targetPath)
return &UnmountResponse{}, nil return &UnmountResponse{}, nil
} }
@@ -201,7 +201,7 @@ func (m *Manager) startMount(req *MountRequest) (*mountEntry, error) {
return nil, errors.New("mountArgs is required") return nil, errors.New("mountArgs is required")
} }
process, err := startWeedMountProcess(m.weedBinary, args, targetPath, req.VolumeID) process, err := startS3MountProcess(m.s3Binary, args, targetPath, req.VolumeID)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -264,14 +264,14 @@ type mountEntry struct {
targetPath string targetPath string
cacheDir string cacheDir string
localSocket string localSocket string
process *weedMountProcess process *s3MountProcess
} }
type weedMountProcess struct { type s3MountProcess struct {
cmd *exec.Cmd cmd *exec.Cmd
target string target string
// exited is closed as soon as cmd.Wait() returns, so callers can // exited is closed as soon as cmd.Wait() returns, so callers can
// detect that the weed mount process is gone without waiting for // detect that the s3 mount process is gone without waiting for
// the post-exit FUSE unmount step. // the post-exit FUSE unmount step.
exited chan struct{} exited chan struct{}
// done is closed after wait() finishes its full cleanup (including // done is closed after wait() finishes its full cleanup (including
@@ -279,7 +279,7 @@ type weedMountProcess struct {
done chan struct{} done chan struct{}
} }
func startWeedMountProcess(command string, args []string, target string, volumeID string) (*weedMountProcess, error) { func startS3MountProcess(command string, args []string, target string, volumeID string) (*s3MountProcess, error) {
cmd := exec.Command(command, args...) cmd := exec.Command(command, args...)
// Capture stdout/stderr and log with volume ID prefix for better debugging // Capture stdout/stderr and log with volume ID prefix for better debugging
@@ -292,17 +292,17 @@ func startWeedMountProcess(command string, args []string, target string, volumeI
return nil, fmt.Errorf("creating stderr pipe: %w", err) return nil, fmt.Errorf("creating stderr pipe: %w", err)
} }
glog.V(0).Infof("[%s] Starting weed mount: %s %s", volumeID, command, strings.Join(args, " ")) glog.V(0).Infof("[%s] Starting s3 mount: %s %s", volumeID, command, strings.Join(args, " "))
if err := cmd.Start(); err != nil { if err := cmd.Start(); err != nil {
return nil, fmt.Errorf("starting weed mount: %w", err) return nil, fmt.Errorf("starting s3 mount: %w", err)
} }
// Forward stdout/stderr with volume ID prefix for better debugging // Forward stdout/stderr with volume ID prefix for better debugging
go forwardLogs(stdoutPipe, volumeID, "stdout") go forwardLogs(stdoutPipe, volumeID, "stdout")
go forwardLogs(stderrPipe, volumeID, "stderr") go forwardLogs(stderrPipe, volumeID, "stderr")
process := &weedMountProcess{ process := &s3MountProcess{
cmd: cmd, cmd: cmd,
target: target, target: target,
exited: make(chan struct{}), exited: make(chan struct{}),
@@ -321,11 +321,11 @@ func startWeedMountProcess(command string, args []string, target string, volumeI
return process, nil return process, nil
} }
func (p *weedMountProcess) wait() { func (p *s3MountProcess) wait() {
if err := p.cmd.Wait(); err != nil { if err := p.cmd.Wait(); err != nil {
glog.Errorf("weed mount exit (pid: %d, target: %s): %v", p.cmd.Process.Pid, p.target, err) glog.Errorf("s3 mount exit (pid: %d, target: %s): %v", p.cmd.Process.Pid, p.target, err)
} else { } else {
glog.Infof("weed mount exit (pid: %d, target: %s)", p.cmd.Process.Pid, p.target) glog.Infof("s3 mount exit (pid: %d, target: %s)", p.cmd.Process.Pid, p.target)
} }
// Signal exit immediately so Manager.Mount can detect a dead // Signal exit immediately so Manager.Mount can detect a dead
@@ -339,9 +339,9 @@ func (p *weedMountProcess) wait() {
close(p.done) close(p.done)
} }
func (p *weedMountProcess) stop() error { func (p *s3MountProcess) stop() error {
if err := p.cmd.Process.Signal(syscall.SIGTERM); err != nil { if err := p.cmd.Process.Signal(syscall.SIGTERM); err != nil {
glog.Warningf("sending SIGTERM to weed mount failed: %v", err) glog.Warningf("sending SIGTERM to s3 mount failed: %v", err)
} }
select { select {
@@ -351,14 +351,14 @@ func (p *weedMountProcess) stop() error {
} }
if err := p.cmd.Process.Kill(); err != nil { if err := p.cmd.Process.Kill(); err != nil {
glog.Warningf("killing weed mount failed: %v", err) glog.Warningf("killing s3 mount failed: %v", err)
} }
select { select {
case <-p.done: case <-p.done:
return nil return nil
case <-time.After(1 * time.Second): case <-time.After(1 * time.Second):
return errors.New("timed out waiting for weed mount to stop") return errors.New("timed out waiting for s3 mount to stop")
} }
} }
pkg/mountmanager/manager_test.go
+6 -6
View File
@@ -6,7 +6,7 @@ import (
) )
// TestWatchProcessExitRemovesStaleEntry verifies that the manager's // TestWatchProcessExitRemovesStaleEntry verifies that the manager's
// background watcher clears a mount entry from its map when the weed // background watcher clears a mount entry from its map when the s3
// mount process exits on its own. Without this, a future Mount call for // mount process exits on its own. Without this, a future Mount call for
// the same volume would falsely receive an "already mounted" no-op // the same volume would falsely receive an "already mounted" no-op
// because the manager's state never noticed the dead process. // because the manager's state never noticed the dead process.
@@ -14,7 +14,7 @@ import (
func TestWatchProcessExitRemovesStaleEntry(t *testing.T) { func TestWatchProcessExitRemovesStaleEntry(t *testing.T) {
m := NewManager(Config{}) m := NewManager(Config{})
process := &weedMountProcess{ process := &s3MountProcess{
target: "/tmp/test-target", target: "/tmp/test-target",
exited: make(chan struct{}), exited: make(chan struct{}),
done: make(chan struct{}), done: make(chan struct{}),
@@ -35,7 +35,7 @@ func TestWatchProcessExitRemovesStaleEntry(t *testing.T) {
t.Fatalf("expected entry tracked before process exit, got %v", got) t.Fatalf("expected entry tracked before process exit, got %v", got)
} }
// Simulate the weed mount process dying. // Simulate the s3 mount process dying.
close(process.exited) close(process.exited)
close(process.done) close(process.done)
@@ -47,7 +47,7 @@ func TestWatchProcessExitRemovesStaleEntry(t *testing.T) {
} }
time.Sleep(10 * time.Millisecond) time.Sleep(10 * time.Millisecond)
} }
t.Fatal("watcher did not remove stale entry after weed mount process exited") t.Fatal("watcher did not remove stale entry after s3 mount process exited")
} }
// TestWatchProcessExitLeavesReplacedEntry verifies that if a fresh // TestWatchProcessExitLeavesReplacedEntry verifies that if a fresh
@@ -58,7 +58,7 @@ func TestWatchProcessExitRemovesStaleEntry(t *testing.T) {
func TestWatchProcessExitLeavesReplacedEntry(t *testing.T) { func TestWatchProcessExitLeavesReplacedEntry(t *testing.T) {
m := NewManager(Config{}) m := NewManager(Config{})
oldProcess := &weedMountProcess{ oldProcess := &s3MountProcess{
target: "/tmp/test-target", target: "/tmp/test-target",
exited: make(chan struct{}), exited: make(chan struct{}),
done: make(chan struct{}), done: make(chan struct{}),
@@ -68,7 +68,7 @@ func TestWatchProcessExitLeavesReplacedEntry(t *testing.T) {
process: oldProcess, process: oldProcess,
} }
newProcess := &weedMountProcess{ newProcess := &s3MountProcess{
target: "/tmp/test-target", target: "/tmp/test-target",
exited: make(chan struct{}), exited: make(chan struct{}),
done: make(chan struct{}), done: make(chan struct{}),
pkg/mountmanager/socket.go
+1 -1
View File
@@ -10,7 +10,7 @@ import (
// DefaultSocketDir is the default directory for volume sockets. // DefaultSocketDir is the default directory for volume sockets.
const DefaultSocketDir = "/var/lib/s3-mount" const DefaultSocketDir = "/var/lib/s3-mount"
// LocalSocketPath returns the unix socket path used to communicate with the weed mount process. // LocalSocketPath returns the unix socket path used to communicate with the s3 mount process.
// The baseDir parameter should be the directory where sockets are stored (e.g., derived from mountEndpoint). // The baseDir parameter should be the directory where sockets are stored (e.g., derived from mountEndpoint).
// Uses SHA256 hash (first 16 hex chars = 64 bits) to minimize collision risk. // Uses SHA256 hash (first 16 hex chars = 64 bits) to minimize collision risk.
func LocalSocketPath(baseDir, volumeID string) string { func LocalSocketPath(baseDir, volumeID string) string {
pkg/mountmanager/types.go
+4 -4
View File
@@ -1,6 +1,6 @@
package mountmanager package mountmanager
// MountRequest contains all information needed to start a weed mount process. // MountRequest contains all information needed to start a s3 mount process.
type MountRequest struct { type MountRequest struct {
VolumeID string `json:"volumeId"` VolumeID string `json:"volumeId"`
TargetPath string `json:"targetPath"` TargetPath string `json:"targetPath"`
@@ -14,7 +14,7 @@ type MountResponse struct {
LocalSocket string `json:"localSocket"` LocalSocket string `json:"localSocket"`
} }
// UnmountRequest contains the information needed to stop a weed mount process. // UnmountRequest contains the information needed to stop a s3 mount process.
type UnmountRequest struct { type UnmountRequest struct {
VolumeID string `json:"volumeId"` VolumeID string `json:"volumeId"`
} }
@@ -28,6 +28,6 @@ type ErrorResponse struct {
} }
const ( const (
// DefaultWeedBinary is the default executable name used to spawn weed mount processes. // DefaultS3Binary is the default executable name used to spawn s3 mount processes.
DefaultWeedBinary = "weed" DefaultS3Binary = "s3"
) )
test/test.sh
+4 -4
View File
@@ -6,13 +6,13 @@ git clone https://github.com/kubernetes-csi/csi-test.git -b v5.0.0
cd csi-test cd csi-test
make install make install
## Start Local weed server with filer ## Start Local s3 server with filer
weed server -dir=/tmp/seaweedfs/data -s3 -volume.max=100 -volume.port=8090 -master.volumeSizeLimitMB=256 s3 server -dir=/tmp/s3/data -s3 -volume.max=100 -volume.port=8090 -master.volumeSizeLimitMB=256
## Run CSI Driver ## Run CSI Driver
cd ../seaweedfs-csi-driver cd ../s3-csi-driver
make build make build
./_output/seaweedfs-csi-driver -endpoint="$endpoint" -alsologtostderr -v=5 -filer=localhost:8888 -nodeid=test ./_output/s3-csi-driver -endpoint="$endpoint" -alsologtostderr -v=5 -filer=localhost:8888 -nodeid=test
# Run CSI Sanity Tests # Run CSI Sanity Tests
../csi-test/cmd/csi-sanity/csi-sanity\ ../csi-test/cmd/csi-sanity/csi-sanity\