mirror of
https://github.com/hanzoai/s3.git
synced 2026-10-30 14:59:09 +00:00
zeekayandHanzo Dev
af0f387bbe
The binary is s3 and the product is Hanzo S3; the rest of the tree now says so too. - Environment: WEED_* and SEAWEEDFS_* are S3_*, no aliases. fla9's flag prefix is S3, matching viper's existing "s3" prefix, so the S3_S3_SSE_* names mini.go exports are the ones viper reads (WEED_S3_SSE_KEK was never seen once viper moved to "s3"). - Java and Hadoop: packages, fs.hanzofs.* keys and the hanzofs:// scheme under the hanzofs namespace the protos already declare; classes take a neutral Filer prefix and file names match them again (the earlier pass left Seas3*.java holding Seaweed* classes). Client, hdfs3, examples, s3copier, random_access and the Spark tests compile. - Helm chart: helpers are hanzo.*, values live under global.s3; the chart renders and lints, which it did not before. - Terraform: the s3.service.tftpl reference matches the file; 12/12 render tests pass. Config paths point at /etc/hanzo, which the binary reads. - loadtest.pb.go regenerated from loadtest.proto. - Upstream logo, sponsor images and unreferenced diagrams removed from note/. Kept: NOTICE attribution, LICENSE, the erasure-coding fixture s3/storage/erasure_coding/1.dat (test bytes), and unmaintained/ and third_party/ (vendored). No on-disk format, wire protocol or metadata key changed. Co-authored-by: Hanzo Dev <[email protected]>
107 lines
4.3 KiB
Docker
107 lines
4.3 KiB
Docker
# Pin the builder to the host arch and cross-compile the (CGO-free) Go binary,
|
|
# so arm64/arm/386 targets skip QEMU emulation of the whole compile.
|
|
FROM --platform=$BUILDPLATFORM golang:1.26.8-alpine AS builder
|
|
ENV GOTOOLCHAIN=auto
|
|
RUN apk add git g++ fuse
|
|
RUN mkdir -p /go/src/github.com/hanzo/
|
|
ARG BRANCH=${BRANCH:-master}
|
|
# Clone with full history and all tags to ensure all commits are available
|
|
RUN git clone --no-single-branch --tags https://github.com/hanzoai/s3 /go/src/github.com/hanzoai/s3
|
|
ARG TAGS
|
|
RUN cd /go/src/github.com/hanzoai/s3 && \
|
|
(git checkout $BRANCH || \
|
|
(echo "Checkout failed, fetching all history..." && \
|
|
git fetch --all --tags --prune && \
|
|
git checkout $BRANCH) || \
|
|
(echo "ERROR: Branch/commit $BRANCH not found in repository" && \
|
|
echo "Available branches:" && git branch -a && exit 1))
|
|
ARG TARGETOS TARGETARCH TARGETVARIANT
|
|
RUN cd /go/src/github.com/hanzoai/s3/s3 \
|
|
&& export LDFLAGS="-X github.com/hanzoai/s3/s3/util/version.COMMIT=$(git rev-parse --short HEAD)" \
|
|
&& export GOOS=$TARGETOS GOARCH=$TARGETARCH \
|
|
&& case "$TARGETARCH" in arm) export GOARM="${TARGETVARIANT#v}";; esac \
|
|
&& CGO_ENABLED=0 go build -tags "$TAGS" -ldflags "-extldflags -static ${LDFLAGS}" -o /go/bin/s3 .
|
|
|
|
# Rust volume server: use pre-built binary from CI when available (placed in
|
|
# s3-volume-prebuilt/ by the build-rust-binaries job), otherwise compile
|
|
# from source. Pre-building avoids a multi-hour QEMU-emulated cargo build
|
|
# for non-native architectures.
|
|
FROM alpine:3.23 as rust_builder
|
|
ARG TARGETARCH
|
|
ARG TAGS
|
|
COPY s3-volume-prebuilt/ /prebuilt/
|
|
COPY --from=builder /go/src/github.com/hanzoai/s3/hanzo-volume /build/hanzo-volume
|
|
COPY --from=builder /go/src/github.com/hanzoai/s3/s3 /build/s3
|
|
WORKDIR /build/hanzo-volume
|
|
RUN if [ -f "/prebuilt/s3-volume-${TARGETARCH}" ]; then \
|
|
echo "Using pre-built Rust binary for ${TARGETARCH}" && \
|
|
cp "/prebuilt/s3-volume-${TARGETARCH}" /s3-volume; \
|
|
elif [ "$TARGETARCH" = "amd64" ] || [ "$TARGETARCH" = "arm64" ]; then \
|
|
apk add --no-cache musl-dev openssl-dev protobuf-dev git rust cargo; \
|
|
if [ "$TAGS" = "5BytesOffset" ]; then \
|
|
cargo build --release; \
|
|
else \
|
|
cargo build --release --no-default-features; \
|
|
fi && \
|
|
cp target/release/s3-volume /s3-volume; \
|
|
else \
|
|
echo "Skipping Rust build for $TARGETARCH (unsupported)" && \
|
|
touch /s3-volume; \
|
|
fi
|
|
# Pre-built binaries arrive via GitHub Actions artifacts, which drop the
|
|
# executable bit, so the copied file is 0644 and exec fails with "Permission
|
|
# denied". Restore it (no-op for the empty placeholder, which stays size 0).
|
|
RUN chmod 0755 /s3-volume
|
|
|
|
FROM alpine AS final
|
|
LABEL author="Hanzo AI"
|
|
COPY --from=builder /go/bin/s3 /usr/bin/
|
|
# Copy Rust volume server binary (real binary on amd64/arm64, empty placeholder on other platforms)
|
|
COPY --from=rust_builder /s3-volume /usr/bin/s3-volume
|
|
RUN mkdir -p /etc/hanzo
|
|
COPY --from=builder /go/src/github.com/hanzoai/s3/docker/filer.toml /etc/hanzo/filer.toml
|
|
COPY --from=builder /go/src/github.com/hanzoai/s3/docker/entrypoint.sh /entrypoint.sh
|
|
|
|
# FIPS 140-3 mode is ON by default (Go 1.24+)
|
|
# To disable: docker run -e GODEBUG=fips140=off ...
|
|
|
|
# Install dependencies and create non-root user
|
|
RUN apk upgrade --no-cache && \
|
|
apk add --no-cache fuse curl su-exec libgcc libcrypto3 libssl3 && \
|
|
addgroup -g 1000 hanzo && \
|
|
adduser -D -u 1000 -G hanzo hanzo
|
|
|
|
# volume server gprc port
|
|
EXPOSE 18080
|
|
# volume server http port
|
|
EXPOSE 8080
|
|
# filer server gprc port
|
|
EXPOSE 18888
|
|
# filer server http port
|
|
EXPOSE 8888
|
|
# master server shared gprc port
|
|
EXPOSE 19333
|
|
# master server shared http port
|
|
EXPOSE 9333
|
|
# s3 server http port
|
|
EXPOSE 8333
|
|
# webdav server http port
|
|
EXPOSE 7333
|
|
|
|
# Create data directory and set proper ownership for hanzo user
|
|
RUN mkdir -p /data/filerldb2 && \
|
|
chown -R hanzo:hanzo /data && \
|
|
chown -R hanzo:hanzo /etc/hanzo && \
|
|
chmod 755 /entrypoint.sh
|
|
|
|
VOLUME /data
|
|
WORKDIR /data
|
|
|
|
# Entrypoint will handle permission fixes and user switching
|
|
ENTRYPOINT ["/entrypoint.sh"]
|
|
# Default to a complete single-process cluster (master+volume+filer+S3+admin)
|
|
# so the image is usable out of the box — including in environments like
|
|
# GitHub Actions service containers that cannot pass arguments to the entrypoint.
|
|
# Override with any other subcommand at `docker run` / compose time.
|
|
CMD ["mini", "-dir=/data"]
|