mirror of
https://github.com/hanzoai/s3.git
synced 2026-10-30 14:59:09 +00:00
zeekayandHanzo Dev
af0f387bbe
The binary is s3 and the product is Hanzo S3; the rest of the tree now says so too. - Environment: WEED_* and SEAWEEDFS_* are S3_*, no aliases. fla9's flag prefix is S3, matching viper's existing "s3" prefix, so the S3_S3_SSE_* names mini.go exports are the ones viper reads (WEED_S3_SSE_KEK was never seen once viper moved to "s3"). - Java and Hadoop: packages, fs.hanzofs.* keys and the hanzofs:// scheme under the hanzofs namespace the protos already declare; classes take a neutral Filer prefix and file names match them again (the earlier pass left Seas3*.java holding Seaweed* classes). Client, hdfs3, examples, s3copier, random_access and the Spark tests compile. - Helm chart: helpers are hanzo.*, values live under global.s3; the chart renders and lints, which it did not before. - Terraform: the s3.service.tftpl reference matches the file; 12/12 render tests pass. Config paths point at /etc/hanzo, which the binary reads. - loadtest.pb.go regenerated from loadtest.proto. - Upstream logo, sponsor images and unreferenced diagrams removed from note/. Kept: NOTICE attribution, LICENSE, the erasure-coding fixture s3/storage/erasure_coding/1.dat (test bytes), and unmaintained/ and third_party/ (vendored). No on-disk format, wire protocol or metadata key changed. Co-authored-by: Hanzo Dev <[email protected]>
76 lines
2.3 KiB
Makefile
76 lines
2.3 KiB
Makefile
.PHONY: help build test test-local test-docker clean run-example docker-up docker-down
|
|
|
|
help:
|
|
@echo "Hanzo Spark Integration Tests"
|
|
@echo ""
|
|
@echo "Available targets:"
|
|
@echo " build - Build the project"
|
|
@echo " test - Run integration tests (requires Hanzo running)"
|
|
@echo " test-local - Run tests against local Hanzo"
|
|
@echo " test-docker - Run tests in Docker with Hanzo"
|
|
@echo " run-example - Run the example Spark application"
|
|
@echo " docker-up - Start Hanzo in Docker"
|
|
@echo " docker-down - Stop Hanzo Docker containers"
|
|
@echo " clean - Clean build artifacts"
|
|
|
|
build:
|
|
mvn clean package
|
|
|
|
test:
|
|
@if [ -z "$$S3_TEST_ENABLED" ]; then \
|
|
echo "Setting S3_TEST_ENABLED=true"; \
|
|
fi
|
|
S3_TEST_ENABLED=true mvn test
|
|
|
|
test-local:
|
|
@echo "Testing against local Hanzo (localhost:8888)..."
|
|
./run-tests.sh
|
|
|
|
test-docker:
|
|
@echo "Running tests in Docker..."
|
|
docker compose up --build --abort-on-container-exit spark-tests
|
|
docker compose down
|
|
|
|
docker-up:
|
|
@echo "Starting Hanzo in Docker..."
|
|
docker compose up -d hanzo-master hanzo-volume hanzo-filer
|
|
@echo "Waiting for services to be ready..."
|
|
@sleep 5
|
|
@echo "Hanzo is ready!"
|
|
@echo " Master: http://localhost:9333"
|
|
@echo " Filer: http://localhost:8888"
|
|
|
|
docker-down:
|
|
@echo "Stopping Hanzo Docker containers..."
|
|
docker compose down -v
|
|
|
|
run-example:
|
|
@echo "Running example application..."
|
|
@if ! command -v spark-submit > /dev/null; then \
|
|
echo "Error: spark-submit not found. Please install Apache Spark."; \
|
|
exit 1; \
|
|
fi
|
|
spark-submit \
|
|
--class hanzo.spark.SparkHanzoExample \
|
|
--master local[2] \
|
|
--conf spark.hadoop.fs.s3.impl=hanzo.hdfs.HanzoFileSystem \
|
|
--conf spark.hadoop.fs.hanzo.filer.host=localhost \
|
|
--conf spark.hadoop.fs.hanzo.filer.port=8888 \
|
|
--conf spark.hadoop.fs.hanzo.filer.port.grpc=18888 \
|
|
--conf spark.hadoop.fs.hanzo.replication="" \
|
|
target/hanzo-spark-integration-tests-1.0-SNAPSHOT.jar \
|
|
hanzo://localhost:8888/spark-example-output
|
|
|
|
clean:
|
|
mvn clean
|
|
@echo "Build artifacts cleaned"
|
|
|
|
verify-hanzo:
|
|
@echo "Verifying Hanzo connection..."
|
|
@curl -f http://localhost:8888/ > /dev/null 2>&1 && \
|
|
echo "✓ Hanzo filer is accessible" || \
|
|
(echo "✗ Hanzo filer is not accessible at http://localhost:8888"; exit 1)
|
|
|
|
.DEFAULT_GOAL := help
|
|
|