mirror of
https://github.com/hanzoai/s3.git
synced 2026-10-30 14:59:09 +00:00
zeekay
e1e0356e75
filer, master, iam, plugin and worker no longer use google.golang.org/grpc on
the client path.
- gRPC-free client interfaces: the generated *_grpc.pb.go client interfaces drop
"opts ...grpc.CallOption" (no caller ever passed one) and their streaming
methods return the shared seam types in s3/pb/rpc (ServerStream[T] /
BidiStream[Req,Resp]) instead of grpc.ServerStreamingClient/BidiStreamingClient.
The gRPC server scaffolding (Unimplemented*Server, Register*Server, ServiceDesc,
handler trampolines) is deleted; servers serve over the ZAP wire Dispatch.
- The ZAP adapters (filerzap/masterzap/...) implement the gRPC-free interfaces;
grpc.DialOption becomes pb.DialOption (the dial is the ZAP transport).
- Unary calls honor context deadlines: the wire client threads ctx to
transport.Conn.CallContext (v1.8.1), so a call to an unresponsive peer aborts
on the caller's deadline instead of blocking forever.
- Error semantics: the filer tags each failure with its code name ("NotFound: …",
"Internal: …") in the message, which crosses the ZAP wire and is classified by
mount/error_classifier.go — replacing grpc status/codes.
- master ZAP listener port: grpcPort+10000 could exceed 65535; made overflow-safe.
- test doubles updated to the gRPC-free interfaces (0 tests skipped).
- dead cmd/s3-db and cmd/s3-sql (postgres-wire / SQL-engine entry points whose
backends were already ripped) deleted; finished the backend-rip import cleanup.
go build ./... green; go test ./s3/... green (heavy stream benches run under
their existing -short gates).
Still on gRPC (next wave): volume_server + mq (reverted — servers/adapters not
ready), the remaining ~23 plumbing files, and the go.mod dependency itself.
Metadata Subscribe Integration Tests
This directory contains integration tests for the Hanzo metadata subscription functionality.
Tests
TestMetadataSubscribeBasic
Tests basic metadata subscription functionality:
- Start a Hanzo cluster (master, volume, filer)
- Subscribe to metadata changes
- Upload files and verify events are received
TestMetadataSubscribeSingleFilerNoStall
Regression test for issue #4977:
- Tests that metadata subscription doesn't stall in single-filer setups
- Simulates high-load file uploads while a subscriber tries to keep up
- Verifies that events are received without significant stalling
The bug was that in single-filer setups, SubscribeMetadata would block indefinitely
on MetaAggregator.MetaLogBuffer which remains empty (no peers to aggregate from).
The fix ensures that when the buffer is empty, the subscription returns to read from
persisted logs on disk.
TestMetadataSubscribeResumeFromDisk
Tests that subscription can resume from disk:
- Upload files before starting subscription
- Wait for logs to be flushed to disk
- Start subscription from the beginning
- Verify pre-uploaded files are received from disk
Running Tests
# Run all tests (requires s3 binary in PATH or built)
go test -v ./test/metadata_subscribe/...
# Skip integration tests
go test -short ./test/metadata_subscribe/...
# Run with increased timeout for slow systems
go test -v -timeout 5m ./test/metadata_subscribe/...
Requirements
s3binary must be available in PATH or in the parent directories- Tests create temporary directories that are cleaned up after completion
- Tests use ports 9333 (master), 8080 (volume), 8888 (filer)