mirror of https://github.com/hanzoai/kafka.git synced 2026-10-30 14:48:25 +00:00

Files

T

zeekayandHanzo Dev 7347068076 upstream attribution belongs in NOTICE, not in the README

The README is the product's front door and carries our brand. NOTICE is where a
fork's provenance is legally required to live, and it is the file a licence audit
reads. Naming the upstream in the README put attribution in the one place nobody
checks for it and branding in a place that had to share.

Where the NOTICE already named the same upstream, the moved line was dropped
rather than duplicated, and markdown link syntax was flattened -- NOTICE is plain
text, not a rendered document.

Co-authored-by: Hanzo Dev <[email protected]>

2026-08-20 16:13:37 -07:00

2.0 KiB

Raw Permalink Blame History

Hanzo Kafka

Kafka wire protocol adapter for Hanzo PubSub (NATS JetStream). Accepts standard Kafka clients and translates requests to NATS underneath, allowing any Kafka producer/consumer to work against Hanzo infrastructure without code changes.

Features

Topic creation, produce, and consume via the Kafka binary protocol
Cluster mode with Raft consensus and Serf membership
Compression support
Multiple log segments per partition
Compatible with standard Kafka CLI tools and client libraries

Quick Start

Start a single-node broker on port 9092:

go run main.go --bootstrap --node-id 1

Test with any Kafka client:

# Create topic
kafka-topics.sh --create --topic events --bootstrap-server localhost:9092

# Produce
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic events

# Consume
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic events --from-beginning

Cluster Mode

Cluster mode uses Raft for distributed state and HashiCorp Serf for membership management.

# Bootstrap first node
go run main.go --bootstrap --node-id 1 --serf-addr 127.0.0.1:3331

# Join additional nodes
go run main.go --node-id 2 --broker-port 9093 --raft-addr localhost:2222 --serf-addr 127.0.0.1:3332 --serf-join "127.0.0.1:3331"
go run main.go --node-id 3 --broker-port 9094 --raft-addr localhost:2223 --serf-addr 127.0.0.1:3333 --serf-join "127.0.0.1:3331"

Kubernetes

In Hanzo infrastructure, Stream runs as a deployment in the hanzo namespace and connects to NATS via --pubsub-url nats://pubsub.hanzo.svc:4222.

Two instances are typically deployed:

Deployment	Service	Purpose
`insights-kafka`	`insights-kafka:9092`	Dedicated to Insights pipeline
`stream`	`stream:9092`	General purpose

Running Tests

Tests use Kafka CLI tools for end-to-end verification:

export KAFKA_BIN_DIR=/path/to/kafka_2.13-3.9.0/bin
go test -v ./...

Credits

License

MIT

2.0 KiB Raw Permalink Blame History