mirror of
https://github.com/zap-proto/benchmarks.git
synced 2026-10-30 15:06:01 +00:00
zeekay
78c3025bf0
Real-world benchmarks comparing ZAP vs JSON/Protobuf: Serialization (Go): - Tool call encode: 21x faster - Passthrough routing: 656x faster - Batch 100 messages: 21x faster, 0 allocations Blockchain/Consensus (Go): - Warp message: 70x faster encode, 244x faster decode - Consensus vote: 1,438x faster - State access: 736,458x faster (mmap simulation) MCP/Agent (Python): - Memory: 99.7% reduction (825MB → 2.4MB for 100 servers) - 20-agent orchestration: 2x faster, 8x less memory Distributed Inference (Python): - KV cache transfer: 926x faster - Batch prompts: 200x faster, 68% smaller - Speculative decode: 210x faster Benchmarks for: - Warp messaging (cross-chain) - Consensus voting and attestations - Multi-agent tool calling (MCP) - Distributed AI inference
50 lines
1.6 KiB
Makefile
50 lines
1.6 KiB
Makefile
.PHONY: all setup bench bench-serialize bench-agents bench-blockchain bench-inference report charts clean
|
|
|
|
all: bench
|
|
|
|
setup:
|
|
@echo "Setting up benchmark environment..."
|
|
cd bench-serialize && go mod tidy
|
|
cd bench-agents && pip install -r requirements.txt
|
|
cd bench-blockchain && go mod tidy
|
|
@echo "Setup complete."
|
|
|
|
bench: bench-serialize bench-agents bench-blockchain bench-inference
|
|
@echo "All benchmarks complete. Results in results/"
|
|
|
|
bench-serialize:
|
|
@echo "Running serialization benchmarks..."
|
|
@mkdir -p results
|
|
cd bench-serialize && go test -bench=. -benchmem -count=10 -json > ../results/serialize.json
|
|
cd bench-serialize && go test -bench=. -benchmem -count=10 | tee ../results/serialize.txt
|
|
|
|
bench-agents:
|
|
@echo "Running agent benchmarks..."
|
|
@mkdir -p results
|
|
cd bench-agents && python benchmark_mcp_overhead.py | tee ../results/agents.json
|
|
cd bench-agents && python benchmark_multi_agent.py | tee ../results/multi_agent.json
|
|
|
|
bench-blockchain:
|
|
@echo "Running blockchain benchmarks..."
|
|
@mkdir -p results
|
|
cd bench-blockchain && go test -bench=. -benchmem -count=10 -json > ../results/blockchain.json
|
|
cd bench-blockchain && go test -bench=. -benchmem -count=10 | tee ../results/blockchain.txt
|
|
|
|
bench-inference:
|
|
@echo "Running inference benchmarks..."
|
|
@mkdir -p results
|
|
cd bench-inference && python benchmark_inference.py | tee ../results/inference.json
|
|
|
|
report:
|
|
@echo "Generating benchmark report..."
|
|
python scripts/generate_report.py
|
|
|
|
charts:
|
|
@echo "Generating charts..."
|
|
python scripts/generate_charts.py
|
|
|
|
clean:
|
|
rm -rf results/
|
|
find . -name "__pycache__" -type d -exec rm -rf {} +
|
|
find . -name "*.pyc" -delete
|