Files
skills/public/hanzo-ane.md

6.5 KiB

Hanzo ANE - Apple Neural Engine Training

Category: Hanzo Ecosystem Related Skills: hanzo/hanzo-engine.md, hanzo/hanzo-candle.md, hanzo/zenlm.md

Overview

Hanzo ANE enables training neural networks directly on Apple Neural Engine across ALL Apple Silicon (M1+). Reverse-engineered private API for direct ANE access — not just inference but actual gradient computation and weight updates on the neural engine.

Why Hanzo ANE?

  • ALL Apple Silicon: Works on M1, M2, M3, M4+ (not just M4)
  • Direct ANE access: Bypass CoreML, use private ANE API
  • Real training: Gradient computation + weight updates on ANE
  • 3 pipelines: Static conv, Static+ANE extras, Dynamic matmul
  • Universal runtime: Chip-specific optimizations auto-detected
  • Research tool: Understand and benchmark ANE capabilities

Key Discovery

The private API works on ALL Apple Silicon (M1+). Earlier failures were caused by wrong weight blob format (64-byte header vs correct 128-byte header).

OSS Base

Fork of maderix/ANE (8 commits merged). Repo: hanzoai/ANE.

When to use

  • Training small models on Apple Silicon ANE
  • Benchmarking ANE vs GPU vs CPU performance
  • Research into neural engine capabilities and limits
  • Optimizing inference for Apple devices
  • Edge AI model development on Mac

Hard requirements

  1. Apple Silicon Mac (M1, M2, M3, or M4)
  2. macOS 13+
  3. Xcode Command Line Tools
  4. C/Objective-C compiler (clang)

Quick reference

Item Value
Repo github.com/hanzoai/ANE (fork of maderix/ANE)
Build make all && make test
Data bash training/download_data.sh
Language Objective-C, Python
Upstream maderix/ANE (8 commits merged, issue #3 has benchmarks)

Performance Benchmarks

Model: Stories110M (109M parameters)

Chip Pipeline Speed TFLOPS Notes
M1 Max Static (conv) 206ms/step 0.45 ANE private API
M1 Max MLX N/A 6.72 peak Best training backend
M4 Static (conv) 107ms/step — Fastest ANE training
M4 Dynamic (matmul) 111ms/step — No recompile needed

Comparison: MLX remains the best overall training backend on Apple Silicon (6.72 TFLOPS peak on M1 Max), but ANE offers unique advantages for deployment-optimized models.

Three Pipelines

Pipeline Description Ops Compatibility
Static (conv) Separate weight files, convolution ops Conv2D M1 → M4+ (all chips)
Static + ANE extras 14% faster with ANE-specific optimizations Conv2D + ANE M1 → M4+ (all chips)
Dynamic (matmul) Uses matmul + slice_by_size, no recompile MatMul M4+ only (fails on M1-M3)

Pipeline Selection

Which chip?
├── M1, M2, M3 → Static (conv) or Static + ANE extras
└── M4+ → Any pipeline (Dynamic recommended)

Weight Blob Format

CRITICAL: Using the wrong format causes silent failures on M1-M3.

Correct Format (128-byte header)

Offset 0: [general header, 64 bytes]
Offset 64: Magic 0xEFBEADDE (4 bytes)
Offset 72: Data size (8 bytes)
Offset 80: Data offset = 128 (8 bytes)
Offset 128: [weight data begins]

Wrong Format (64-byte header)

  • Missing magic at offset 64
  • Data starts at offset 64 instead of 128
  • Works accidentally on M4 but fails on M1-M3

One-file quickstart

# Clone
git clone https://github.com/hanzoai/ANE.git
cd ANE

# Download training data
bash training/download_data.sh

# Build everything
make all

# Run tests
make test

# Run training (auto-detects chip and selects best pipeline)
make train

# Run specific pipeline (targets in training/Makefile)
cd training
make train # Standard training
make train_large # Large model training
make train_large_ane # Large model + ANE extras

# Benchmark
make bench

Universal Runtime

// ane_universal.h — auto-detects chip capabilities
#include "ane_universal.h"

ane_device_t device = ane_open();

// Check capabilities
bool has_inmem = ane_supports_inmem(device); // true on all chips now
bool has_dynamic = ane_supports_dynamic(device); // true on M4+ only

// Select best pipeline
ane_pipeline_t pipeline;
if (has_dynamic) {
 pipeline = ane_create_dynamic_pipeline(device);
} else {
 pipeline = ane_create_static_pipeline(device);
}

// Load weights (128-byte header format)
ane_weights_t weights = ane_load_weights("model.ane");
ane_bind_weights(pipeline, weights);

// Forward pass
ane_tensor_t input = ane_create_tensor(device, shape, data);
ane_tensor_t output = ane_forward(pipeline, input);

// Backward pass (training)
ane_tensor_t grad = ane_backward(pipeline, output, target);
ane_update_weights(pipeline, grad, learning_rate);

ane_close(device);

Project Structure

ANE/
├── ane_universal.h # Universal runtime (chip detection)
├── Makefile # Top-level build targets
├── training/
│ ├── Makefile # Training targets (train, train_large, train_large_ane)
│ ├── download_data.sh # Download Stories110M data
│ ├── train.m # Training entry point (Objective-C)
│ ├── test_*.m # Tests (Objective-C)
│ └── configs/ # Training configs
└── README.md

MLX Comparison

For most training workloads, MLX is recommended over direct ANE:

Feature ANE (Direct) MLX
Peak TFLOPS (M1 Max) 0.45 6.72
Ease of use Low (C API) High (Python)
Model support Custom only HuggingFace compatible
Training Basic Full (optimizers, schedulers)
Best for Research, deployment-optimized General training

Troubleshooting

Issue Cause Solution
Fails on M1/M2/M3 Wrong weight format Use 128-byte header, not 64-byte
Dynamic pipeline crash M1-M3 chip Use static pipeline instead
Build error Missing Xcode tools xcode-select --install
Low TFLOPS Wrong pipeline Try Static + ANE extras
Magic number wrong Byte order Use little-endian 0xEFBEADDE
  • hanzo/hanzo-engine.md - Rust inference engine (can use ANE for serving)
  • hanzo/hanzo-candle.md - Rust ML framework (Metal backend complement)
  • hanzo/zenlm.md - Zen model family (training targets)
  • hanzo/hanzo-jin.md - Multimodal framework

Last Updated: 2026-03-13 Category: Hanzo Ecosystem Related: ane, apple-silicon, neural-engine, training, m1, m4 Prerequisites: Apple Silicon Mac, C/Objective-C, macOS 13+