papers/hanzo-candle/sections/09-related-work.tex

\section{Related Work}

\label{sec:related}

\subsection{Inference Frameworks}

PyTorch~\cite{paszke2019pytorch} dominates both training and inference with extensive model support. TensorRT~\cite{tensorrt2017} provides NVIDIA-specific optimization. ONNX Runtime~\cite{onnxruntime2019} offers cross-platform inference. vLLM~\cite{kwon2023vllm} introduces PagedAttention for efficient LLM serving. SGLang~\cite{zheng2024sglang} provides efficient structured generation. Candle complements these with Rust-native safety and predictable performance.

\subsection{Rust ML Ecosystem}

Burn~\cite{burn2023} provides a Rust deep learning framework with automatic differentiation. Tract~\cite{tract2023} offers ONNX inference in Rust. tch-rs~\cite{tchrs2023} provides Rust bindings to LibTorch. Candle differs by implementing the tensor engine from scratch in pure Rust rather than binding to C++ libraries.

\subsection{C/C++ Inference}

llama.cpp~\cite{llamacpp2023} provides minimal C/C++ inference for Llama models. ggml~\cite{ggml2023} provides the underlying tensor library. whisper.cpp~\cite{whispercpp2023} handles speech recognition. These projects prioritize minimalism; Candle provides a more complete framework with type safety and multi-backend support.

\subsection{Kernel Optimization}

FlashAttention~\cite{dao2022flashattention,dao2023flashattention2} introduced IO-aware attention. Triton~\cite{tillet2019triton} provides a Python-based GPU kernel compiler. ThunderKittens~\cite{thunderkittens2024} explores hardware-aware kernel abstractions. Candle's fusion compiler operates at a higher level, automatically identifying fusion opportunities in model definitions.