mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
23 lines
953 B
TeX
23 lines
953 B
TeX
\section{Related Work}
|
|
|
|
\label{sec:related}
|
|
% ============================================================================
|
|
|
|
\textbf{vLLM}~\cite{vllm2023} introduced PagedAttention for efficient KV-cache
|
|
management. We adopt this technique and extend it with FP8 quantized cache and
|
|
cross-model cache pool sharing.
|
|
|
|
\textbf{Orca}~\cite{orca2022} proposed iteration-level scheduling for
|
|
continuous batching. Our system adds priority-aware preemption and
|
|
prefill--decode disaggregation.
|
|
|
|
\textbf{TensorRT-LLM}~\cite{tensorrt} provides optimized inference kernels but
|
|
requires NVIDIA-specific compilation. Our Candle-based approach achieves
|
|
comparable kernel performance with a more portable Rust codebase.
|
|
|
|
\textbf{Lux Network}~\cite{luxwhitepaper} provides the underlying consensus
|
|
layer for ACI compute attestation, ensuring that inference results can be
|
|
verified on-chain.
|
|
|
|
% ============================================================================
|