papers/hanzo-llm-gateway/sections/08-security.tex

\section{Security}

\label{sec:security}

\subsection{Credential Management}

Provider API keys are stored encrypted at rest using AES-256-GCM with keys derived from a master secret via HKDF. The gateway supports integration with external secret managers (HashiCorp Vault, AWS Secrets Manager, Infisical) for key rotation without service restart.

\subsection{Request Isolation}

Multi-tenant deployments require strict request isolation. The gateway enforces:

\begin{itemize}[leftmargin=*]
    \item \textbf{Key scoping}: API keys are bound to specific organizations with provider access lists.
    \item \textbf{Cache isolation}: Cache entries are scoped by organization to prevent cross-tenant information leakage.
    \item \textbf{Rate limit isolation}: Per-organization rate limits prevent noisy-neighbor effects.
    \item \textbf{Credential isolation}: Each organization can configure its own provider credentials, with the gateway's credentials as fallback.
\end{itemize}

\subsection{Content Filtering}

The gateway supports pluggable content filters executed as middleware:

\begin{itemize}[leftmargin=*]
    \item \textbf{PII detection}: Regex and NER-based detection of personally identifiable information with configurable actions (block, redact, log).
    \item \textbf{Prompt injection detection}: Heuristic and classifier-based detection of prompt injection attempts~\cite{perez2022ignore}.
    \item \textbf{Output filtering}: Post-processing filters for compliance-sensitive content.
\end{itemize}