papers/lux-institutional-custody/sections/08-operational-procedures.tex

\section{Operational Procedures}

\label{sec:ops}
%% =====================================================================

The custody architecture is only as strong as its day-to-day
operations. This section specifies the procedures that the custody
operations team must execute and against which they must be audited.

\subsection{Daily}

\begin{enumerate}[nosep]
\item \textbf{Balance reconciliation across nine chains.} At
00:00 UTC the orchestrator publishes per-chain balance proofs (BTC,
ETH, all EVM chains in the fund's policy, the Lux X-Chain and
C-Chain). Each balance is computed two ways: (a) from the chain's
canonical RPC, (b) from the M-Chain custody ledger
(LP-006 \cite{lp006}). The two figures must match to within one
chain block; any mismatch raises an incident.
\item \textbf{Hot-tier policy review.} The previous twenty-four hours
of hot-tier movements are reviewed by an ops engineer not on the
prior day's signing rotation. Each movement is checked against the
day's pre-authorised destination allowlist and per-day cap.
\item \textbf{Heartbeat across HSMs.} A signed-nonce heartbeat is
issued to every cold and warm HSM; non-response within ten minutes
raises an incident.
\item \textbf{Dependency monitoring.} Vendor security advisories
(Thales, Utimaco, AWS, GCP, Azure, Yubico) are scanned by the on-call
engineer.
\end{enumerate}

\subsection{Weekly}

\begin{enumerate}[nosep]
\item \textbf{Independent share-custody attestation.} Each share
custodian (the on-prem DC team, the cloud HSM admin, the airgap vault
team) submits a signed weekly attestation that the share remains
under their control and that no anomalous access occurred.
\item \textbf{Operations rota review.} The signing-shift rota is
reviewed for two-person-rule compliance: no single individual was
on every shift, no single individual was on a hot shift adjacent to
their warm shift, etc.
\item \textbf{Allowlist diff review.} Any addition to the destination
allowlist in the prior week is reviewed and re-approved.
\end{enumerate}

\subsection{Monthly}

\begin{enumerate}[nosep]
\item \textbf{Hot-tier proactive refresh.} CGGMP21 share refresh on
hot-tier shares; public key unchanged. Approximately 30 minutes of
ops work.
\item \textbf{Disaster-recovery tabletop.} A scenario (cloud provider
outage, on-prem DC fire, vault unreachable) is exercised on paper;
the ops team produces a written response.
\item \textbf{Vendor firmware review.} Pending vendor firmware
updates are reviewed against the firmware-update policy
(Section~\ref{sec:hsm}).
\end{enumerate}

\subsection{Quarterly}

\begin{enumerate}[nosep]
\item \textbf{Warm-tier proactive refresh.} CGGMP21 share refresh
on warm-tier shares.
\item \textbf{Live disaster-recovery drill.} A real failover is
executed: one cold or warm share is taken offline and a real signing
ceremony is completed using the remaining quorum. Drills are
publicly logged on M-Chain.
\item \textbf{Allowlist re-baselining.} The full per-tier destination
allowlist is reviewed and stale entries removed.
\end{enumerate}

\subsection{Annually}

\begin{enumerate}[nosep]
\item \textbf{Cold-tier proactive refresh.} CGGMP21 share refresh
on cold-tier shares; public key unchanged.
\item \textbf{Full SOC 2 Type II audit.} External audit firm conducts
the annual SOC 2 Type II audit \cite{soc2}.
\item \textbf{ISO 27001 surveillance audit.} ISO 27001 surveillance
audit \cite{iso27001}.
\item \textbf{Threat-model review.} The threat model
(Section~\ref{sec:threat}) is re-examined; if any adversary capability
has materially shifted, the architecture is reviewed.
\end{enumerate}

\subsection{Multi-Year}

\begin{enumerate}[nosep]
\item \textbf{Cold-tier full re-DKG.} Every five years, or sooner if
the threat model shifts (e.g.\ a credible quantum advance, a serious
HSM-firmware vulnerability), a full cold-tier re-DKG is performed.
The new public key is published; assets are migrated through a tier
transfer.
\item \textbf{Quantum-readiness review.} Annual review of NIST PQC
schedule, IBM and Google quantum-volume disclosures, and academic
quantum-attack progress on secp256k1.
\end{enumerate}

\subsection{Roles and Two-Person Rule}

\begin{table}[h]
\centering
\small
\begin{tabular}{p{3.5cm}p{4cm}p{6.5cm}}
\toprule
\textbf{Role} & \textbf{Headcount} & \textbf{Authority} \\
\midrule
Custody Operations Engineer &
$\geq 6$ &
Initiate signing requests; one share's HSM access \\
Custody Approver &
$\geq 4$ &
Approve signing requests; one share's HSM access \\
Airgap Operator &
$\geq 2$ &
Carry transport medium across airgap; vault access \\
Custody Lead &
$1$ + deputy &
Cross-tier authorisation; allowlist edits \\
Witness (external) &
audit firm &
Ceremony attendance; signed transcripts \\
Auditor (external) &
audit firm &
SOC 2 Type II, ISO 27001 \\
\bottomrule
\end{tabular}
\caption{Custody team roles.}
\label{tab:roles}
\end{table}

The two-person rule is enforced at every step: every signing request
involves at least one engineer, one approver, and (for cold) two
airgap operators. No individual can be both engineer and approver
on the same request. No individual can hold material access to more
than one share. Quarterly attestations confirm the access boundaries.

\subsection{Logging and Audit Trail}

Every operation produces a structured log entry committed to the
M-Chain custody ledger (LP-006 \cite{lp006}). The entry contains:
operator identity, role, action, request hash, share index, HSM
serial, timestamp, and a signed acknowledgement. The log is hash-chained
and signed with both BLS and ML-DSA so that the audit log is forward-secure
against a future quantum adversary (LP-020 \cite{lp020}).

%% =====================================================================