papers/hanzo-analytics-ml/sections/06-real-time-scoring.tex

\section{Real-Time Scoring}


\subsection{Serving Architecture}

Model predictions are served with sub-50ms latency:

\begin{lstlisting}[language=Python, caption=Prediction Service]
class PredictionService:
    def __init__(self):
        self.feature_store = FeatureStore()
        self.models = {
            'purchase': load_model('purchase_xgb.model'),
            'churn': load_model('churn_cox.model'),
        }

    def predict(self, customer_id: str) -> Predictions:
        features = self.feature_store.get(customer_id)
        return Predictions(
            purchase_prob=self.models['purchase'].predict(features),
            churn_prob=self.models['churn'].predict(features),
            segment=self.segment(features)
        )
\end{lstlisting}

\subsection{Caching Strategy}

Predictions are cached with TTL based on feature volatility:

\begin{equation}
TTL(c) = \min(TTL_{max}, \frac{1}{\lambda_c})
\end{equation}

where $\lambda_c$ is the customer's event rate.