papers/verus-analytics/sections/02-background.tex

\section{Background}


\subsection{Commerce Analytics Requirements}

E-commerce analytics spans multiple use cases:

\textbf{Operational dashboards.} Real-time visibility into key metrics: orders per minute, revenue, conversion rate, error rate. Latency requirement: seconds.

\textbf{Marketing attribution.} Understanding which channels and campaigns drive conversions. Requires joining touchpoints across sessions. Latency requirement: minutes to hours.

\textbf{Cohort analysis.} Comparing behavior across customer segments defined by acquisition date, first purchase category, or other attributes. Latency requirement: hours.

\textbf{Ad-hoc investigation.} Drilling into anomalies or exploring hypotheses. Query patterns unpredictable. Latency requirement: seconds to minutes.

\subsection{Event Model}

Commerce events follow a common structure:

\begin{lstlisting}[language=Python,caption=Event schema]
@dataclass
class CommerceEvent:
    event_id: str          # UUID
    event_type: str        # page_view, add_to_cart, purchase, etc.
    timestamp: datetime    # Event time
    user_id: Optional[str] # Authenticated user
    session_id: str        # Session identifier

    # Dimensions
    store_id: str
    device_type: str       # desktop, mobile, tablet
    country: str
    referrer: str
    utm_source: Optional[str]
    utm_medium: Optional[str]
    utm_campaign: Optional[str]

    # Measures (event-type specific)
    product_id: Optional[str]
    product_category: Optional[str]
    quantity: Optional[int]
    revenue: Optional[Decimal]
\end{lstlisting}

Event volumes vary widely. A small merchant might generate 1,000 events per day; a large retailer during a flash sale might generate 100,000 events per second.

\subsection{Existing Approaches}

\textbf{Batch data warehouses.} Systems like Amazon Redshift, Google BigQuery, and Snowflake excel at complex analytical queries over historical data. They process data in batches, typically with hour-scale latency. Query flexibility is excellent; freshness is poor.

\textbf{Stream processors.} Apache Storm, Flink, and Kafka Streams process events in real-time. They excel at continuous aggregations but struggle with ad-hoc queries requiring historical context.

\textbf{OLAP cubes.} Pre-aggregated cubes (Druid, Pinot, ClickHouse) offer fast queries over predefined dimensions. They trade flexibility for speed—queries outside the cube's schema require falling back to raw data.

\textbf{Lambda architecture.} Combining batch and stream layers provides both freshness and flexibility but doubles infrastructure complexity. Maintaining consistency between layers is notoriously difficult.