Real-Time Shopper Features using Apache Flink

This accelerator demonstrates how to leverage Snowplow's Behavioral Data to monitor and act on Shoppers’ behaviors while they're still navigating.

This accelerator demonstrates how to leverage Snowplow's Behavioral Data to monitor and act on Shoppers’ behaviors while they're still navigating.

Traditional Analytics stacks have focused on deriving insights on user behavior after the fact, in a business intelligence sense. Instead of this, it's possible to empower new ways to convert sales by proactively initiating a chat with a live agent, sending a unique discount code, or detecting influx of shoppers or products.

In this accelerator we show how to calculate key metrics based on behavioral data that can be used by diverse other systems. These metrics will be updated in near real-time and stored in fast and cheap storage, allowing ML features, Dashboards, Notifications, Chat systems, and others to consume the Shopper behavior at any point in time to take its best next action.

Those metrics can also be used to later feed longer-term dashboards and promote a shift-left architecture on Analytics, by processing metrics in the data pipeline itself to allow for a single source of truth for computations and aggregations logic, further enhancing the quality of reusability of data.

Behavioral data in real-time enables unique opportunities to retail, social/video networks, gaming/gambling, and industries where engagement is a key metric, allowing a deep look into each user, but also at general trends in the platform while they happen.

Use Cases

  • Individual User tracking in real-time
  • Trends and viral detection: See, monitor, alert and act on abnormal user behavior at scale, provide real-time enriched data to other systems and business decision makers
  • Real-time Analytics: Build dashboards with low-latency data refresh, ideal for high engagement and in-the-moment industries like sports, gaming and retail.
  • Dynamic windows with Data Streaming: Classify user behavior on the fly
  • Convert Shoppers on ecommerce: dynamic pricing, ads, offers, individual discount codes as the Shopper's behavior happens and conversion is threatened.

Expanded Use Cases

  • Live chat triggers: Detect hesitation or high-value cart activity and prompt a live agent interaction.
  • Real-time discount offers: Identify high-engagement users and offer personalized discounts during active sessions.
  • Session-based recommendations: Recommend products dynamically based on real-time browsing patterns within the session.
  • Anomaly detection: Identify and act on sudden traffic spikes, abnormal cart behavior, or unusual product interest in real time.
  • Dynamic content adjustments: Adjust banners, promotions, or product highlights based on live session behavior.
  • User re-engagement strategies: Track bounce behavior and trigger retargeting campaigns immediately after session drop-off.

Infrastructure Overview

The infrastructure for this accelerator includes:

  • Snowplow E-commerce Example Store - An example retail & ecommerce store, powered by Next.js and instrumented with Snowplow tracking.
  • Snowplow JavaScript Tracker - Captures events such as product views, engagement time, and add-to-cart actions on an e-commerce site.
  • Snowplow & Snowbridge - The Snowplow pipeline processes and enriches events before routing them to Kafka via Snowbridge.
  • Apache Kafka - The messaging layer to stream enriched events in real-time to the Live Shopper
  • Flink (Data streaming engine) - A processor that utilizes the data provided by Kafka and performs aggregations based on defined time buckets (windows). Here is where the business logic resides.
  • Redis - The platform that stores the result of the computed data by Flink. Each calculated metric will be stored here as `user:U:feature:F_frequency`, i.e.: `user:trent@anowplowanalytics.com:product_view_5m`.

Additional Insights

This accelerator demonstrates how real-time behavioral data can be implemented, to observe and improve user engagement and conversion rates in e-commerce. By processing shopper activity as it happens, businesses can deliver targeted interactions like personalized offers, support prompts, and dynamic content updates.

The system is designed for scalability, low latency, and easy integration with downstream applications such as dashboards, machine learning models, and marketing platforms. It also supports a shift-left approach by embedding data processing directly into the pipeline, improving consistency across operational and analytical workflows.

Beyond e-commerce, the framework can be adapted to industries that require a fast response to user behavior, including gaming, media, and live events.

Cloud Platforms

Data Platforms

Data Platforms

All Supported

Activation Products

Solution Partners

Key Outcomes

  • Individual User tracking in real-time
  • Trends and viral detection: See, monitor, alert and act on abnormal user behavior at scale, provide real-time enriched data to other systems and business decision makers
  • Real-time Analytics: Build dashboards with low-latency data refresh, ideal for high engagement and in-the-moment industries like sports, gaming and retail.
  • Dynamic windows with Data Streaming: Classify user behavior on the fly
  • Convert Shoppers on ecommerce: dynamic pricing, ads, offers, individual discount codes as the Shopper's behavior happens and conversion is threatened.

Get Started

Whether you’re modernizing your customer data infrastructure or building AI-powered applications, Snowplow helps eliminate engineering complexity so you can focus on delivering smarter customer experiences.