Snowplow 0.9.8 released for mobile analytics

We are hugely excited to announce the release of the long-awaited Snowplow version 0.9.8, adding event analytics support for iOS and Android applications. Mobile event analytics has been the most requested feature from the Snowplow community for some time, with many users keen to feed their Snowplow data pipeline with events from mobile apps, alongside their existing websites and server software.

Mobile event analytics is a major step in Snowplow’s journey from a web analytics tool to a general-purpose event analytics platform. It doesn’t make much sense to silo your company’s event streams into separate analytics packages based on whether those events originate on web, mobile or somewhere else; at Snowplow we believe that there is huge value in unifying those disparate event streams into a single event pipeline like Snowplow.

Adding mobile support for Snowplow is really a few different releases:

• Snowplow 0.9.8, which adds POST support to our Clojure Collector and upgrades our Enrichment process to support POST payloads containing multiple events
• A new event tracker for iOS, see today’s accompanying iOS Tracker blog post
• A new event tracker for Android, see today’s accompanying Android Tracker blog post
• New mobile-specific JSON Schemas available in Iglu Central, mobile_context and geolocation_context

Huge thanks to Snowplow intern Jonathan Almeida for his excellent work on the mobile trackers over the summer!

Below the fold we will cover:

1. Our updated Clojure Collector
2. Our updated Enrichment process
3. Upgrading
4. Documentation and help

1. Our updated Clojure Collector

We have upgraded the Clojure Collector to support POST requests. Batching multiple events in the client and then sending that batch via a POST request is standard operating procedure in mobile analytics, designed to minimize data usage and preserve battery life.

Both the new Android and iOS Trackers can send events via POST; you will also find POST support in the most recent versions of the Python and Ruby Trackers.

As we cannot support POST in the CloudFront Collector, we recommend that Snowplow users wanting to send events from mobile devices switch over to the Clojure Collector. Adding POST support to the Scala Stream Collector is on our roadmap.

2. Our updated Enrichment process

We have updated the Hadoop Snowplow Enrichment process to version 0.7.0; the Shredding process is unchanged.

The updated Enrichment process can now handle raw events sent to the updated Clojure Collector via POST. The updated process supports all existing collector formats, but additionally it can now:

1. Validate a log record’s POST payload and content-type if set
2. Extract multiple raw events from a POST payload
3. Feed each of those raw events through the rest of the Enrichment process

Read on for upgrading instructions.

3. Upgrading

This release bumps the Hadoop Enrichment process to version 0.7.0.

In your EmrEtlRunner’s config.yml file, update your Hadoop enrich job’s version to 0.7.0, like so:

 :versions: :hadoop_enrich: 0.7.0 # WAS 0.6.0

For a complete example, see our sample config.yml template.

Please make sure that you upgrade the Hadoop Enrichment process to 0.7.0 before upgrading your collector.

This release bumps the Clojure Collector to version 0.7.0.

To upgrade to this release:

1. Download the new warfile by right-clicking on this link and selecting “Save As…”
2. Log in to your Amazon Elastic Beanstalk console
3. Browse to your Clojure Collector’s application
4. Click the “Upload New Version” and upload your warfile

Both of the new trackers send mobile-related context conforming to the mobile_context JSON Schema, as a custom context automatically attached to each event.

If you are running Redshift, you can deploy the mobile_context table into your database using this this script.

The Android Tracker also optionally sends a geolocation-related context relating to the geolocation_context JSON Schema; support for this in the iOS Tracker is planned soon.

If you are running Redshift, you can deploy the mobile_context table into your database using this this script.

4. Documentation and help

Mobile event analytics represents a major new feature for Snowplow – there are likely to be some initial bugs and issues to iron out, in 0.9.8 itself or indeed in the new trackers. As always, if you do run into any issues or don’t understand any of the above changes, please raise an issue or get in touch with us via the usual channels.

For more details on this release, please check out the 0.9.8 Release Notes on GitHub.