We are pleased to announce the immediate availability of Snowplow 0.9.11. For the first time, you can now use Snowplow to collect, store and analyze event streams generated by supported third-party software.
Many Software-as-a-Service vendors publish their own internal event streams for customers to consume – these event stream APIs are often referred to as “webhooks”, sometimes as “streaming APIs”, “postbacks” or “HTTP response APIs”. Snowplow 0.9.11 adds first-class support for an initial set of these third-party webhooks.
This release is the first step in making our Snowplow trackers just one set of possible event streams into your Snowplow event warehouse and unified log:
For our initial 0.9.11 release we are adding support for three different webhook sources:
- MailChimp – for tracking email and email-related events delivered by MailChimp
- CallRail – for tracking completed telephone calls recorded by CallRail
- Iglu – for tracking Iglu-compatible self-describing events, enabling you to use schema-less webhook APIs such as AD-X Tracking
Here are the sections after the fold:
- MailChimp webhook support
- CallRail webhook support
- Iglu webhook support
- Updated Clojure Collector
- Updated change_form table
- Upgrading
- Roadmap and contributing
- Documentation and help
1. MailChimp webhook support
The MailChimp webhook adapter in Snowplow 0.9.11 lets you track email and email-related events delivered by MailChimp. Using this functionality, you can warehouse all email-related events alongside your existing Snowplow events.
For help setting up MailChimp support, see the MailChimp webhook setup wiki page.
For technical details on this adapter, see the MailChimp webhook adapter wiki page.
2. CallRail webhook support
The CallRail webhook adapter lets you track completed telephone calls recorded by CallRail, a SaaS call tracking system. This is great for working with online and mobile conversion funnels which conclude in a telephone call.
For help setting up CallRail support, see the CallRail webhook setup wiki page.
For technical details on this adapter, see the CallRail webhook adapter wiki page.
3. Iglu webhook support
The Iglu webhook adapter lets you track events sent via a GET request containing an Iglu-compatible event payload. You can send in whatever name-value pairs on the querystring that make sense for your event, but you must also include a schema parameter, set to a valid Iglu self-describing schema URI corresponding to the name-value pairs you sent through.
You can use this adapter with vendors who allow you define your own event types for “postback”. An example of a vendor who does this is AD-X Tracking.
For help setting up Iglu-compatible event webhook support, see the Iglu webhook setup wiki page.
For technical details on this adapter, see the Iglu webhook adapter wiki page.
4. Updated Clojure Collector
We have updated (#1131) the Clojure Collector so that it now returns 200 rather than an error code if you make a GET request to a path such as:
http://<collector host>/api-vendor/api-version?...This type of request paths were already supported for POST requests.
This is important for supporting webhooks which send their events via GET requests; it is also important for webhook providers like MailChimp who, although they send events via POST, send an initial GET and expect a 200 response code to confirm that the webhook endpoint (aka our Snowplow collector) is up-and-running.
5. Updated change_form table
We became aware of an issue loading events into the new com_snowplowanalytics_snowplow_change_form_1 table introduced in Snowplow 0.9.10 last week. While we continue to investigate this issue (#1134), we have updated the change_form table definition, specifically removing the not null constraint on the value column (#1162).
If you have already installed this table following the 0.9.10 release and are using enableFormTracking with the Snowplow JavaScript Tracker, we recommend upgrading to this updated table to prevent possible Redshift COPY failures.
6. Upgrading
This release bumps the Hadoop Enrichment process to version 0.9.0.
In your EmrEtlRunner’s config.yml file, update your Hadoop enrich job’s version to 0.9.0, like so:
:versions: :hadoop_enrich: 0.9.0 # WAS 0.8.0For a complete example, see our sample config.yml template.
This release bumps the Clojure Collector to version 0.9.0.
To upgrade to this release:
- Download the new warfile by right-clicking on this link and selecting “Save As…”
- Log in to your Amazon Elastic Beanstalk console
- Browse to your Clojure Collector’s application
- Click the “Upload New Version” and upload your warfile
If you have installed the com_snowplowanalytics_snowplow_change_form_1 table following the 0.9.10 release, then please upgrade it to the latest version. The 0.9.11 release contains an upgrade script, migrate_change_form_1_r1_to_r2.sql.
Also make sure to deploy Redshift tables for any webhooks you plan on ingesting into Snowplow. You can find the Redshift table deployment instructions on the corresponding webhook setup wiki pages:
7. Roadmap and contributing
We are already working on the next set of webhook adapters for Snowplow – you can follow our progress in the Webhooks 2 milestone in GitHub. The webhooks we will be supporting next will likely be:
We welcome any contributions of webhook adapters for other services – for details on getting started, please see How to integrate a webhook into Snowplow. Please note that contributing will require some experience in Scala at this time.
Similarly, if you would like to sponsor the Snowplow team to build a webhook adapter, do get in touch!
8. Documentation and help
Documentation relating to the new webhook support is available on the wiki:
As always, if you do run into any issues or don’t understand any of the new features, please raise an issue or get in touch with us via the usual channels.
For more details on this release, please check out the 0.9.11 Release Notes on GitHub.
