Snowplow Golang Tracker 0.1.0 released

We are pleased to announce the release of the Snowplow Golang Tracker. The Tracker is designed to work asynchronously within your Golang code to provide great performance in your applications and servers, even under heavy load, while also storing all of your events persistently in the event of network failure.

It will also be used as a building block for a number of projects, including a new daemon to support robust asynchronous sending for the PHP Tracker.

In the rest of this post we will cover:

1. How to install the tracker
2. How to use the tracker
3. Core features
4. Roadmap
5. Documentation
6. Getting help

1. How to install the tracker

The release version of this Tracker is available directly from the GitHub repo and you can download it by running the following:

$ go get gopkg.in/snowplow/snowplow-golang-tracker.v1/tracker

We are using gopkg.in as a way of providing easy versioning within the Golang environment.

Please check out the Golang Tracker setup guide on our wiki for more information on setup.

That’s it! You’re now ready to start using the Tracker.

2. How to use the tracker

To setup the Go Tracker you first need to import the package into your code:

import "gopkg.in/snowplow/snowplow-golang-tracker.v1/tracker"
// You can also import the package with a shorter name...
import sp "gopkg.in/snowplow/snowplow-golang-tracker.v1/tracker"

You can then instantiate and start a new tracker like so:

emitter := sp.InitEmitter(sp.RequireCollectorUri("com.acme"))
tracker := sp.InitTracker(sp.RequireEmitter(emitter))

You are now ready to Track events, so let’s send a screen view event:

tracker.TrackScreenView(sp.ScreenViewEvent{
  Id: sp.NewString("Screen ID"),
})

And that’s all there is to it! Please check out the Golang Tracker technical documentation on our wiki for the Tracker’s full API.

3. Core features

The core features of the Snowplow Go Tracker include:

• Asynchronous event sending
• Outbound events are cached in a SQLite database to prevent event loss
• Value checking for all events to ensure invalid events are caught early

The general flow of the Tracker as an event goes through:

• A tracker.TrackXXX method is invoked
• The event is stored in the local database
• A long running goroutine is invoked to start processing these events (only started if it is not currently running)
• This process will then pull a range of events from the database and begin sending them to your configured collector URI
• More events arriving during sending will just be written to the database, and will then be picked up by the background sending process
• Each request is sent in its own goroutine

This model is very closely related to the one used in the Android, Objective-C and Unity Trackers, which are all backed by Sqlite databases.

If you absolutely have to have the Tracker be non-blocking from end to end you can use the go keyword to track events within another go routine. The Tracker is thread-safe and will behave as normal; however in load-testing the Tracker better performance was actually found with the blocking approach to adding the event to the database.

In our tests sending 6,000 events to a Snowplow Mini collector instance:

• Blocking addition: ~8 seconds
• Async addition: ~10 seconds

As the database can only accept one insert at a time the creation of go routines for individual Track invocations actually results in worse performance.

The Tracker also contains full support for secure event sending by both GET and POST request types, and the ability to combine events, for POST, up to a configurable byte limit. This yields much better performance versus our old approach of buffer limits.

4. Roadmap

We have big plans for the Snowplow Golang Tracker at Snowplow, including:

• Building a daemon to be used with the PHP Tracker for robust async sending (issue #54)
• Powering the Snowplow Tracking CLI, to let Snowplow users send events from the command-line on Linux, Windows and OS-X
• Building an equivalent to Logstash for tailing logfiles into Snowplow as well-structured Snowplow events (working title the Snowplow Logfile Source)

5. Documentation

• Golang Tracker setup guide for setting up the tracker
• Golang Tracker usage manual for instrumenting a Go app with the tracker
• Golang Tracker GoDoc documentation for GoDoc’s auto-generated documentation for the package

6. Getting help

We hope that you find the Snowplow Golang Tracker useful – of course, this is only its first release, so don’t be afraid to get in touch or raise an issue on GitHub!