Snowplow Golang Tracker 0.1.0 released
We are pleased to announce the release of the Snowplow Golang Tracker. The Tracker is designed to work asynchronously within your Golang code to provide great performance in your applications and servers, even under heavy load, while also storing all of your events persistently in the event of network failure.
It will also be used as a building block for a number of projects, including a new daemon to support robust asynchronous sending for the PHP Tracker.
In the rest of this post we will cover:
1. How to install the tracker
The release version of this Tracker is available directly from the GitHub repo and you can download it by running the following:
We are using gopkg.in as a way of providing easy versioning within the Golang environment.
Please check out the Golang Tracker setup guide on our wiki for more information on setup.
That’s it! You’re now ready to start using the Tracker.
2. How to use the tracker
To setup the Go Tracker you first need to import the package into your code:
You can then instantiate and start a new tracker like so:
You are now ready to Track events, so let’s send a screen view event:
And that’s all there is to it! Please check out the Golang Tracker technical documentation on our wiki for the Tracker’s full API.
3. Core features
The core features of the Snowplow Go Tracker include:
- Asynchronous event sending
- Outbound events are cached in a SQLite database to prevent event loss
- Value checking for all events to ensure invalid events are caught early
The general flow of the Tracker as an event goes through:
tracker.TrackXXXmethod is invoked
- The event is stored in the local database
- A long running goroutine is invoked to start processing these events (only started if it is not currently running)
- This process will then pull a range of events from the database and begin sending them to your configured collector URI
- More events arriving during sending will just be written to the database, and will then be picked up by the background sending process
- Each request is sent in its own goroutine
This model is very closely related to the one used in the Android, Objective-C and Unity Trackers, which are all backed by Sqlite databases.
If you absolutely have to have the Tracker be non-blocking from end to end you can use the
go keyword to track events within another
go routine. The Tracker is thread-safe and will behave as normal; however in load-testing the Tracker better performance was actually found with the blocking approach to adding the event to the database.
In our tests sending 6,000 events to a Snowplow Mini collector instance:
- Blocking addition: ~8 seconds
- Async addition: ~10 seconds
As the database can only accept one insert at a time the creation of
go routines for individual Track invocations actually results in worse performance.
The Tracker also contains full support for secure event sending by both
POST request types, and the ability to combine events, for
POST, up to a configurable byte limit. This yields much better performance versus our old approach of buffer limits.
We have big plans for the Snowplow Golang Tracker at Snowplow, including:
- Building a daemon to be used with the PHP Tracker for robust async sending (issue #54)
- Powering the Snowplow Tracking CLI, to let Snowplow users send events from the command-line on Linux, Windows and OS-X
- Building an equivalent to Logstash for tailing logfiles into Snowplow as well-structured Snowplow events (working title the Snowplow Logfile Source)
- Golang Tracker setup guide for setting up the tracker
- Golang Tracker usage manual for instrumenting a Go app with the tracker
- Golang Tracker GoDoc documentation for GoDoc’s auto-generated documentation for the package