How Software.com fueled a 250% increase in its user base with Snowplow

Background

Software.com quantifies the craft of software development by collecting data across the stack, providing insights that help developers and teams measure and improve. By tracking numerous data points across a team’s stack—from DevOps systems metadata and code editor activity, to Slack messaging, calendar data and even Spotify plays—Software.com can identify and protect valuable code time for engineering teams around the world.

In order to do this, it needs rich, reliable and highly customizable eventstream data. By integrating Snowplow into their data stack, Software.com has been able to purposefully create this data and build a data platform that gives them a real competitive advantage, overcoming the challenges of using byproduct exhaust data from multiple, disparate sources.

Snowplow has enabled us to stop worrying about the data ingestion side and the question of will it work… now we get to ask ‘what do we want it to do, and what are we going to deliver when we do?”

MASON MCLEAD | CTO AT SOFTWARE.COM

Challenges

Data is the lifeblood of Software.com’s products. They need high-quality, reliable data delivered at speed from multiple sources in order to provide their clients with real value. What makes their product so useful, however, was also causing challenges for their data team. Software.com tracks different types of events across numerous platforms: code editor telemetry, Git events, meetings and calendar events, Slack messaging metadata, and Spotify song plays. Each one contributes to building an accurate picture of a team’s productivity.

The challenge for Software.com was that the data from each source, and each of their individual metrics, was collected in different formats. This meant that their engineers had to spend time and effort preparing the data for storage and consumption—a thankless task that distracted from more meaningful projects. In addition, versioning was practically non-existent; the team had to manually track any changes to the growing amount of incoming data.

And while the engineers were grappling with different data formats, Software.com’s servers were struggling to keep up with the pace of new installs. As more and more events were tracked, the database would become locked. In order to reduce the amount of data flowing through and prevent this from happening, Software.com’s data team began aggregating data at the source. While this helped with reducing the frequency of database lock-ups, it meant that the data lost some of its granularity, ultimately rendering it less useful for the end users.

Solution

To resolve issues around data formats, versioning, and scalability, Software.com considered a number of options. One involved using an end-to-end AWS IoT solution, which would allow for a single data format across different sources and included a degree of modification, but ultimately lacked the flexibility that the team needed. It also implied that AWS would essentially ‘own’ Software.com’s data pipeline, which was a significant drawback.

The team also considered building a Redis-only solution, but discounted it in favor of standardizing all of their data using SQL (and avoiding narrow specialization in Redis CLI).

A more flexible option came in the form of Snowplow. The team at Software.com

appreciated that Snowplow could track event data from multiple sources and deliver it all in a single, defined format, without the need to clean and prepare for processing. The versioning of JSON schema also proved to be a major upside of Snowplow, as did the complete ownership of the pipeline and the possibility to add highly customized, granular events.

From the beginning, I was able to see that it was a really flexible platform with the custom JSON schemas you can make…that’s one of the primary reasons I went with Snowplow.”

MASON MCLEAD | CTO AT SOFTWARE.COM

With demand for Software.com’s products expected to grow significantly, Snowplow’s scalable pricing also proved to be a game changer; as the number of events collected each month increases, so does the cost-saving over similar tools.

Why did Software.com choose Snowplow?

• Universal Data Language – Snowplow ensures that all data that enters a warehouse has been validated and adheres to a unified format
• Pipeline ownership – Snowplow gives you complete control and ownership over your data infrastructure
• Customized events – Snowplow offers 130 standard out-of-the-box trackers, and provides great flexibility with custom event tracking

Results

By integrating Snowplow, powered by AWS, into their data stack, Software.com has created a unified format for its data and liberated its engineering team to work on more impactful projects. By choosing to buy rather than build their data infrastructure, Software.com has also been able to save on both time and costs.

Perhaps most important, however, is that Snowplow has enabled Software.com to gain a real competitive advantage in its market. The team’s data stack, combining Snowplow with AWS Kinesis and Snowflake, has allowed it to pass data from collection to analysis in roughly 20 seconds—hours ahead of the closest competition. As a result, Software.com’s user base has grown significantly. Since implementing Snowplow, Software.com’s users have jumped from 50,000 to 175,000, an increase of 250%.

In a market which increasingly demands real-time insights, Software.com is leading the way. By integrating Snowplow into their stack and harnessing the power of granular, high-quality behavioral data, it’s providing development teams around the world with genuinely valuable insights on how they work, and how to improve.

The biggest thing is that I don’t have to worry about that side [data ingestion] anymore, and I can refocus my energy onto the thing that’s actually going to drive revenue and real value.”

MASON MCLEAD | CTO AT SOFTWARE.COM

How you can get started with Snowplow

To learn more about how Snowplow can empower your organization with behavioral data creation, book in a chat with our team today. Alternatively, Try Snowplow is our free, easy-to-use version of our technology, which allows you to create your own behavioral data in under 30 mins.

‍