Snowplow is often used to create behavioral data in web and mobile applications, but what about the billions of IoT devices that are in use in the world today? IoT has made it possible for the physical world to meet the digital world and cooperate with each other. Snowplow is well positioned to create behavioral data from IoT devices; IoT devices naturally have access to a lot of data points to describe user behavior.
The world is shifting into incorporating smaller and smarter technology in day-to-day life. The modern consumer wants a seamless experience where the technology knows what to do without much effort. Behavioral data is the key for this experience; providing value to the user. Behavioral data can also be used for smart, data informed business decisions; providing value to the organization.
IoT devices are often built and put in the wild, but by thinking about tracking user behavior and contextual information at the same time from the devices sensors, we can start to understand more deeply about how our customers’ environment affects their behavior; Snowplow gives you the powerful tools to be able to do that. Snowplow can be used by IoT device manufacturers to collect and create data on how their customers are using their product, this data can be analyzed to determine which features are most popular, how they are being used and what value they bring to the customers. Data on how a device is used by their customer base, and what behaviors it is driving, can be massively beneficial to the organization.
In this blog post, we will show how Snowplow can be used in IoT devices and the potential value it can bring. We will be creating a mock IoT device using a Raspberry Pi Pico W and a few sensors which can be used in retail stores to collect customer satisfaction data. Snowplow forms an important part of this project as it will validate and enrich the data with additional information and send it through to a data warehouse. The purpose of the device is so that a retail store can ask questions such as how customer satisfaction varies with temperature, humidity, weather, sound levels, light levels, etc. and obtain data-backed answers.
IoT Data Creation opportunities
Snowplow is often used in web and mobile applications to track user interactions. This will be exactly the same case for IoT devices but the user interface will be completely different. Typical data points on web and mobile are when a user enters a page, clicks a button on the application, how long they are watching a video for, etc. Whereas from IoT devices we can take advantage of the sensors that IoT devices have to collect data. Organizations can customize an IoT device however they want, incorporating any sensors that they would like.
Sensors can better track behavioral and environmental data; the data recorded from sensors can give a better understanding of the real life context the consumer is in, which could suggest behaviors.
The type of data that can be collected from IoT devices can vary significantly; it could be real-time tracking to predict user intentions, collect information about how their device is used, collect user interactions, collect environmental data etc.
Why Snowplow?
As we touched on in the intro, Snowplow can track data providing value to both organization and consumer. Real-time tracking can make a user’s experience smoother with predictive algorithms or spot potential issues with anomaly detection. It can also assist in making data-informed decisions. A powerful feature of Snowplow is that you can create your own data structures, using JSON schemas, which is useful for IoT devices as an event can have the contextual sensor data attached.
- Your organization may wish to invest more resources on already popular features to maintain and develop it further.
- Likewise, your organization may wish to invest more resources on features which are not as popular currently to develop it to a popular state and provide more value to their customers.
- Using this data, your organization may also decide to split their product offering, the original product and a new product with only the most popular and valuable features included which will have a reduced production cost due to its reduced functionality. Therefore it can be marketed as an inexpensive substitute to those in the market who would like your organization’s product but cannot afford their fully equipped product.
The Raspberry Pi Pico W device
For this example, we make use of a Raspberry Pi Pico W and a range of sensors and controls. In the rest of this article we will describe how to make a satisfaction meter with red, yellow and green buttons, temperature and humidity sensors, and which then sends environmental information to Snowplow as the user performs actions with the device.
Once a button is pressed, a Snowplow event can be generated with the temperature and humidity readings and weather from a weather API as context. This data can be used to find trends in customer behavior and satisfaction with the temperature and humidity levels the store has whilst taking the weather into account. We can use a free API such as the OpenWeathermap API in Python by feeding it the IP or geo-location of the device to determine the current weather
- Get response (red, yellow or green)
- Get readings from sensors
- Generate snowplow event e.g. satisfaction (good/average/bad)
- Attach sensor reading context e.g. (temperature and humidity readings)
- Enrich the event with location
- Use an API to determine the current weather conditions at location of the device
- Attach result to the event as context
- Send the Snowplow event!
Conclusion
IoT devices have a unique value in which they can collect real time contextual information, and when paired up with Snowplow’s Behavioral Data Platform, they can give you a better insight on how the environment can affect customer behavior.
This is very useful for making data-backed business decisions and be certain it’ll have a positive impact. More sensors can be added to build more and more environmental data such as light and sound intensity sensors.
You could look at trends in satisfaction, for example satisfaction rating and humidity. If a certain store suffers from high humidity then AC can be installed to lower that and improve the overall satisfaction.
The list of IoT devices is endless. If we’re able to create a mock satisfaction meter and gain valuable information about customers, imagine what you could do with other more complex devices. From smart assistants, smart home monitors, digital control systems to management systems. Any organization can benefit from Snowplow’s ability to gain extra information to improve business decisions, increase productivity of business operations and improve user experience.
Continue reading if you’d like a deep dive of the project.
The satisfaction meter project
How to assemble the device
For this project, we will be using a Raspberry Pi Pico W which is ideal for IoT devices as it can connect to a network through WiFi and is small enough to fit into any device. As this project is only a prototype, we will be assembling all the components to a breadboard for easy demonstration.
Firstly, I will be connecting the RGB LED to the Pico (diagram below) which will flash the color of the buttons when they’re pressed.
Next, we will be connecting the Buttons which the user can press to leave their satisfaction rating. We will be using green, yellow and red buttons which will allow for a good, average and bad rating. These will be connected like the diagram below, for all three buttons.
Finally, we can connect the sensors. In this example we will be using the DHT11 temperature and humidity sensor. This will be connected like this:
The final device should look like this.
All the components are connected so now let’s take a look at the code.
How to send and view the Snowplow events
In this section, we will explain how to use Snowplow by integrating with a Snowplow collector, allowing us to track custom events and entities with JSON schemas.
I will be using three schemas: satisfaction, sensors, and weather. JSON Schemas are an important feature of Snowplow which allow you to create and track your own data structure; giving you the flexibility and control of what each data point should look like. Snowplow then validates the data stream against your schemas automatically. This ensures that all the data that ends up in your data warehouse is clean, high quality and ready for analysis. The Webhook is also great as you can create fast, small data points for simple IoT devices.
Unsurprisingly, the satisfaction schema will take the satisfaction rating, the sensor schema will take the sensor readings and the weather schema will take the weather information from the weather API. We have implemented the geo-location and weather API on-device in this project but Snowplow can also do that for you using enrichments! This is great to reduce the bandwidth required for the IoT device. The schemas for this project are available in the related Github Repository.
# function to send a POST request to the snowplow collector
def event(satisfaction, temperature, humidity, time, weather):
post_data = ujson.dumps({
"schema": "iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4",
"data": [
{
"e": "ue",
"tna": "snplow5",
"tv": "upython-0.0.1",
"aid": "satisfaction-meter",
"p": "iot",
"dtm": str(time),
"stm": str(time),
"ue_pr": ujson.dumps({
"schema": "iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0",
"data": {
"schema": "iglu:com.myvendor/satisfaction/jsonschema/1-0-2",
"data": {
"satisfaction_rating": satisfaction
}
}
}),
"co": ujson.dumps({
"schema": "iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0",
"data": [
{
"schema": "iglu:com.myvendor/weather/jsonschema/1-0-1",
"data": {
"city": weather[0],
"temperature_celsius": weather[1],
"windspeed": weather[2],
"humidity": weather[3],
"pressure": weather[4],
"conditions": weather[5]
}
},
{
"schema": "iglu:com.myvendor/sensors/jsonschema/2-0-0",
"data": {
"temperature": temperature,
"humidity": humidity
}
}
]
})
}
]
})
request_url = "http://"+collector+"/com.snowplowanalytics.snowplow/tp2"
res = requests.post(request_url, headers = {'content-type': 'application/json'}, data = post_data)
print(res.status_code)
As you can see, sending events through the tracker protocol endpoint is pretty simple, fill in your JSON schemas with the data and send the event through to your collector. The data can then be viewed in your database, we’ll be using Postgres to query the data like this:
SELECT app_id, events.collector_tstamp, sat.satisfaction_rating, sensors.temperature, sensors.humidity,
weather.temperature_celsius, weather.windspeed, weather.humidity, weather.pressure, weather.conditions
FROM atomic.events
LEFT JOIN atomic.com_myvendor_satisfaction_1 AS sat ON events.event_id = sat.root_id
LEFT JOIN atomic.com_myvendor_sensors_2 AS sensors ON events.event_id = sensors.root_id
LEFT JOIN atomic.com_myvendor_weather_1 AS weather ON events.event_id = weather.root_id
WHERE events.collector_tstamp >= '2022-08-02T00:00:00Z'
ORDER BY events.collector_tstamp DESCRandomly clicking the buttons a few times for some test data produces a table like this:
The code
In this section, we will explore the code of the project for those who are interested.
If you connect your Raspberry Pi in the way described in the earlier diagrams above (See schematic) then you will need to initialize the pins like this.
# device initialisation
# RGB LED
redpin = PWM(Pin(13))
greenpin = PWM(Pin(12))
bluepin = PWM(Pin(11))
freq_num = 10000
redpin.freq(freq_num)
greenpin.freq(freq_num)
bluepin.freq(freq_num)
# temp and humidity sensor
temperature = 0
humidity = 0
dht = dht11.DHT11(15)
time.sleep(1)
# buttons
redbutton = Pin(7, Pin.IN, Pin.PULL_UP)
yellowbutton = Pin(8, Pin.IN, Pin.PULL_UP)
greenbutton = Pin(9, Pin.IN, Pin.PULL_UP)RGB LED to specific colors, where 0 means ON and 65535 means OFF (very counterintuitive!).
# functions to turn RGB LED specific colors
def setRed():
redpin.duty_u16(0)
greenpin.duty_u16(65535)
bluepin.duty_u16(65535)
def setGreen():
redpin.duty_u16(65535)
greenpin.duty_u16(0)
bluepin.duty_u16(65535)
def setYellow():
redpin.duty_u16(0)
greenpin.duty_u16(0)
bluepin.duty_u16(65535)
def setOff():
redpin.duty_u16(65535)
greenpin.duty_u16(65535)
bluepin.duty_u16(65535)Next are the functions to make the LED blink when the corresponding button is pressed.
# functions to flash RGB LED to certain colors
def redButtonClick():
for i in range(3):
setRed()
time.sleep(0.5)
setOff()
time.sleep(0.5)
def yellowButtonClick():
for i in range(3):
setYellow()
time.sleep(0.5)
setOff()
time.sleep(0.5)
def greenButtonClick():
for i in range(3):
setGreen()
time.sleep(0.5)
setOff()
time.sleep(0.5)Next are all the interesting functions which return the data that we are interested in such as the sensor and API functions.
This function reads the temperature and humidity from the DHT11 sensor and returns the values.
# function to read from sensors and return the readings
def sensorRead():
# temp and humidity sensor - DHT11 sensor
measurement = dht.measure()
while measurement == 0:
print("DHT data error")
time.sleep(1)
measurement = dht.measure()
temperature = dht.temperature()
humidity = dht.humidity()
print("temperature: %0.2fC humidity: %0.2f"%(temperature, humidity) + "%")
return temperature, humidityThe following two functions work together to find the geo-location of the device, then with the geo-location find the weather information in the area. I’ve manually done this using APIs which increase traffic to my device but Snowplow can do this using enrichments which would be a better idea to reduce the complexity and traffic to the small IoT device.
# gets the location using an API and returns the city
def getLocation():
API_KEY = "b7f4bf8ea2406b68a8a39ad45ebfbc6e"
baseURL = "http://api.ipstack.com/check?access_key="
url = baseURL + API_KEY
res = requests.get(url).json()
return res["city"]# gets the location using an API and returns the city
# gets weather from API and returns data
def getWeather(city):
baseURL = "http://api.openweathermap.org/data/2.5/weather?"
API_KEY = "ccef0e85f58e1c3617d949fd57685074"
url = baseURL + "appid=" + API_KEY + "&q=" + city
res = requests.get(url).json()
city = res["name"]
temperature = float( res["main"]["temp"] ) - 273.15
windspeed = float( res["wind"]["speed"] )
humidity = int( res["main"]["humidity"] )
pressure = int( res["main"]["pressure"] )
conditions = res["weather"][0]["description"]
print(city, temperature, windspeed, humidity, pressure, conditions)
return [city, temperature, windspeed, humidity, pressure, conditions]
The event function is the main function in this whole program which allows us to send the Snowplow event to the collector URL with all the data accumulated from all the other functions as entities. The way we’ve laid out the JSON is by following the schemas, for more about making your own schemas look at our design tracking documentation.
# function to send a POST request to the snowplow collector
def event(satisfaction, temperature, humidity, time, weather):
post_data = ujson.dumps({
"schema": "iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4",
"data": [
{
"e": "ue",
"tna": "snplow5",
"tv": "upython-0.0.1",
"aid": "satisfaction-meter",
"p": "iot",
"dtm": str(time),
"stm": str(time),
"ue_pr": ujson.dumps({
"schema": "iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0",
"data": {
"schema": "iglu:com.myvendor/satisfaction/jsonschema/1-0-2",
"data": {
"satisfaction_rating": satisfaction
}
}
}),
"co": ujson.dumps({
"schema": "iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0",
"data": [
{
"schema": "iglu:com.myvendor/weather/jsonschema/1-0-1",
"data": {
"city": weather[0],
"temperature_celsius": weather[1],
"windspeed": weather[2],
"humidity": weather[3],
"pressure": weather[4],
"conditions": weather[5]
}
},
{
"schema": "iglu:com.myvendor/sensors/jsonschema/2-0-0",
"data": {
"temperature": temperature,
"humidity": humidity
}
}
]
})
}
]
})
request_url = "http://"+collector+"/com.snowplowanalytics.snowplow/tp2"
res = requests.post(request_url, headers = {'content-type': 'application/json'}, data = post_data)
print(res.status_code)
This is the event loop in which the Raspberry Pi looks for inputs from the buttons. Once a button is clicked, all the functions collect the data and feed it into the event function which sends an event to my collector.
# event loop which checks if buttons have been pressed
while True:
if not redbutton.value():
redButtonClick()
# sensor and API data collection
temperature, humidity = sensorRead()
time = getTime()
city = getLocation()
weather = getWeather(city)
# send event
event('bad', temperature, humidity, time, weather)
elif not yellowbutton.value():
yellowButtonClick()
temperature, humidity = sensorRead()
time = getTime()
city = getLocation()
weather = getWeather(city)
event('average', temperature, humidity, time, weather)
elif not greenbutton.value():
greenButtonClick()
temperature, humidity = sensorRead()
time = getTime()
city = getLocation()
weather = getWeather(city)
event('good', temperature, humidity, time, weather)
With all this code in place and setting up your Raspberry Pi Pico W as we have you will be able to track behavioral data from your very own IoT device! If you’d like to look at the full code, check out our GitHub Repository.
Bryan wrote this blog post as part of a 6 week summer internship at Snowplow, if you would like to be part of Snowplow, take a look at our careers page.
More about
the author
