Introducing Snowly, a local Snowplow request validator


#1

Hi guys,

Before Snowplow Mini was released I worked on a Snowplow project and realized how intensive its setup can be. That project had a lot of custom contexts and unstructured events and the schemas were evolving really fast.

In order to avoid going through the whole ETL process to validate the requests, I created a minimal Collector implementation, designed to run locally and with some nice validation extensions. Even with Mini’s release, I still think it can deliver a faster debug cycle, although it is completely limited to that purpose.

The core concept is mapping everything to JSON schemas, even the Snowplow’s protocol. It allows customizations to bring the requirements closer to the domain model. For instance, you can define more specific restrictions like a list of permitted values for structured event actions. You can even define requirements based on the content of another field: If event action is ‘viewed_product’, event property is required.

This gem is a complete revamp of the first endeavor and hasn’t been tested in the real world, outside of automated tests. I hope you can use it and I welcome any feedbacks.

Thanks


#2

Hey @angelim - this is awesome, thanks so much for sharing!

It looks great, and a really nice complement to Snowplow Mini. I particularly like the way you’ve come up with a JSON Schema for the Snowplow Tracker Protocol - very nicely done!


#3

Hi @alex,
I’m glad you liked it. Snowplow rules!