We’re trying to figure out a good infrastructure for sending server-side events to Snowplow. Our tech team is concerned about sending events directly as the HTTP requests are blocking, and might have an impact on the application itself in case of latency or availablity issue with the collector. As analytics/trackings are orthogonal to the application, it makes sense to avoid this extra point of failure.
This is the infrastructure we’re currently evaluating:
- Application: Write events to a local events.log file
- Fluentd: Send out the events to a SQS queue
- Logstash: Get messages from SQS and send them to the collector, through a plugin that uses the Snowplow gem
It would be nice to hear your feedback about this How have you structured your applications to send data to Snowplow?