I’m Lucas Souza, CTO of GetNinjas.
Currently, we are using a complex architecture in order to avoid the use of snowplow ruby tracker with the async emitter. So why do we do that?
Basically, because we have a Rails application running on a Unicorn web server. Since Unicorn works with different process and those can be killed when hitting a giving timeout, we are afraid to lose some events.
Our current architecture involves sending events to a file, a process that reads that file using Fluentd, sends those events to SQS and finally getting them and sending to Snowplow using a multi-threaded application written in Ruby.
It’s important to say real-time events are a prerequisite for us. We are not reaching it with this architecture (with a lot of steps), and actually, I think are too many steps in reality.
We would like to know if one of you have any better idea to solve that problem? Did you try Ruby Async Emitter with Rails Application running under Unicorn? Or do you have another architecture in mind to solve this real-time event for backend applications problem?