Stream Collector 2.3.0 released

We have released version 2.3.0 of the Snowplow Stream Collector. The main feature of this release is the addition of a new health endpoint, which can be used to monitor the status of the sinks to which the collector is trying to write events.

Users of the stream collector already had a health endpoint that could be polled like so:

$ curl -X GET mycollector/health

Similarly, the new sink health endpoint can be monitored via:

$ curl -X GET mycollector/sink-health

If the collector is itself healthy (returning 200 on the health endpoint), there are two possible responses from the sink-health endpoint. If the collector is not detecting any issues with the sinks, then you can expect a response of 200 OK. If, on the other hand, there is a suspected outage, the response will be 503 Service Unavailable.

This new health endpoint is meant to provide an alarm trigger in case of cloud outages (eg Kinesis or Pub/Sub being unavailable). By contrast, non-cloud sinks, such as NSQ, Kafka and stdout will always return a 200 response (as long as the collector itself is healthy and able to respond).

Outage detection with surge protection on

If you are running a Kinesis collector, with an SQS buffer for surge protection, then the status of the Kinesis stream to which the collector is trying to write will not be considered when deciding whether there is a suspected outage. As long as the SQS buffer is available, the sink-health endpoint will respond with 200 even if Kinesis is having issues.

JAR files

From this release, the assets built under this project will also be available as jar files attached to the Github release.

For a full list of changes, please refer to the release notes:

5 Likes