Snowplow 82 Tawny Eagle released

Announcements New releases
1

We’re pleased to announce the release of Snowplow R82 Tawny Eagle!

This release updates the Kinesis Elasticsearch Sink with support for sending events via HTTP, allowing us to now support the Amazon Elasticsearch Service as a storage target.

As always, you can read more on our blog here:

1 Like
2

Thx @josh,

a long awaited feature, which will let us use AWS ES :slight_smile:

Thx a bunch!

3

Hi @josh,

Have just checked http transport protocol with Elasticsearch 2.0, 2.4 and 5. In all the cases, storage process throws “Unexpected Elasticsearch response” exception and commits suicide. Unfortunately…

4

Hi,

As per commit by @josh (hash 10aa3cb5cfc89936ab66cd09f5d964ed42ac4a7c), Elasticsearch 2.3 seems to be supported - just have verified.

Thanks.

5

Hey @grzegorzewald is that a commit within Release 82? As in the HTTP client appears to be supported up to 2.3?

6

Hi @josh,

As far as I see, this commit is within feature/es-2.x-support branch, committed by yourself? Artifact generates elasticsearch sink 0.8 rc1.

I was unable to connect to Elasticsearch 2.x through http with 0.7 binary form bintray. I have not tried to compile sink from r82 with ELASTICSEARCH_VERSION=2x. Would this help? As I am just to test hosted Elasticsearch in production soon and i would like not to stuck/use temporary ES1.5as long as I am not forced…

7

Ahh no worries, as the hash wasn’t hyperlinked was not sure where it was coming from!

There is an RC built for both the 1.x and 2.x versions of the latest Elasticsearch Sinks available from here if you would like to test those:

https://bintray.com/snowplow/snowplow-generic/snowplow-kinesis-elasticsearch-sink/0.8.0-rc1#files

8

Hi,

I must have had some malicious issue previously. Now I am able to connect through HTTP form both: 0.7 and 0.8 rc1.

THis makes some stuff much easier. Many thanks

9

OK, figured it out. So if I use 0.7 and Elasticsearch index does not exists, I get ElasticsearchEmitter threw an unexpected exception, but when I create index before running storage, everything works flawlessly. On 0.8 rc1 there is no such an issue. It seems, that Elasticsearch 2.x has different response when indexing record to non-existing index. Fix: curl -s -XPUT http://$ELASTICSEARCH:$ELASTICSEARCH_PORT/snowplow

10

Ahh that is really interesting to know. In any case after the next snowplow release we will have properly supported 2.x support which should make all of this a lot easier!

1 Like