X-Pack Security

For engineers Storage targets
1

I’m trying to use xpack security to secure my elasticsearch instance to control authentication. Where can I find the setting for input username password for my elasticsearch?

Thank you…

2

Perhaps try the Elasticsearch forums?

1 Like
3

I see. So I take that snowplow loader doesn’t have any authentication method for elasticsearch (for non AWS service). I’ll guess I’ll be going to elasticsearch forums then.

Thank you

4

It doesn’t look like the loader currently supports it (though it does support AWS request signing).

I don’t know enough about how Elasticsearch security works in 6.0 but if the authentication is just Basic Authentication then this should be reasonably easy to add to the ElasticsearchSenderHTTP class (not too sure about over TCP).

6

I thinks it’s the same like previous version, they use basic authentication with http, or token that can be generated using base64 encoder.

I’ll check that class then.

Thank you for your help

7

Hey Hamrio we’ve found that with old versions (the ones named elasticsearch-kinesis-sink) you could just set the cluster endpoint to be https://user:password@host and it’ll work fine. However with the newer elasticsearch-loader this doesn’t seem to work anymore, I believe because of the upgrade to elastic4s which doesn’t support passing basic auth credentials like that with their default ElasticClient, see https://github.com/sksamuel/elastic4s#x-pack-security

8

Hi asabourin,

Until current elasticsearch, I think it still support the basic authentication as mention in this document

Xpack Security Http Client

I already tried to used curl and it still works, wether using user:password@host or using parameter. But when I tried to use it on cluster endpoint, it doesn’t work. That’s why I am asking this question. Currently I just disable my xpack security to let snowplow works until I got answer from elastic team how to bypass the security

Thank you

11

I think supporting X-Pack for both tcp and http is a valid use case, I’ve logged an issue accordingly: https://github.com/snowplow/snowplow-elasticsearch-loader/issues/74

1 Like
12

thank you guys a lot for all the links and all the information i found here. it seems to be super helpful and that’s something that i was exactly searching for as i also want the x pack security. but i wanted to ask you, may i come back if i am going to have questions during the process? thanks a lot!