Hello,
I have an Elasticsearch Loader for consuming enriched events and send them to AWS Elasticsearch. But the Elasticsearch Loader task stops working after a while (sometimes after 8mins, sometimes after 5hrs, etc.). The maxTimeout is set to 20000 ms and the error is:
I’m using this docker image for ES loader: https://bintray.com/snowplow/registry/snowplow%3Aelasticsearch-loader/1.0.0
I have no idea why it fails. Would be great to know your opinions.
Thanks!
It sounds like your ES cluster may be intermittently unavailable or you may be running into network issues within the stream loader. How are you running Elasticsearch and connecting to ES?
Thanks for your reply @mike .
I created the ecs task of Elasticsearch loader using terraform. So when it’s deployed, a task runs and after a while, it stops and another task starts to run and the same happens again.
The ES is also deployed with terraform as an aws_elasticsearch_domain resource.
The connection is created by setting the ELASTICSEARCH_ENDPOINT as my ES endpoint in the docker image config of ES loader.
