Request a Demo

Cuaght Exception for InitializeTask on Enrichment and Loader

For engineers Enrichment
#1

Hey,

Yesterday, I started receiving the following error messages on the enrichment logs. Apparently both the enrichment and the loader module have problems writing or reading into the respective dynamodb tables.

I am using snowplow/stream-enrich-kinesis:1.4.0
Error log enrichment:

Bildschirmfoto 2020-11-06 um 18.36.36
Bildschirmfoto 2020-11-06 um 18.36.361672×367 110 KB

I am using snowplow/snowplow-s3-loader:0.7.0:
Error log loader:

Bildschirmfoto 2020-11-06 um 18.37.53
Bildschirmfoto 2020-11-06 um 18.37.531671×759 222 KB

First, I thought that it is a problem of access and that I did not set the permissions correctly. I checked that and it seems to be right. In order to test that. I removed the leaseKey entries in the dynamodb tables and the error disappeared on the subsequent test and the data could pass the pipeline again.

However, after some time the error appears again. Deleting the leaseKeys is not a permanent solution.
What are they used for and how does that problem come about? More importantly how can I solve it?

Note, I have not created the dynamodb tables myself. They were created by the modules on the fly (as it is recommended in the documentation)

Cheers!

#2

Hi @mgloel,

We’ve seen similar behavior previously when runtime exceptions were happenning inside enrichment code. But now they all should be get caught. Any chance you can configure Sentry to trace it?

#3

Hey Anton, thanks for your quick reply. How can I set up Sentry? Is it a service within AWS?

#4

Hi @mgloel,

Sentry is a service to track all the errors that are happening accross your components : https://sentry.io

But I’m afraid that in your case it won’t give you more info that what you already have in the logs.

This is surprising that both Stream Enrich and S3 loader fail with Kinesis errors. Have you checked that your Redshift quotas (both for writing and reading) are not reached ?

1 Like