No data in rds pipline db

1

Hi

I have followed:

And created pipline in AWS. Now when I send test event following this:

But I don’t see any data in the rds db.

Can someone tell me, what I am missing?

2

Hi @achintya.c,

Snowplow pipeline consists of multiple components: collector, enrich, shredder/loader. Also every component can have different variations, for example you can have ElasticSearch, S3 and RDB Loaders. Moreover an event can end up in a “bad bucket” or “bad stream” tied to any of these components.

So, in order to figure out why your event is not ending up in a DB, we need figure out:

  1. What Loader are you using. The only one that can load data into RDS is Postgres Loader. Am I right that it’s the one you deployed?
  2. Where is your event. You need to check every bad and good stream in order to see where it’s stuck. If it’s stuck in a “bad” stream - the payload will have all details on why it happened. If it’s suck in a “good” stream - perhaps a downstream consumer is configured incorrectly.
  3. Is there anything suspicious in logs of your components, e.g. enrich can fail to autheticate

This is a (semi-arbitrary) list of places that I’d check:

  1. collector logs
  2. collector good stream
  3. collector bad stream
  4. enrich logs
  5. enrich good stream
  6. enrich bad stream
  7. (postgres) loader logs
  8. (postgres) loader bad stream
3

anton

Thanks for your reply. I will check all the corresponding resources for good/bad stream.

2 Likes
4

Hi @achintya.c
Welcome to the Snowplow Community!
Cheers,
Eddie