First, i’m really sorry if this is a dumb question, i’m learning how to deal with snowplow.
I have a snowplow setup that uses the redshift for final storage and any structured and unstructured event is working really fine with it, but now i need to use postgresql for that final storage and the storageloader only load structured events, here are my configs:
./bin/snowplow-emr-etl-runner -c ./config/snowplow-emr-etl-runner.yml -r ./config/resolver.json -n ./enrichments/
./bin/snowplow-storage-loader -c ./config/snowplow-emr-etl-runner.yml
Obs: the same approach work with redshift:
Some other data:
All s3 bucket are created and with the right permissions
The storage loader download all enriched data to tempdir
On the log, no error is issued.
○ → find tempdir/run=2016-07-26-20-19-17/atomic-events/|xargs wc -l
140908 total -> all that events are loaded in atomic.events on postgresql
○ → find tempdir/run=2016-07-26-20-19-17/|grep -v atomic|xargs wc -l
527505 total -> these not
Ps: sorry for the bad english, it’s not my native language.