In my workflow I don’t use neither Postgres nor Redshift as storage target. I just download the enriched events from s3 and use the python sdk kit to work with them.
So it’s perfectly ok if the emr-etl-runner ended right after enriching and skipped the shredding + everything after altogether. Is this possible? Running it with
--skip shred option results in error
No run folders in [s3n://splw-company-out/shredded/good/] found