so this is the second time this has happened and very hard to get snow plow ETL going again. We found out why it happened and will make sure that other process doesn’t run at the same time as load. Last time this happened, i tried running StorageLoader (we are on R90 of course) and it would complete the load with no errors. When i tried running the whole process, it would keep complaining about StorageLoader wanting to be run again. I tried all the usual flag combos (–skip staging etc) and nothing got it going again. i had to resort to cleaning up enriched/shredded folders and have it re-process the raw logs again. but the records still got loaded into redshift but with unique event ids and unique event finger print ids. so makes it hard to de-duplicate. anyone have any suggestions on the cleanest way to get this going again without doing what i had to do above? Thanks everyone.
ERROR: Data loading error Amazon Invalid operation: 1023
Serializable isolation violation on table - 4237826, transactions forming the cycle are: 41525643, 41526274 (pid:10734);
Following steps completed: [Discover]
INFO: Logs successfully dumped to S3 [s3://ga-snowplow-production/snowplow-log/rdb-loader/2017-09-28-14-07-04/5c29862e-ee7c-45c6-90ef-38d78eed49f6]
This article was very helpful of course to find the culprit and why it happened: