We are using R35 RDBLoader and we have the following questions:
In our CICD we update the RDBLoader image on every deployment which enforces a new task definition on ECS Fargate. By that the current Fargate task will be stopped and a new one would launch. Now the question is whether killing the old RDBLoader task leads to a data loss on Redshift, if there are still unprocessed sqs messages on the queue?
We are already facing the situation that some runs on the shredded-bucket are not loaded to Redshift and we are considering writing a script that scans the shredded-bucket to find the runs which are not loaded yet and put their
shredding_complete.jsonon sqs. Is there any alternative solution to this approach that you may suggest?