Yesterday, the rdbloader step of our batch pipeline (R92) failed after data discovery and logged the following error message:
ERROR: Data loading error Amazon Error setting/closing connection: Connection reset by peer.
Following steps completed: [Discover]
I was able to load the shredded data by re-runing with the
--resume_from rdbloader option but want to see if there’s something I should be doing to prevent this from happening again.
We’re currently running R92 (i.e.,
rdb_shredder: 0.12.0 and
rdb_loader = 0.13.0). I’ve seen a few references (linked below) to the
60000 error, but am unsure if those are what caused the above error.
We upgraded to R92 several weeks ago, and things have been working well. In addition, the failure we saw yesterday was on a small, weekend volume of data compared to what we normally process for a regular business day.
Is there a way I can definitively tell whether or not our error is related to one of the following?
Thanks for your help!