What is _temporary folder generated while processing task in EMR?


#1

hi, I have a recovery process that sometimes generates this _temporary files. can anyone please help understand. ( this folder is not just present in the s3 but its also on HDFS , so that means that it was not created by s3-dist-cp ).

  1. why is this folder created
  2. does this mean that the job completed successfully ? but the final step died and was not able to accumulate data processed in different task.
  3. or that the job didn’t complete all the way up till the end.