@alex @mike can you please help me what bootstrap steps is the job trying to invoke. Is this can be a region issue. In EMR console, below is the bootstrap step which is failing.
Location: s3://snowplow-hosted-assets-ap-south-1/common/emr/snowplow-ami4-bootstrap-0.2.0.sh Arguments:1.5
Error receiving
./snowplow-emr-etl-runner --config ~/config.yml --resolver ~/snowplow-master/3-enrich/config/iglu_resolver.json --skip elasticsearch,staging
D, [2016-09-29T12:53:35.808000 #3362] DEBUG -- : Initializing EMR jobflow
D, [2016-09-29T12:53:40.264000 #3362] DEBUG -- : EMR jobflow j-WO53WX7OSZEI started, waiting for jobflow to complete...
W, [2016-09-29T12:55:41.747000 #3362] WARN -- : Job failed. 2 tries left...
W, [2016-09-29T12:55:41.748000 #3362] WARN -- : Bootstrap failure detected, retrying in 54 seconds...
D, [2016-09-29T12:56:35.756000 #3362] DEBUG -- : Initializing EMR jobflow
D, [2016-09-29T12:56:38.446000 #3362] DEBUG -- : EMR jobflow j-8MQ3W0PCDX8H started, waiting for jobflow to complete...
W, [2016-09-29T12:58:39.804000 #3362] WARN -- : Job failed. 1 tries left...
W, [2016-09-29T12:58:39.805000 #3362] WARN -- : Bootstrap failure detected, retrying in 250 seconds...
D, [2016-09-29T13:02:49.810000 #3362] DEBUG -- : Initializing EMR jobflow
D, [2016-09-29T13:02:52.258000 #3362] DEBUG -- : EMR jobflow j-2VE9KE7D9PZS5 started, waiting for jobflow to complete...
W, [2016-09-29T13:04:53.741000 #3362] WARN -- : Job failed. 0 tries left...
F, [2016-09-29T13:04:53.742000 #3362] FATAL -- :
Snowplow::EmrEtlRunner::BootstrapFailureError (EMR jobflow j-2VE9KE7D9PZS5 failed, check Amazon EMR console and Hadoop logs for details (help: https://github.com/snowplow/snowplow/wiki/Troubleshooting-jobs-on-Elastic-MapReduce). Data files not archived.
Snowplow ETL: TERMINATING [BOOTSTRAP_FAILURE] ~ elapsed time n/a [ - ]
- 1. Elasticity S3DistCp Step: Shredded HDFS -> S3: CANCELLED ~ elapsed time n/a [ - ]
- 2. Elasticity Scalding Step: Shred Enriched Events: CANCELLED ~ elapsed time n/a [ - ]
- 3. Elasticity S3DistCp Step: Enriched HDFS _SUCCESS -> S3: CANCELLED ~ elapsed time n/a [ - ]
- 4. Elasticity S3DistCp Step: Enriched HDFS -> S3: CANCELLED ~ elapsed time n/a [ - ]
- 5. Elasticity Scalding Step: Enrich Raw Events: CANCELLED ~ elapsed time n/a [ - ]):
uri:classloader:/emr-etl-runner/lib/snowplow-emr-etl-runner/emr_job.rb:469:in `run'
uri:classloader:/gems/contracts-0.11.0/lib/contracts/method_reference.rb:43:in `send_to'
uri:classloader:/gems/contracts-0.11.0/lib/contracts/call_with.rb:76:in `call_with'
uri:classloader:/gems/contracts-0.11.0/lib/contracts/method_handler.rb:138:in `block in redefine_method'
uri:classloader:/emr-etl-runner/lib/snowplow-emr-etl-runner/runner.rb:68:in `run'
uri:classloader:/gems/contracts-0.11.0/lib/contracts/method_reference.rb:43:in `send_to'
uri:classloader:/gems/contracts-0.11.0/lib/contracts/call_with.rb:76:in `call_with'
uri:classloader:/gems/contracts-0.11.0/lib/contracts/method_handler.rb:138:in `block in redefine_method'
uri:classloader:/emr-etl-runner/bin/snowplow-emr-etl-runner:39:in `<main>'
org/jruby/RubyKernel.java:973:in `load'
uri:classloader:/META-INF/main.rb:1:in `<main>'
org/jruby/RubyKernel.java:955:in `require'
uri:classloader:/META-INF/main.rb:1:in `(root)'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1:in `<main>'
Thanks in advance.