I’m getting the following error when running the StorageLoader on custom unstructured events:
Unexpected error: Cannot find JSON Paths file to load s3://snowplow-etl-data/shredded/good/run=2016-07-25-15-13-02/com.mycompany/gb-events-context/jsonschema/1- into atomic.com_mycompany_gb_events_context_1
However, I have already uploaded a JSON path file generated by SchemaGuru and named “gb-events-context_1” to an s3 bucket within a “com.mycompany” folder whose path is specified in the configuration file as follows:
aws: # Credentials can be hardcoded or set in environment variables access_key_id: *** secret_access_key: *** s3: region: us-east-1 buckets: assets: s3://snowplow-hosted-assets # DO NOT CHANGE unless you are hosting the jarfiles etc yourself in your own bucket jsonpath_assets: s3://my-schemas-bucket/jsonpaths log: s3n://my-emr-bucket/logs raw: in: # Multiple in buckets are permitted - s3n://my-in-bucket # e.g. s3://my-in-bucket processing: s3n://my-emr-bucket/processing archive: s3n://my-archive-bucket/raw # e.g. s3://my-archive-bucket/in enriched: good: s3://my-out-bucket/enriched/good # e.g. s3://my-out-bucket/enriched/good bad: s3://my-out-bucket/enriched/bad # e.g. s3://my-out-bucket/enriched/bad errors: s3://my-out-bucket/enriched/errors # Leave blank unless continue_on_unexpected_error: set to true below archive: s3://my-out-bucket/enriched/archive # Where to archive enriched events to, e.g. s3://my-archive-bucket/enriched shredded: good: s3://my-out-bucket/shredded/good # e.g. s3://my-out-bucket/shredded/good bad: s3://my-out-bucket/shredded/bad # e.g. s3://my-out-bucket/shredded/bad errors: # Leave blank unless continue_on_unexpected_error: set to true below archive: s3://my-out-bucket/shredded/archive # Where to archive shredded events to, e.g. s3://my-archive-bucket/shredded
I can’t seem to figure out why the storage loader can’t find the paths file in the s3 bucket when everything has been specified. Is there something else that could be causing the problem?
Thanks in advance,