Loading JS Tracker enhanced ecommerce events into Redshift

For data modelers & consumers Redshift
1

Hi,
I am using snowplow version 75 and have implemented the javascript Tracker version 2.6.0. I am facing issue for Enhanced Ecomm events while running the storage loader . PFB the error log of storage loader:

Loading Snowplow events and shredded types into My Redshift database (Redshift cluster)…
Unexpected error: Cannot find JSON Paths file to load s3://udmd-p-enriched/shredded/good/run=2016-05-04-05-46-42/com.google.analytics.enhanced-ecommerce/action/jsonschema/1- into landing.com_google_analytics_enhanced_ecommerce_action_1

The Contents of the Iglu resolver is :-

{
  "schema": "iglu:com.snowplowanalytics.iglu/resolver-config/jsonschema/1-0-0",
  "data": {
    "cacheSize": 500,
    "repositories": [
      {
        "name": "Iglu Central",
        "priority": 0,
        "vendorPrefixes": [ "com.snowplowanalytics" ],
        "connection": {
          "http": {
            "uri": "http://iglucentral.com"
          }
        }
      } ,
          {
        "name": "UDMD Repository",
        "priority": 0,
        "vendorPrefixes": [ "udmd.unileversolutions" ],
        "connection": {
          "http": {
            "uri": "http://s3-eu-west-1.amazonaws.com/udmd-p-schemas"
          }
                }
      } ,
          {
        "name": "UDMD Repository",
        "priority": 0,
        "vendorPrefixes": [ "udmd.videocomponent" ],
        "connection": {
          "http": {
            "uri": "http://s3-eu-west-1.amazonaws.com/udmd-p-schemas"
          }
                }
      } ,
      {
        "name": "UDMD Repository",
        "priority": 0,
        "vendorPrefixes": [ "udmd.product" ],
        "connection": {
          "http": {
            "uri": "http://s3-eu-west-1.amazonaws.com/udmd-p-schemas"
          }

Could you please let me know where do I need to mention the JSOn path files for Enhanced Ecommerce.

Thanks,
Shilpi Singh

2

Hi @ssingh195,

Please, review the following wiki page which should answer your question: https://github.com/snowplow/snowplow/wiki/4-Loading-shredded-types

In short, the JSONPaths serves as a mapping means to let Redshift know which values from JSON file in shredded/good to go to which field in the corresponding table. The file itself has to be generated/created as a reflection of the DDL command used to create the table. it needs to be placed in a private dedicated S3 bucket and referenced with jsonpath_assets key of your config.yml configuration file.

On a general note, you can read the Amazon doc page explaining the concept of JSONPaths file and the way COPY from JSON command works. It is the command used to load custom events and contexts, which are available in JSON format post shredding process.

Regards,
Ihor

3

Hi @ssingh195 - following on from Ihor’s post, could you let us know the values you have set here in your config.yml:

  s3:
    region: ???
    buckets:
      assets: ???

Thanks!

4

Hi Alex,

The contents of config.yml are -

 s3:
    region: eu-west-1
    buckets:
      assets: s3://snowplow-hosted-assets # DO NOT CHANGE unless you are hosting the jarfiles etc yourself in your own bucket
      jsonpath_assets: s3://udmd-p-jsonpathfiles/

As per my understanding for Enhanced Ecomm Events , the JSON path should be taken from the hosted assets and I dont have to host it on S3 buckets as done in the case of custom context Table.Please correct me If wrong.

5

Hi @ssingh195 - you are exactly right - you should not have to self-host any JSON Paths relating to events or contexts “bundled” with Snowplow - they should be hosted by us. I have just checked and the enhanced ecommerce JSON Paths files seem to be missing from s3://snowplow-hosted-assets - apologies for the oversight.

I will post back here when this has been fixed.

6

This should be fixed now - can you try again Shilpi?