Events not being created?

1

I’m not sure if the event is not being created or if it’s not reaching the enriched stream.

I created a number of schemas:

[
    "iglu:com.acme/myschema/jsonschema/1-0-0",
    "iglu:com.quix.steve.test/my-really-new-schema/jsonschema/1-0-0",
    "iglu:com.quix.steve.test/my-really-new-schema/jsonschema/1-0-1",
    "iglu:com.quix.steve.test/mouse-move-event/jsonschema/1-0-1",
    "iglu:com.quix.steve.test/mouse-click-event/jsonschema/1-0-0",
    "iglu:com.quix.steve.test/mouse-click-event/jsonschema/1-0-1",
    "iglu:com.quix.steve.test/new-mouse-click-event/jsonschema/1-0-0",
    "iglu:com.acme/new-event-acme/jsonschema/1-0-0",
    "iglu:com.quix.steve.test/abc123/jsonschema/1-0-0"
]

The first 4 seem to work when I send events.
The rest don’t produce any events. I created most of them today, but 1, mouse-click-event v1-0-1 was created a few weeks ago along with the other working ones.

Any clues as to why I can’t generate events on these schemas?

This code:

	mouse_click = {'button': 1}
	snowplow('trackSelfDescribingEvent', {
		  event: {
			schema: 'iglu:com.quix.steve.test/abc123/jsonschema/1-0-1',
			data: JSON.stringify(mouse_click)
		  }
		});

Works for the mouse-move-event schema but not for the mouse-click-event schema.

:face_with_spiral_eyes:

2

If your event is failing validation, it will end up in failed events - you can read about them here: Understanding failed events - Snowplow Docs

There should be an error message giving at least a hint as to the problem.

A good option to debug this kind of thing is to use snowplow micro on your machine. That way you can have a faster feedback loop to iterate tracking and schema definition, before pushing to prod.

3

Thanks for the speedy response.
I looked in sp-bad-1-stream and there is was…

It looks like the schema isn’t valid for a number of reasons (see below). Not sure why. I thought I’d used the exact same method to create the new ones.

This is how I created the schemas… old and new.

curl -XPOST https://xxxxx/api/schemas \
  -H "apikey: xxxxx" \
  -d '
        {
          "$schema": "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#",
          "self": {
            "vendor": "com.quix.steve.test",
            "name": "abc123",
            "format": "jsonschema",
            "version": "1-0-0"
          },
          "type": "object"
        }'


  "schema": "iglu:com.snowplowanalytics.snowplow.badrows/schema_violations/jsonschema/2-0-0",
  "data": {
    "processor": {
      "artifact": "streamCommon",
      "version": "2.0.5"
    },
    "failure": {
      "timestamp": "2022-08-10T12:29:56.231199Z",
      "messages": [
        {
          "schemaKey": "iglu:com.quix.steve.test/abc123/jsonschema/1-0-1",
          "error": {
            "error": "ResolutionError",
            "lookupHistory": [
              {
                "repository": "Iglu Central",
                "errors": [
                  {
                    "error": "NotFound"
                  }
                ],
                "attempts": 8,
                "lastAttempt": "2022-08-10T12:29:56.177Z"
              },
              {
                "repository": "Iglu Central - Mirror 01",
                "errors": [
                  {
                    "error": "NotFound"
                  }
                ],
                "attempts": 8,
                "lastAttempt": "2022-08-10T12:29:56.231Z"
              },
              {
                "repository": "Iglu Client Embedded",
                "errors": [
                  {
                    "error": "NotFound"
                  }
                ],
                "attempts": 1,
                "lastAttempt": "2022-08-10T11:45:34.563Z"
              },
              {
                "repository": "Iglu Server",
                "errors": [
                  {
                    "error": "NotFound"
                  }
                ],
                "attempts": 8,
                "lastAttempt": "2022-08-10T12:29:55.962Z"
              }
            ]
          }
        }
      ]
    },
    "payload": {
      "enriched": {
        "app_id": "my-other-app-id",
        "platform": "web",
        "etl_tstamp": "2022-08-10 12:29:55.936",
        "collector_tstamp": "2022-08-10 12:29:54.090",
        "dvce_created_tstamp": "2022-08-10 12:29:53.822",
        "event": "unstruct",
        "event_id": "fcf2e1ee-8a38-4c9e-b526-bcf89ab47be9",
        "txn_id": null,
        "name_tracker": "sp",
        "v_tracker": "js-3.5.0",
        "v_collector": "snowplow-stream-collector-kinesis-2.4.5-kinesis",
        "v_etl": "streamCommon-2.0.5-common-2.0.5",
        "user_id": null,
        "user_ipaddress": "x.x.x.x",
        "user_fingerprint": null,
        "domain_userid": "13dcae9e-eb28-480b-94db-e4500e896cd3",
        "domain_sessionidx": 5,
        "network_userid": "39b6b714-bbae-49d1-83aa-0e7b8b0b01ce",
        "geo_country": null,
        "geo_region": null,
        "geo_city": null,
        "geo_zipcode": null,
        "geo_latitude": null,
        "geo_longitude": null,
        "geo_region_name": null,
        "ip_isp": null,
        "ip_organization": null,
        "ip_domain": null,
        "ip_netspeed": null,
        "page_url": "http://127.0.0.1:5500/test.html",
        "page_title": null,
        "page_referrer": "http://127.0.0.1:5500/test.html",
        "page_urlscheme": null,
        "page_urlhost": null,
        "page_urlport": null,
        "page_urlpath": null,
        "page_urlquery": null,
        "page_urlfragment": null,
        "refr_urlscheme": null,
        "refr_urlhost": null,
        "refr_urlport": null,
        "refr_urlpath": null,
        "refr_urlquery": null,
        "refr_urlfragment": null,
        "refr_medium": null,
        "refr_source": null,
        "refr_term": null,
        "mkt_medium": null,
        "mkt_source": null,
        "mkt_term": null,
        "mkt_content": null,
        "mkt_campaign": null,
        "contexts": "{\"schema\":\"iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0\",\"data\":[{\"schema\":\"iglu:com.snowplowanalytics.snowplow/web_page/jsonschema/1-0-0\",\"data\":{\"id\":\"3f748e6f-879e-4a6b-a7a8-e6bacd857be6\"}}]}",
        "se_category": null,
        "se_action": null,
        "se_label": null,
        "se_property": null,
        "se_value": null,
        "unstruct_event": "{\"schema\":\"iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0\",\"data\":{\"schema\":\"iglu:com.quix.steve.test/abc123/jsonschema/1-0-1\",\"data\":\"{\\\"button\\\":1}\"}}",
        "tr_orderid": null,
        "tr_affiliation": null,
        "tr_total": null,
        "tr_tax": null,
        "tr_shipping": null,
        "tr_city": null,
        "tr_state": null,
        "tr_country": null,
        "ti_orderid": null,
        "ti_sku": null,
        "ti_name": null,
        "ti_category": null,
        "ti_price": null,
        "ti_quantity": null,
        "pp_xoffset_min": null,
        "pp_xoffset_max": null,
        "pp_yoffset_min": null,
        "pp_yoffset_max": null,
        "useragent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
        "br_name": null,
        "br_family": null,
        "br_version": null,
        "br_type": null,
        "br_renderengine": null,
        "br_lang": "en",
        "br_features_pdf": null,
        "br_features_flash": null,
        "br_features_java": null,
        "br_features_director": null,
        "br_features_quicktime": null,
        "br_features_realplayer": null,
        "br_features_windowsmedia": null,
        "br_features_gears": null,
        "br_features_silverlight": null,
        "br_cookies": 1,
        "br_colordepth": "24",
        "br_viewwidth": 1153,
        "br_viewheight": 1070,
        "os_name": null,
        "os_family": null,
        "os_manufacturer": null,
        "os_timezone": "Europe/London",
        "dvce_type": null,
        "dvce_ismobile": null,
        "dvce_screenwidth": 2560,
        "dvce_screenheight": 1440,
        "doc_charset": "UTF-8",
        "doc_width": 1153,
        "doc_height": 1070,
        "tr_currency": null,
        "tr_total_base": null,
        "tr_tax_base": null,
        "tr_shipping_base": null,
        "ti_currency": null,
        "ti_price_base": null,
        "base_currency": null,
        "geo_timezone": null,
        "mkt_clickid": null,
        "mkt_network": null,
        "etl_tags": null,
        "dvce_sent_tstamp": "2022-08-10 12:29:53.823",
        "refr_domain_userid": null,
        "refr_dvce_tstamp": null,
        "derived_contexts": null,
        "domain_sessionid": "e5afa9e9-4066-44b4-b38f-ff1d874936bb",
        "derived_tstamp": null,
        "event_vendor": null,
        "event_name": null,
        "event_format": null,
        "event_version": null,
        "event_fingerprint": null,
        "true_tstamp": null
      },
      "raw": {
        "vendor": "com.snowplowanalytics.snowplow",
        "version": "tp2",
        "parameters": [
          {
            "name": "e",
            "value": "ue"
          },
          {
            "name": "duid",
            "value": "13dcae9e-eb28-480b-94db-e4500e896cd3"
          },
          {
            "name": "vid",
            "value": "5"
          },
          {
            "name": "eid",
            "value": "fcf2e1ee-8a38-4c9e-b526-bcf89ab47be9"
          },
          {
            "name": "url",
            "value": "http://127.0.0.1:5500/test.html"
          },
          {
            "name": "refr",
            "value": "http://127.0.0.1:5500/test.html"
          },
          {
            "name": "aid",
            "value": "my-other-app-id"
          },
          {
            "name": "cx",
            "value": "eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uc25vd3Bsb3dhbmFseXRpY3Muc25vd3Bsb3cvd2ViX3BhZ2UvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsiaWQiOiIzZjc0OGU2Zi04NzllLTRhNmItYTdhOC1lNmJhY2Q4NTdiZTYifX1dfQ"
          },
          {
            "name": "tna",
            "value": "sp"
          },
          {
            "name": "cs",
            "value": "UTF-8"
          },
          {
            "name": "cd",
            "value": "24"
          },
          {
            "name": "stm",
            "value": "1660134593823"
          },
          {
            "name": "tz",
            "value": "Europe/London"
          },
          {
            "name": "tv",
            "value": "js-3.5.0"
          },
          {
            "name": "vp",
            "value": "1153x1070"
          },
          {
            "name": "ds",
            "value": "1153x1070"
          },
          {
            "name": "res",
            "value": "2560x1440"
          },
          {
            "name": "cookie",
            "value": "1"
          },
          {
            "name": "p",
            "value": "web"
          },
          {
            "name": "dtm",
            "value": "1660134593822"
          },
          {
            "name": "lang",
            "value": "en"
          },
          {
            "name": "ue_px",
            "value": "eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy91bnN0cnVjdF9ldmVudC9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJzY2hlbWEiOiJpZ2x1OmNvbS5xdWl4LnN0ZXZlLnRlc3QvYWJjMTIzL2pzb25zY2hlbWEvMS0wLTEiLCJkYXRhIjoie1wiYnV0dG9uXCI6MX0ifX0"
          },
          {
            "name": "sid",
            "value": "e5afa9e9-4066-44b4-b38f-ff1d874936bb"
          }
        ],
        "contentType": "application/json",
        "loaderName": "snowplow-stream-collector-kinesis-2.4.5-kinesis",
        "encoding": "UTF-8",
        "hostname": "xxxxxxx",
        "timestamp": "2022-08-10T12:29:54.090Z",
        "ipAddress": "x.x.x.x",
        "useragent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
        "refererUri": "http://127.0.0.1:5500/",
        "headers": [
          "Timeout-Access: <function1>",
          "X-Forwarded-For: 2.27.203.27",
          "X-Forwarded-Proto: http",
          "X-Forwarded-Port: 80",
          "Host: xxxxxxxx",
          "X-Amzn-Trace-Id: Root=1-62f3a4c2-31606bc24d41d918436f0052",
          "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
          "Accept: */*",
          "Origin: http://127.0.0.1:5500",
          "Referer: http://127.0.0.1:5500/",
          "Accept-Encoding: gzip, deflate",
          "Accept-Language: en",
          "application/json"
        ],
        "userId": "39b6b714-bbae-49d1-83aa-0e7b8b0b01ce"
      }
    }
  }
}
4

So the ‘not found’ response is a list of places that the pipeline looked for the schema - and it wasn’t there in any of them.

So in the genral case that means either it was never uploaded, or there’s a mistake in the path provided.

From a quick check of what you’ve provided above, it seems you may have uploaded 1-0-0 and tracked against 1-0-1. But if it’s not that, I’d check that the schema was uploaded, and double-check the schema path provided to the tracker. :slight_smile:

5

OK! Thanks you put me on the right track.

It seems that I created the mouse-move-event with type = “string” but I created the mouse-click-event with type= object… then tried to send it a string.

Without looking in the errors stream (the bad-1-stream) I would never have guess it…

But on the bright side I now have a bunch of Snowplow postman calls setup and I know a bit more about Snowplow in general… plus a bit of a headache.

Thanks for your help!! Really appreciate it.

Steve

1 Like