Tracking clicks on email clicks - using unstructured event


#1

Hi there!

I am building a link click tracker for email clicks and have been using the excellent guides written by Yali here and here. But I was having a problem setting up an unstructured event for capturing the event data and am hoping someone can help.

Following the guidelines I have attached the e=ue&ue_pr= query parameters. When clicking, the links redirect to the final link and specificed in the ?u= but the event is not getting captured at all. Here’s an example of the link:

https://collector.company.com/r/tp2?u=https%3A%2F%2Fwww.nike.com%2F&e=ue&ue_pr=%7B%22schema%22%3A%20%22iglu%3Acom.snowplowanalytics.snowplow%2Funstruct_event%2Fjsonschema%2F1-0-0%22%2C%22data%22%3A%20%7B%22schema%22%3A%20%22iglu%3Acom.company.iglu%2Flink_click%2Fjsonschema%2F1-0-0%22%2C%22data%22%3A%20%7B%22linkType%22%3A%22goods%22%2C%2C%22linkUrl%22%3A%22https%3A%2F%2Fwww.nike.com%2F%22%2C%22subscriberEmail%22%3A%22test%40company.com%22%7D%7D%7D

Here’s the decoded version of the params:

u=https://www.nike.com/
e=ue
ue_pr={"schema": "iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0",
"data": {"schema": "iglu:com.company.iglu/link_click/jsonschema/1-0-0",
      "data":{"linkType":"goods",
      "linkUrl":"https://www.nike.com/",
      "subscriberEmail":"test@company.com"}
    }
}

Is there something incorrect with the structure of the link above? Or am I missing any additional params? Please help!!

EDIT 1: Initially I had tested this with the standard url_direct event and when doing that, data got tracked and captured in the database correctly! Hence I think its something to do with the url formatting for the UE schema thats preventing the correct analysis and storing of the event.

Thanks very much!


#2

@kjain,

When I decoded your ue_pr value it appears to be an incorrect JSON - it contains extra comma "linkType": "goods",,

{
  "schema": "iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0",
  "data": {
    "schema": "iglu:com.company.iglu/link_click/jsonschema/1-0-0",
    "data": {
      "linkType": "goods",,
      "linkUrl": "https://www.nike.com/",
      "subscriberEmail": "test@company.com"
    }
  }
}

#3

Hi Ihor

Thanks for mentioning that - I had actually already fixed that in my code but had failed to update my post accordingly! So even with the fixed JSON, I am still not seeing the data being processed. Is there a way to debug the EMR process ? Perhaps that can shed some light on where the ETL is failing…

Thanks!


#4

@kjain,

I’m assuming you are running batch pipeline. If so, the events are likely to be rejected at enrichment due to not validating against your JSON schema. After all, it’s a custom event and you might not have implemented it right. The rejected events would end up in your “bad” bucket.

We recommend using Athena to debug the “bad” events as per Debugging bad rows in Athena [tutorial]. If it’s too complicated you can even examine your “bad” bucket manually but it’s more tedious.


#5

Have you tested this with Snowplow Mini? It’s quick to setup and is typically easiest to test this kind of stuff (new schemas) with a reasonable turn around time. If the data is going into bad it will also give you an error message to help you debug.

Another question - does the iglu:com.company.iglu/link_click/jsonschema/1-0-0 schema exist in your Iglu schema repository? If not that could be a reason why it’s failing.


#6

Thanks for your replies!

I will try with Athena, but is there a guide for manually debugging the bad events data (i am using the scala - kinesis - lzo batch pipeline)?

@mike - yes - I do have the schema and jsonpaths uploaded to the iglu repository. I have other custom unstructured events and all of them work perfectly.

Also, just wanted to make sure that the link url structure I am using is following the correct syntax?

Thanks!


#7

Hi Ihor

Thanks for that GREAT Athena debugging process! By checking the bad logs, I was able to figure out the problem (an error in one of the event parameters). Got that fixed and the event is capturing now!

Thanks!


#8

Now that I can see data thats being captured, I am noticing that a lot of data that the original url_redirect event tracks doesn’t get captured with a custom event eg. page_url, page_urlhost, page_urlpath etc. These could be useful to track the referrer for the email (and hence detect if the link click is first party or 2nd/3rd party ie forwards). So is it possible at all to be able track such data in link clicks through unstructured events at all?

Also, I see that the platform is set to ‘web’. Should this not be ‘email’ ?

Thanks!


#9

@kjain,

Your custom data is loaded into a dedicated table. The “original” url_redirect data has also its dedicated table as it’s implemented (under the hood) as unstructured (self-describing) event too.

You normally would have to join the custom/dedicated table with its parent events table by means of a relation like event_id = root_id and collector_tstamp = root_tstamp where event_id and collector_tstamp are from events and the root_id and root_tstamp are from the custom/dedicated table.


#10

Thanks Ihor! That helped…

One additional question: I was using iglu webhooks to generate pixel tracking events as well. I was wondering if its possible to use webhooks with link tracking unstructured events as well? It would simplify the link building considerably and provide some abstraction of the data schema.

Thanks!


#11

Sure. The wiki page shows you how to do it (unless I misunderstood you).


#12

Thanks for your reply Ihor!

I had used that wiki page to build the webhook for pixel tracking. But I was wondering how I could use to build it for redirection tracking. The issue is that the uri)redirect uses a special url (r/tp2?u=) to receive and then redirect. So how how can I integrate a webhook into this?

Thanks!


#13

@kjain,

I’m afraid you cannot do that. You cannot send a request to two endpoints at the same time. Potentially, you could replace the value for u parameter with Iglu adapter URI. However, you will not be redirected to the intended resource afterwards.

If it’s a Javascript enabled application you could probably do an AJAX call to send link_click to Iglu webhook as you already did and then do a redirect to the intended resource. Though, you would have to have some means of tieing these 2 separate events together during your data modeling stage (by, say, introducing the same ID to both events).

Alternatively, you could turn your custom unstructured link_click event into structured event in which case there’s no need for Iglu adapter at all as the data will be loaded into atomic.events table rather than the dedicated table.