JS Tracker Edge Analytics: Aggregating Page Ping events client side

Version 2.13.0 of the Snowplow JavaScript Tracker introduces edge analytics within our JavaScript tracker, in particular a new callback function on activity tracker has been introduced so it is now possible to receive the activity tracking (page ping) events in the browser and aggregate them together into a single event.

2.13.0 is available on GitHub releases.

Activity Tracking is a feature of the Snowplow JavaScript tracker, that sends events to a Snowplow collector at specified time intervals. These events can be aggregated in a data modelling step to calculate page activity, from length of time on page to how far the user scrolled through the page.

However, one of the downsides to this technique is the large amount of events that are sent to achieve good levels of accuracy. With version 2.13.0 it is now possible to do this aggregation client side to reduce, or potentially elimate, page ping events. You will need to do some work to aggregate the information and then send this information to the Snowplow collector, below we look at one example of how you might achieve this.

The Snowplow JavaScript tracker now exposes a new function called enableActivityTrackingCallback (See docs). Using this callback means that each time a page ping event would normally be sent, your callback function will be executed with the page activity information instead.

window.snowplow('newTracker', 'sp', '<<collectorUrl>>', {
    appId: 'my-app-id',
    eventMethod: 'beacon',
    contexts: {
        webPage: true,
        performanceTiming: true
    }
});
var aggregatedEvent = {
    pageViewId: null,
    minXOffset: 0,
    maxXOffset: 0,
    minYOffset: 0,
    maxYOffset: 0,
    numEvents: 0
};
window.snowplow('enableActivityTrackingCallback', 10, 10, function (event) {
    aggregatedEvent = {
        pageViewId: event.pageViewId,
        minXOffset: aggregatedEvent.minXOffset < event.minXOffset ? aggregatedEvent.minXOffset : event.minXOffset,
        maxYOffset: aggregatedEvent.maxYOffset > event.maxYOffset ? aggregatedEvent.maxYOffset : event.maxYOffset,
        minYOffset: aggregatedEvent.minYOffset < event.minYOffset ? aggregatedEvent.minYOffset : event.minYOffset,
        maxYOffset: aggregatedEvent.maxYOffset > event.maxYOffset ? aggregatedEvent.maxYOffset : event.maxYOffset,
        numEvents: aggregatedEvent.numEvents + 1
    };
});
window.addEventListener('unload', function() {
    window.snowplow('trackSelfDescribingEvent', {
        schema: 'iglu:com.acme_company/page_unload/jsonschema/1-0-0',
        data: {
            minXOffset: aggregatedEvent.minXOffset,
            maxXOffset: aggregatedEvent.maxXOffset,
            minYOffset: aggregatedEvent.minYOffset,
            maxYOffset: aggregatedEvent.maxYOffset,
            activeSeconds: aggregatedEvent.numEvents * 10
        }
    });
});
window.snowplow('trackPageView');

Note: For this technique of sending on page unload to work reliably, we recommend initialising the Snowplow JavaScript Tracker with eventMethod: 'beacon' and/or stateStorageStrategy: 'cookieAndLocalStorage' (if navigating to a page that also contains the JS Tracker). Using the page unload technique will not work for Single Page Applications (SPA), you would need to send the aggregated event to the Snowplow collector on navigation within your application.

In this example, we have introduced a new event schema called page_unload. This would need to be uploaded to your Iglu repository and will likely need to look something like this:

{
        "$schema": "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#",
        "description": "Schema for a context of an aggregated activity tracking on page unload",
        "self": {
                "vendor": "com.acme_company",
                "name": "page_unload",
                "format": "jsonschema",
                "version": "1-0-0"
        },
        "type": "object",
        "properties": {
                "minXOffset": {
                        "type": "integer",
                        "minimum": 0
                },
                "maxXOffset": {
                        "type": "integer",
                        "minimum": 0
                },
                "minYOffset": {
                        "type": "integer",
                        "minimum": 0
                },
                "maxYOffset": {
                        "type": "integer",
                        "minimum": 0
                },
                "activeSeconds": {
                        "type": "integer",
                        "minimum": 0
                }
        },
        "required": ["minXOffset","maxXOffset","minYOffset","maxYOffset","activeSeconds"],
        "additionalProperties": false
}

If you have any other thoughts about this feature, other edge analytics ideas, or examples of where you have utilised it, we would love to hear from you!

4 Likes

Paul - this is a brilliant idea and a really nice execution. Anything that makes moves towards removing page_pings from events and into it’s own context is a win in my book!

Is it worth publishing a com.snowplowanalytics.snowplow/jsonschema/page_unload/1-0-0 schema in Iglu Central to use by default but allowing users to override this schema in the tracker?

We thought about publishing a new schema but we felt there were a number of different use cases for this new callback concept (on page_unload, on page navigation in a SPA, batching or aggregating page pings and sending on a less frequent timer) that we didn’t feel there was a definitive schema to publish.

In the end leaving this open for users of this functionality to describe their own event types based on their usage of the data in the callback felt like the best approach.

1 Like